The poles move miles per year normally. Its not a big deal for most of us but pilots notice it.
And surveyors have declination tables to precisly adjust past surveys for the wandering poles.