You are ignoring the possibility of convergent evolution. The same environmental pressures that gave the Neanderthals light skin would have been working on Africans as they moved northward. After many generations, pigmentation would have changed.
I find it hard to believe that Europeans became white solely from Neanderthals give that Neanderthal genes account for only a few percent in European genomes.
Yes, you may be right, lactose tolerance mutated only about 8 thousand years ago as steppe people moved into Europe.