Recent piece on DNA studies of Spanish men. Seems lots and lots of Jewish (and earlier Phoenician) guys stayed behind, as did North Africans, et al.
Genealogists have been saying for years that 20% to 25% of Spanish ancestry (outside of Galicia and the Basque country) is of Jewish origin. These studies suggest 20% is right on target. Another 10% is attributed to North Africans (Berber) and Arab ancestry.
A quote worth remembering: "Studies such as the new one “tell the true history of everyone’s ancestors and not just the history book lessons of kings and queens,” says James Wilson, a population geneticist at the University of Edinburgh in Scotland who was not involved in the study."
No doubt it became popular at some time in Spain to believe that all the Jews and all the Moors "went back where they came from", but they didn't!
Well, the Moors (in terms of the religious group) DID go away. if you mean the racial group, they would of course have mingled — to go back further in history, why couldn’t the Celto-Iberians have been related to the Berbers? there’s only the narrow straits of Hercules separating the two.