I’ve been mistaken for Mexican by both whites and Latinos but I never thought to be offended by it.
Well, there you go - you've experienced racism. It doesn't matter whether you realized it or not - you're a victim, a protected class, and therefore are due special treatment, favoritism and (gasp!) even reparations.
I live near the Mexican border in the city of Chula Vista (near San Diego). I encounter Spanish speakers right and left. Some don’t speak English but I never ask them if they do. I simply say I speak English.