I hope you responded to the person who “started” to question your right to park in handicapped spaces something like this: “As it happens, we have the right to use this spot, but on behalf of all handicapped persons, I appreciate your willingness to confront people who appear to be violating our rights.”
Just having them shut up in mid sentence as my obviously disabled friend got out of the car was reward enough. For once in my life I didn’t say anything, or need to. My buddy can barely walk, so I just ignored the person and help him out of the car.
I can’t wrap my head around an able bodied person who would use a handicapped space at all... like people who beat or abuse spouses, kids or pets, just can’t even comprehend. Personally I take the farthest space I can, but I run and lift a lot.