Try following some random young lady around and let me know how that works out for you.
AS I said, you might be questioned, and if you didn't have a plausible explanation as the neighborhood watch guys did, then you might have a restraining order put on you in which case it would be a crime for you to keep following them, but you wouldn't be arrested, and you wouldn't be jailed, because IT ISN'T A CRIME. Got it?