Correct me if I'm wrong, but didn't you have to be a property owner at that time to vote?
I know that this was true in Maryland in the past. It was probably the case in other states too.
My only problem with restricting suffrage to property owners is that there are plenty of high income people who pay a lot of taxes who do not own property.