Punish the action not the thought.
I agree
The way I always figured it is the general range of punishment is set by the action. The exact amount of punishment - within the law's range - should be set by a fair and impartial judge, who gets to judge just how malicious the perp's intent was. Hence the separate sentencing round in a trial.
Basically, by creating "hate-crime" laws, we take away this power away from the judge. Ironically, this is one of the powers the judiciary is supposed to have!