That accusation seems quite stifling to the free expression of opinions that Horowitz claims to champion.
I'll never advocate stifling someone's right to free speech. Anyone can say what they like, just as others are free to criticize them for saying it. But do you not feel that the term "homosexual agenda" rings of an epithet? While not very overt, it doesn't seem entirely innocuous to me.