Specifically, it was his vehement opposition to laws allowing for the medical use of marijuana. I think the Libertarian party was wrong to target someone who, overall, was much closer to its philosophy of government than almost any other Republican at the national level.
The L-party obsession with legalizing drugs is the single largest factor in its continued marginalization. If they stood for something useful, like repealing some genuinely unpopular laws, or shutting down a few State bureaucracies and federal cabinet departments, they might actually begin to win some offices. But I guess winning isn't really what being a libertarian is all about. If it was, they'd have done it by now, somewhere.