I think yes. I'm not much concerned about the occasional pot smoker. I am concerned about the chronic pot smoker sitting on his ass collecting welfare while me and mine pay for it. I also am not much in favor of giving the state more tax revenue to dole out and that is inevitable. That horse is out of the barn. From decriminalization to taxation is a certainty. More government intervention in the affairs of it's citizens. More votes bought. Another protected class. Maybe I'm wrong but I doubt it.
Libertarian ideals can only work in a libertarian state. The hybrid of libertarian drug use with government taxation/regulation is a bastard child that will never do anybody any good.