Libertarianism is something many of us have flirted with. Heck, I did as a young college student. Once I started studying philosophy and actually worked through the implications, I realized that it was actually libertine.
Simply put, all societies organize themselves according to their moral outlook. From speed limits to dress codes. To dump all that and go to “What I want to do is all!” leads not to a nice polite society, but anarchy.
I flirted with it too. I will add that far too many libertarians are douchebags who will cheat you six ways to Sunday if they can get away with it. Their rationalization often seems to be that they’re not their brother’s keeper and, if you’re enough of a sucker to fall for what their selling, it’s your fault for not exercising your freedom properly.