It seems pretty clear he is referring to the libertarian movement within the Tea Party. He said the movement within the Republican party and Tea Party trying to change conservatism. He didn't oppose the Tea Party but opposed it changing.
The Tea Party has not once, at any time that I am aware of, take a position on a social issue...for or against.