Basically, you’re arguing that a minority can tyrannize the majority on a whim.
No, just the opposite, you are arguing that the majority can tyrannize a minority. A minority voluntarily and peacefully withdrawing themselves from the union doesn’t force the majority to do anything against their will or abrograte any of their rights. On the other hand, a majority forcibly keeping a minority in the union against their will does both of those things. So “tyranny” only really applies in the latter situation.
This is one iteration of the democratic tyranny that our founders wanted to avoid by forming a republican government.