Several states (Virginia and New York, for example) reserved the right to secession as a condition to their ratification of the Constitution. Thus, you either have to argue that these states were never legally part of the Union (since the conditions of ratification were unacceptable) or that these states have the right to secession as set forth at the time of ratification. |
Grant for the sake of argument that you're correct about Virginia. The problem is, most of the secessionist states did not (could not!) have such clauses. The independent country of Virginia entered into a compact (which included significant military support) with states that were in open revolt against the U.S.
Virginia thereby declared itself to be our enemy, and as such the U.S. was justified in a military campaign against it.