The North passed an amendment to the Constitution to make slavery permanent and not subject to further amendments (Corwin amendment). The great majority of the north cared little about southern slaves. The war was fought to subjugate the south.
So are you saying that if the South had won the war that slavery would have ended? No. What you are doing is splitting a hair to divert from the fact that the confederate flag was the flag that represented a government that supported slavery. Regardless of where the North stood on slavery at the beginning of the war, the end result was that by the North winning, slavery was ended.