Well, first of all, a vow isn't a contract; it's a promise made to God, or with God as a formal witness. You don't say, "I will keep my promises to God unless it gets reallly difficult." You say, "I will keep my promises, so help me God."
Second, if you love someone, you are committed to their well-being, and especially to their spiritual well-being. A straying spouse will never experience spiritual well-being until he or she stops straying and re-commits to their marriage partner. That's why the faithful spouse must continually pray and hope that the offending spouse will repent and turn their life around, and that the marriage will be restored.
II can only think of the prophet Hosea, whose forgiveness and fidelity toward his wife (who was a whore) is the image of God's never-failing love toward our straying souls.
This is a hard saying, very hard: but love calls us beyond what is pleasant for ourselves. You're faithful to a faithless spouse because you love him or her. You love your spouse, not because of feelings (delighted, wounded, blissful, miserable) but because of a permanent commitment, which is an act of freedom.
Or participation in ritual abuse. There's a fine line between being faithful to a faithless spouse and being that spouse's doormat. This is why we must be selective before taking vows.