No doubt you're hoping that only men respond to your point, but, as a woman, and a mother, I find your stance disgusting and immoral. Presumably the biological father of the child is also an adult and I see absolutely no reason whay he should not be held jointly accountable with the biological mother of the child. If he is a poor provider, then the onus is on HER and HER poor choice for the father of her child, not on her cuckolded husband.
If my marriage vows meant so little to me that I was willing to set them aside for a sexual encounter with another man, how could I possibly expect to hold my husband accountable to a higher standard regarding the 'unity' of the family than I am willing to hold myself?