I meet all sorts of women while they are with other men (married or dating). Every man does. If I later start a relationship with one of them, after she has parted ways with her prior lover, how does that make me a bad guy?
All we know is he MET her when she was married to someone else, not that he began an affair with her then.
What goes around come around.
Character counts. You can try to rationalize your way to happiness, but the odds are against you.
It doesn't mean your a bad guy, just someone would not mind parting ways with lots of money for a women. If a women is married with a man before and then leaves that and ends up with you, whose to say she won't do it again?