There's a tension between wanting to minimize the number of divorces and wanting to allow divorces because of the least infidelity. If you wnt to set the bar high for divorce, that means excluding some of the minor infidelities like phone sex or masturbation. Should a family be ruined because a wife catches her husband with his hand on his secretary's bosom? Let the couple figure out how to punish each other for those minor foibles and reserve court-sanctioned divorce for the major cases of adultery.
Should a family be ruined because the husband couldn't keep his hands to himself? What if he comes home with AIDS? He dies, the innocent wife dies, the kids are orphans. What if the secretary files an EEO type complaint? Even if he's innocent of harrassment he still will lose everything. And his family suffers for it. He and his family are far better off if he remembers to only touch what really belongs to him.
Every form of sexual or romantic activity outside of marriage is adultery. A married man should not even give a car ride to a woman he is not married to without a chaperone. Phone sex or masturbation have no place in a marriage. Both fall into the range of adultery.
Note however that the bible gives permission to divorce for adultery. It does not command it. It is the preferred solution for the wronged spouse to forgive.