If they aren’t man enough to actually marry the women with whom they have conceived said children, then I would argue they have no rights at all.
The right of fathers to protect their daughters and sons between conception and birth arises from nature, not from a government document.