Aside from the sexual relations part - he’s not in violation of any priests’ oath by owning guns or... drinking.
I’m curious, though, as to why the church AND the bishop is in fear of this guy.
Someone of his character is likely to be misappropriating
(aka Stealing) Church funds. If he still has ties to gangs, he is vulnerable for blackmail for all manner of reasons.
Oh, and anyone who would point a gun to his head AND threaten his lover is a control freak and extremely unstable. Keep him away from the parish at all costs!