Indeed. The priest is the spiritual father of his entire parish. Which of his two families would take precedence if, for instance, his wife or one of his kids had a medical emergency just as Mass was scheduled to begin, or as he was being summoned for last rites, or was scheduled to perform a wedding ceremony? Which of his families would be subordinated to the other? How is it spiritually and physically possible to live two separate vocations simultaneously (let alone unreservedly, as we are called to do by God)?
Russian Orthodox priests and wives/families, who married before priesthood, have this conundrum as well.