I don't have my historical sources at hand, but I know Igantius Press has published a book on this subject. I'll try to cite you something specific if I get a chance this evening.
Pope Leo the Great writes to Bishop Rusticus of Narbonne (458/9):The law of continence is the same for the ministers of the altar, for the bishops and for the priests; when they were (still) lay people or lectors, they could freely take a wife and beget children. But once they have reached the ranks mentioned above, what had been permitted is no longer so. [Epist. ad Rusticum Narbonensem episcopum, Inquis, III., Resp. PL 54, 1 204a.]Introduced here is the technical expression law of continence (lex continentiae). It can also be called the law of celibacy in a wide sense. Early Western legislation tends to focus on clerical continence as specifically applied to married clergy: the discipline of abstinence from marital relations. If a bishop, priest or deacon (and subdeacon from the fifth century onwards) was prohibited from having sexual relations once in orders, then it is obvious that his commitment to continence would be the major impediment to subsequent marriage (quite apart from the general disfavour shown towards second marriage). For there could be no real marriage unless it was potentially open to sexual consummation. The same law of continence would also impede the unmarried deacon or priest from marrying. The laws, so clearly expressed in the East, prohibiting marriage to the already ordained may thus be reasonably understood to be but the reverse expression of this more basic discipline of continence. This possibility needs to be taken into account when reconstructing the history of clerical celibacy.