As I understand it, if you read the first 126 sonnets all together it is clear that they are all referring to one person who is male.
I a’int buyin it, though I admit its been awhile since I read the complete works and the Sonnets.
I think i would have recalled something to the effect of “Harketh, dude! Thou hast a package most fair.”
:]