The Catholic Church teaches that those described in the Bible as "brothers of the Lord" and so forth were not children of Mary. There is nothing, in fact, that mentions their being children of Mary, so it's a question of interpretation of "brothers/sisters."
One line of analysis posits James, Joses, etc., as more distant relatives through His mother, because of the lack of nouns in Hebrew to designate "first-cousin-once-removed" and so forth. Another theory is that they may have been children of Joseph's from a previous marriage, and therefore Jesus' step-brothers.
It is, of course, a matter of one's accepting the interpretive authority of the Church on the question. On the other hand, the issue of Jesus' being conceived "of the Holy Spirit," rather than the natural son of Joseph, is clearly settled in the Gospels.
I'd always understood (mind you, I'm of that heretical sect known as Episcopalian) that Mary was a virgin when she married (but not necessarily after Jesus' birth). Was unaware of the nuance - thanks for the explication!