The tradition that Mary did not have any more kids starts very early. So early, that you don’t really see any mention of James being the full blooded brother of Jesus.
Again, I will not get to bent out of shape about it. Luther and Calvin both believed Mary remained a virgin till her death. Both were quite clear about it.
That's because he wasn't. Jesus' father was God the Father, so He was only half-brother of James. That has nothing at all to do with whether or not Mary remained a virgin for the rest of her life.