The evidence actually seems to be the other way. this little passage, written long after the events record in the Gospels, certainly suggests otherwise.
1 Corinthians 9:5
New International Version (NIV)
5 Dont we have the right to take a believing wife along with us, as do the other apostles and the Lords brothers and Cephas[a]?
Footnotes:
a.1 Corinthians 9:5 That is, Peter
Half-brothers or Cousins!!!