If Joseph had other children, then why weren't they WITH him and Mary on the trip to Bethlehem for the mandated census? Seeing as Jesus was the eldest son and NONE of his younger brothers were there at His crucifixion, it was only right that He entrusted the care of His mother to the ONLY Apostle there, John. His brothers James and Jude eventually came to saving faith in Jesus as Messiah. They were younger than Him, not older, which would show that they were NOT children of Joseph from before his marriage to Mary.
What's the big deal about Mary having a normal marriage and children with her husband? It had NOTHING to do with the virgin birth of the incarnation since Jesus was the firstborn son of Mary and Joseph. Placing her on a pedestal as an icon to perpetual virginity doesn't honor her or Christ.
Maybe they were a LOT older, had families of their own, and traveled separately.