I always thought Byrne was from Arbutus. There's a big difference between Arbutus (blue collar, just south of Balto.) and Annapolis. I never had the idea that Byrne had a privileged background, but I could be wrong.
Come to think of it, you're probably right. I have a few wires crossed about the geneaology of the Heads.