That has actually happened to someone I know. Spoke to him on Easter about it. He said he left the Boy Scouts because they had girls involved. And he was tracking to go Eagle and loved it. And this young man is quite impressive. Would have made a hell of an Eagle.
I believe there are a couple of religious organizations that fill the role, but I don't know how extensive their membership is.