That's because most perverted freaks and pedophiles using the cover of Catholicism found a way to get close to boys. That's different than saying most Catholic priests are perverted freaks. They weren't priests in the first place, but used the priesthood to mask their homosexual tendencies.
People like that take all sorts of jobs to be close to kids. They get involved in scouting, or teaching younger kids – nothing Catholic about that.
Also, this is something a lot of single moms deal with. They are often desperate because good mean are rare to find, and they entrust their children to what they considered a mentor or a guidance to their children. These pedophiles look for jobs which give them that opportunity, and priesthood isn't the only job that allows them to do that.
You're very right that pedophiles often take jobs or do volunteer work that brings them close to children. It's incumbent on administrators who oversee these types of roles to be on the lookout for anything suspicious. Great observations!