So what's the fuss?
If sexual urges toward children are "not a problem", then such people are not "dangerous" and cannot be legitimately watched. They can work as teachers or day-care providers, guidance counselors, bus drivers, ice cream truck drivers. If anyone shouts a warning, the response is: "But he hasn't done anything."
Can I put a ticking time bomb in your lap? It might go off -- but I assure you it is perfectly harmless (until it does go off). You're willing to take that chance?