In all fairness, society has tried to gradate sexuality with minors: adults with young children, extremely bad; adults with older children, or older children with young children, or young children with young children, bad; older children with older children, questionable; and as soon as children turn 16 or 18, they can do what they want, depending on where they live.
While most people are satisfied with these grades of offense, more or less, they obviously have some real problems.
To start with, minor children are to a great extent legal chattels of their parents. The law sees them both as property and as having considerable monetary value to their parents.
So a stranger having sex with children is often considered not just rape, since minors cannot legally agree to have sex; but it is also considered as theft and vandalism. And if pregnancy or disease results there could be significant and extensive costs, both financial and personal.
Add on top of that children maturing at very different rates, and the law can seem bizarre, in that one person may not be physically, emotionally or financially ready to have sex until their 20’s, which others might be in many ways mature at age 16 (at least according to several states.)
Your posit fails to account for the current state of humanity, where children do not have parents, are treated as cattle by the state and have no monetary value until they are working age...until then, you can abort them, dispose of them and leave them in childcare as much as you like.