Your post is exactly right.
When visiing friends in rural Northeastern PA, I see them all the time. Not in tremendous numbers, but enough to make you think. The numbers posted are amazing!
Just go to a K-Mart or Wal_mart....you will see them, in small numbers. They clean up the floors in those stores, late at night. They live around the area. And my tale about Home Depot is true: they are being used to do installation work.
They are primarily using I-80 as a mainline to NYC - and so do not (yet!!!!) live in these areas in great numbers. First of all, there are not that many apartments available. Secondly, there is no community of their kind to join up with, and not much work for them to do. Also, the smaller towns and cities usually do not have good public transit - nor bus/train service to connect with larger cities such as NY or Philly.
But the legal citixens who live in these areas are in literal fear.....of illegals coming and causing a job crunch, crime, and houseing shortages, as they have caused in major cities like NYC.