It is true that a loss of members who simply left the Catholic Church would not immediately show up in statistics that just measure deaths and new members (both infant baptisms and adult conversion). But it would be reflected in those statistics eventually, because people who leave will not have their children baptized.
Another point is that polls show that the percentage of Americans who identify themselves as “Catholic” has held very steady at about 25% for several decades. That would imply that the number who think of themselves as Catholic is increasing at roughly the same rate as the US population.
As far as people leaving the catholic Church at four times the rate taht they enter, that is somewhat misleading. Most of the exodus from the catholic Church in the US happened in the late 60s and 70s, when the Church was going through rapid changes and turmoil. There were also very few converts to the Catholic Church in that period. Things have stabilized a great deal since then. More people are coming in and fewer going out.
Right now it seems that the Catholic population in the US is growing slowly.