Now, our highest rates of unemployment and underemployment are among unskilled and semi-skilled workers, who are in direct competition with both illegal and legal immigrants, mostly from Mexico. The H1B visas mostly apply to engineers and other professionals from places like India or China. These aren't as much of a concern demographically because in sheer numbers they are far fewer, but it does still leave us with the question of why we need to import them in any numbers at all when the "recovery" involves American engineers finding part-time dead end clerical/retail/service jobs.
No one seems to think “unlimited immigration” is a problem until their job is threatened/eliminated or the wages are (in some cases, sharply) reduced by “guest workers”.
Case in point: Walt Disney (see recent news).