The French were indeed the first enemy of the United States (not counting the indians). Under the Adams administration we fought what was called the "Quasi War" against them for authorizing privateers against our shipping.
Our new Navy and our new Marines kicked their butt. Our privateers made out OK too.
And let's not forget "The XYZ Affair", also under President Adams. When the American deligation to France was told, in increasingly less and less uncertain terms that if they wanted any business started/continued/concluded the Americans had better wake up and pony-up more than a few,er, "inducements".