In a sense he is right. You really can build most of the necessary stuff from readily-available materials and get to some sort of orbit.
The cost comes when you start trying for high performance, precision, reliability, and (for manned flight) getting the crew home alive.