1. Oil is a very price-inelastic commodity, in the short term. Perhaps only food is more price-inelastic. It only takes a very small shortfall of supply, to send prices skyrocketing. Similarly, a very small surplus would send prices diving. (You won't see that effect from a small surplus though — because OPEC would dial back production just enough to eliminate the surplus, and keep prices high. OPEC can't keep large surpluses from forming — but, they can keep small ones from happening. Even OPEC’s market-manipulating powers are limited — and they have far, far more power to manipulate the markets than the speculators in futures markets.)
2. Commodities futures market speculators (any futures market) provide liquidity. Without them, a large spread would open up between bid and ask prices. Producers and large consumers would have a more difficult time using the futures market to hedge (protect themselves from price rises or drops). Punters at a horse race bet on the race results — but they don't determine the race results. Oil futures speculators bet on the future price of oil — but, they don't determine that price.
3. Speculators are on both sides of the market. There are both longs and shorts — one every single trade.
Consider who wants you to believe that the speculators are the blame for high oil prices. Don't let the real culprits trick you.
This is caused by new phenomena in the futures markets in which large institutional, long-only, investors are entering, and using some tricks like swaps to get around margin requirements by being classified as "commercial" traders.
You can deny that there is currently a structural problem in the futures markets until you're blue in the face, but the fact is that CALPERS and other large pension funds, combined with big Wall Street firms which have discovered "swaps" and need somewhere to park their money after pulling it out of subprime mortgages and real estate are having an effect.
Oil should be $40-70/bbl., when you take into account actual supply and demand.