I have worked wells where we experimented with different bits (some one-off experimentals), different drilling fluids, and different techniques. It has been one heck of an education, but it wasn't federally funded. The risks are there, but so are the rewards, and like in the wild, the most efficient hunters eat best.
From seismic techniques to other exploration methodologies to the nuts and bolts of drilling and producing a well, private interests stand to profit from innovation, and that is a powerful motive indeed.