If it is only one, then if I can borrow zero energy for an "infinite" amount of time and I can borrow an "infinite" amount of energy for zero amount of time, it appears to me that I can borrow something close to zero which keeps other things from occurring for a very very very ... long time. I also get the feeling that small pilfering is more likely than large pilfering. So looking at space that way, why would anything happen?
On the other hand, if many different particles can arise from the same point, I would expect something to "always" occupy the point.