[Tarpaulin]

Wouldn't that cause an object located at any point inside the sphere to be drawn towards the center?

[Doctor Stochastic]

No. Imagine a cone with constant aperture at the object. The cone intersects the surface of the sphere (which is all one has) in a spherical cap. The attraction of the cap is inversely proportional to the square of the distance to the cap, but so is the amount of material contained in the cap. These cancel exactly in each direction. The net attraction is zero.

The same thing happens with a charged spherical surface too.