I am not sure how much the actors are bringing the audience in. Some of the highest-profit films have been animated or have built-in audiences (like much of sci-fi and the horror films) no matter who the stars are.
A lot of films with high paid actors bomb too.
This tends to hurt what they can demand for following films. They sell a product -- themselves. If demand for the product is up, their price goes up, and vice-versa. Standard economics.