The judges scores and caller votes do nothing to determine the winner. The producers of the show sit down every week and decide who is staying and who is going. It is about entertainment and having the right mix of contestants to get the highest viewership.
They have done it masterfully this season and will insure that the right person wins to be able to go into next season with even a bigger audience.
In my opinion, you are spot on. That is exactly what is happening.