I always lie to pollsters. It is my mission to make them useless.
That said, the most important thing revealed here, and common to nearly every poll, is the treatment of hangups, no answers, or refusal to participate.
They just ignore these and move on, as if they never happened. But in the real world of statistical sampling, they did happen, they are critically important, and they cannot be ignored.
Suppose a pollster is told he needs to sample 1000 people for his results to be “statistically significant”. He makes 1000 calls, but 400 of them don’t yield responses. He just continues until he has his 1000. But perhaps he has had to call 1600 or more in total to get 1000 responses. The fact is, those other 600 are samples, for which his response should be included as “UNKNOWN”.
Unknowns must be included in the sample for the statistical calculations about the “confidence interval”, which measures the expected range of variability due to the random sample, and is critical to estimating the “margin of error” that they love to quote. Unknowns can VASTLY reduce the confidence interval (increase the margin of error), when they are properly counted.
Second, when they attempt to sample X percent Democrats and 100-X percent Republicans, the same effect occurs. They will obviously need to sample some excess numbers to get the party bias they want. They just ignore the excess calls. But they don’t vanish. They were sampled. Again they are unknown.
So these statistical abuses ALWAYS have the effect of increasing the quoted “margin of error” by reducing the confidence in the result.
For the curious, “margin of error” means that in the process of taking many samples there will be variation from sample to sample just due to randomness. This is the nature of sampling. A confidence interval is chosen for the study, typically 90%. So a 3% margin of error should be stated “We anticipate that in 90% of the samples taken, the true result is within 3% of the sample. The other 10% of samples would fall outside this range.”
Throw in the Unknowns, and that confidence interval can quickly fall, so it could really mean something like “”We anticipate that in 60% of the samples taken, the true result is within 3% of the sample. The other 40% of samples would fall outside this range.”
Barely better than a coin toss.
You remind me of a concern I had four years ago, when there were rumors around that some of the polls to get 1000 responses, were having to make over 4000 calls. People assured me then that I was wrong, but I still have concerns that the response rate is so low that separate polls are not consistently getting different respondents. Thus groups like Real Clear Politics that average the polls together to help reduce random error are actually increasing systemic error. It is widely believed that polling firms try to periodically remove non-responding numbers from their call lists, and are the non-respondent statistically the same as those that do answer their questions?