Posted on 12/11/2004 11:06:54 PM PST by dangus
I've analyzed the pre-election surveys of the 10 major multi-state polling firms. They were: American Research Group (ARG), Fox, Gallup, Quinnipiac (Q), Research 2000 (R2000), Rasmussen (Ras), Strategic Vision (SV) Survey USA (SUSA), and Zogby.
I found Survey USA to be the best. Of the 35 states which I researched, SUSA polled in 29 of them. They were correct every time. What was truly astounding was their track record in states where the contest was not close. SUSA was largely on their own polling in these states, and such contests are often very unpredictable since creating turnout models is virtually impossible. Nonetheless, SUSA was perfect in six states (Calif., Maine, Mich., NY, Ohio and Ore.), and within two points in 11 more (Col., Ill., Ind., Kent., Iowa, Mary., Mo., Okla., Penna., SC., and Texas) SUSA's worst state was Nevada, where Bush won by only two points whereas SUSA had predicted an eight-point blowout.
Mason Dixon was probably best in the presidential races, edging out SUSA. They were correct in all 22 predictions. Their worst state was also Nevada, and they came significantly closer, missing by four points. They were within the margin of error in every state presidential race. Unfortunately, Mason-Dixon underestimated Republicans in several Senate races, calling South Dakota wrong, and being off by six points in South Carolina. However, to be fair, Mason-Dixon did their last Senate polls much earlier, and Republican candidates uniformally used a strategy of saving up campaign cash for a last minute push.
Gallup, once the gold standard of polls, was horrible. Their national polls showed wild swings of 20 points between surveys, before hitting the final number very close, but they were dreadful on the state level. They were wrong in five of the seven races they polled, predicting an easy win for Bush in Wisconsin, and a solid win in Penna., while predicting big wins for Kerry in Ohio and Florida, and a win for Betty Castor in Florida.
Fox was also quite poor, showing Kerry with a 5-point romp in Fla., where Bush won surprisingly easily. They also showed Castor romping by six over Martinez. They only conducted three other polls.
Zogby showed a persistent leftward bias. They overestimated Kerry in 11 out of 13 states. The closest hit Zogby had was Penna., where they were only 2 points off. They were wrong in three states, but inexplicably they showed Bush having a comparably easy time in Ohio, winning by six.
American Research Group was also presistently left-leaning, getting 4 out of 7 wrong. That might look worse than Zogby, but at least they were close, with Florida being the only state they were as much as 3 points off on. But even though their misses were within the margin of error, it is hard to strike their record up to bad luck... they were persistently left-leaning the entire campaign.
Research 2000 went 8 for 8, but passed on the most closely watched races. Their predictions in the uncontested races weren't terribly good: 6 off in NC, 5 off in Ind., 4 off in Ill., 4 off in Mo. They were, however, perfect in Iowa and Nevada, which tripped up other polling firms.
Rasmussen partly atoned for dreadful showings in 2000 (when they called the popular vote for Bush by nine points) and 2002 (when they skewed persistently left-wing.) Ras was 12 for 12, although their margins were often well off.
Quinnipiac also never called a state wrong, although they called Penna. too close to call. They hit all eight state presidential races, plus they were perfect in S.D., and correct in Col.
Strategic Vision got 2 out of 11 wrong, plus they inexplicably called N.J. too close to call, as well as Minn., so they really only got seven right.
One last note: It’s possible that the media’s premature calling of the race for Kerry did hurt Bush. He underperformed poll expectations in the three pacific time zone states for which multiple last-week polls were taken, Oregon, Nevada and Washington. Out of 11 polls, Bush under-performed in 8 and over-performed in none of them. In most of the rest of the country, he did slightly better than he was expected to. The western states have histories of turning against the candidate they perceive as losing.
Just as a note Fox isn't a polling firm. They contracted with a polling company (name escapes me, but it might actually be one of the other ones you looked at) for their polls.
Yes, Fox used a polling firm out of Pennsylvania. It's name escapes me, but it was definitely not one of the firms I listed above.
Good work, Dangus.
Opinion Dynamics is the name. I simply listed them as Fox because I figure no-one knows who Opinion Dynamics are; their polls are sold exclusively to Fox, and Fox creates the formula, so I don't think it's wrong to call it a "Fox" poll... OD just does the labor.
Who needs a polling firm. Cheney said it would be 51% to 47%
I show SUSA being perfect in Ohio (Bush up 2), and being 4 off in Florida, but still correctly calling Bush.
Rasmussen did have several direct hits; I did say they redeened their previously awful recent track record. But they stank on NJ, and (although I didn't include it in my data analysis becaus ethe poll was old) called Virginia to be much clser than it was. Plus, their data techniques are highly questionable.
Does that polling firm still exist?
"Portrait of America" was Rasmussen's web publication. They reinvented themselves somewhat after a disaterous performance in 2000, largely dropping that catch-phrase.
I actually thought they performed rather respectably during the 2000 presidential election, apparently, memory does not serve me correctly.
No, they might have been favorites among a lot of right-wingers, but that was probably they were telling us what we wanted to hear. They were the worst, predicting a nine-point landslide.
Personally, the only polling outfit that seems to be reliable-on a consistent basis-is the bipartisan one operated by Celinda Lake and Ed Goeas.
In all fairness, Florida was not easy for anyone to poll this year, due to the hurricanes hitting predominately Republican areas.
Zogby was the joke of the year. He called the race for Kerry last spring, and then kept a consistent bias to the left all year. In an effort to make his final numbers correct, he announced that his final predictions would be released at 5:00 p.m. on election day. He later admitted that he used the early exit polls in making his final predictions, which was a joke on many levels. First of all, he is supposed to be a pollster himself. Why was he depending on exit polls? Secondly, within a few minutes of the earliest wave of exit polls being released, many on the net were crunching the numbers, and realizing they were seriously flawed. How on earth did he not notice this himself? Thirdly, even with his cheating by using early exit polls in his final "predictions" he still came up with one of the worst predictions. His career is over, which is why he's now going to do everything possible to try to make his numbers look better. He started the day after the election by using one of his earlier polls as his final prediction, which directly contradicted his Nov. 1 statement that his final numbers would come out at 5:00 on Nov. 2.
I keep hearing this name zogby.Who dat be ;)
I think FOX uses Opinion Dynamics ...
Zogby will continue to be hailed by the leftist media as reliable.
As for Zogby, his credibility started to crumble when he began polling for Tom Delay's opponent, Tim Riley (D), in the 2002 elections. Zogby has denied this, but the reason was that DeLay is a gung-ho Israel supporter, and, of course, Zogby is an Arab.
You touched on it with respect to Gallup, but one thing that's always bugged me is that the only way to judge a polling organization's accuracy is to compare its final survey to the actual results. I don't think there's any way that swings anything like as big as those shown by Gallup (among others) actually took place, but there's no way to know. I'm sure it has crossed the minds of many that some of the wild fluctuations were agenda-driven.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.