Posted on 11/08/2004 12:00:54 PM PST by Cableguy
By and large the pollsters did an excellent job this year, especially when you consider the significant variables they were grappling with (massive registration increases, historic levels of intensity, cell phones, etc) and also that they were working under intense public scrutiny and scorching partisan attacks from both sides.
At the national level the answer to the question is pretty straightforward. Ed Goeas's GW-Battleground Vote Projection and Pew Research got it exactly right. Goeas's final Battleground projection was Bush 51.2, Kerry 47.8, Nader 0.5 and Pew's final allocation was 51-48-1.
CBS News/NY Times also nailed the final spread in the race (Bush +3), though they didn't allocate undecideds which makes their final less impressive than Battleground's and Pew's.
Raghavan Mayur at TIPP also deserves a mention for outperforming almost all of the big media pollsters and coming in just a tick off the final results with their final Bush 50.1, Kerry 48.0, Nader 1.1 projection. Scott Rasmussen of Rasmussen Reports also had a solid final result with their Bush 50.2, Kerry 48.5 final projection.
USA Today/CNN/Gallup finished poorly this year. Ironically, Gallup would have finished at or near the top of the list had they allocated undecideds 50/50 like Pew, as opposed to giving what appears to be 100% of the undecideds to Kerry.
By the way, just as a point for future reference Bush's Job Approval rating among likely voters in Gallup's final poll was 51%. President Bush's percentage of the national vote total this election? 51.0%.
Of the entire group of pollsters this year, only three got it wrong. Celinda Lake of Lake, Snell, Perry & Associates (the Democratic polling firm for the GW-Battleground 2004 Poll), Marist College and FOX News/Opinion Dynamics all projected a John Kerry popular vote victory. (The Democratic polling firm Democracy Corps also called for a one-point Kerry victory.)
| Poll |
Bush
|
Kerry
|
Nader
|
Error
|
|
Final Results
|
51.0%
|
48.0%
|
0.4%
|
-
|
| Battleground/Tarrance |
51.2
|
47.8
|
0.5
|
Dead On
|
| Pew Research |
51
|
48
|
1
|
Dead On
|
| CBS/NY Times |
49
|
46
|
1
|
0%
|
| TIPP |
50.1
|
48.0
|
1.1
|
1%
|
| CBS News |
49
|
47
|
1
|
1%
|
| Rasmussen |
50.2
|
48.5
|
-
|
1%
|
| ABC/Wash Post |
49
|
48
|
0
|
2%
|
| Harris |
49
|
48
|
1
|
2%
|
| NBC/WSJ |
48
|
47
|
1
|
2%
|
| Reuters/Zogby |
48
|
47
|
1
|
2%
|
| Newsweek |
50
|
44
|
1
|
3%
|
| CNN/USAT/Gallup |
49
|
49
|
1
|
3%
|
| ARG |
48
|
48
|
1
|
3%
|
| Marist College |
49
|
50
|
0
|
4%
|
| Battleground/Lake |
49
|
51
|
0
|
5%
|
| FOX/Opn Dyn |
46
|
48
|
1
|
5%
|
Now to the battleground state polls. Let's stipulate up front that both the national poll and the battleground state analyses are not meant to be comprehensive or quantitative, but rather quick snapshots using a simple, commonsense grading system. We looked at two basic factors to determine accuracy rankings: the percentage of states where the pollster correctly picked the winner and how much the pollster's projections varied from the final result in each state.
One last thing. In addition to calculating the difference between the projections from each polling firm and the final vote totals, we also tried to determine whether we could detect any trends that might show favorability toward one party or another by consistently underestimating or overestimating support for one candidate or another.
Here is what we found from all of the final polling conducted in the 2004 battleground states at the Presidential level (Click Here to View the Full Tabulated Results):
1) Mason-Dixon
Failed to Project Winner: 6.2% | Average Error = 1.8
A final Minnesota poll showing a one-point Bush win is the only blemish on Mason-Dixon's otherwise perfect scorecard this year. Not only did Brad Coker project the correct winner in 15 out of the 16 battleground states we looked at, he did so with amazing accuracy. Four states were dead on the final number and the overall difference between Mason-Dixon's final polls and the actual election results was a minuscule 1.8 points. Furthermore, if you look down through the list of Mason-Dixon's projections it's impossible to detect any consistent leanings toward one candidate or another.
2) Rasmussen
Failed to Project Winner: 6.2% | Average Error = 2.3
Rasmussen's battleground state polling this year was extremely solid and a close second to Mason-Dixon. He was dead on in PA and just missed in IA. Average error was a very good 2.3 points, with NJ and AZ the only states where he wasn't within three points of the final spread. No partisan trends either way.
3) SurveyUSA
Failed to Project Winner: 7.1% | Average Error = 2.8
Some people have questioned methodology and reliability of SurveyUSA's polls. Their performance in the battleground states this year should answer a good number of those questions: 13 out of 14 states called correctly, including dead-on numbers in Maine, Michigan, and Ohio. The difference between projections and actuals in a few of the states (CO, FL, NV & NJ) was on the high side, even if it was within the margin of error. No identifiable leanings toward either candidate.
4) Research 2000
Failed to Project Winner: 14.3% | Average Error = 3.1
Del Ali's firm only conducted polling in seven battleground states this year. They got six of those states right, including nailing a Bush one-point victory in Iowa. The big miss came in Florida, where Research 2000's final poll called for a one-point Kerry win. Average error from the final results was 3.1 percent - which is respectable - though in every instance except one (Iowa) they overestimated support for Kerry and underestimated support for Bush.
5) Quinnipiac
Failed to Project Winner: 33.3% | Average Error = 2.3
Many people dismissed Quinnipiac's final poll in Florida (Bush +8) as an outlier. Wrong. Quinnipiac was closer than most in FL, and they also did a nice job in the only two other states where they polled, NJ and PA. In fact, Quinnipiac would have finished pretty high in our rankings except they called for a tie in Pennsylvania. All three of their projections overestimated the spread for President Bush.
6) Zogby
Failed to Project Winner: 27.3% | Average Error = 3.8
As we all know, Zogby had been on record for months saying that Kerry was going to win this race. Despite his final tracking poll that put Bush ahead by one point nationally, Zogby's polling at the state level reflected his belief that Kerry was going to be the beneficiary of huge turnout - especially among the youth vote. The result is that Zogby missed three of the eleven states he polled in (FL, IA, and NM), had a relatively high error rate across the board (3.8%), and his numbers generally skewed in favor of John Kerry.
Adding insult to injury, Zogby's bizarre election day antics calling for "surprises" in Colorado and Virginia and a decisive 311 electoral vote victory for Kerry suggest he was relying on (not to mention taken in by) the badly skewed early exit poll data.
Let's be honest: Zogby's conduct this year bordered on outrageous. No other independent pollster was out making public predictions of a John Kerry or George W. Bush victory months before hand. And no other pollster decided to wait until 5:30pm Eastern time on election day to post their final numbers.
7) American Research Group
Failed to Project Winner: 50% | Average Error = 2.0
ARG got a bit unlucky this year. They called for a 1-point Bush win in NH and the result was Kerry +1. They also projected a 1-point Kerry win in New Mexico and the result was Bush +1. The big miss, however, came in Florida where ARG's last poll had Kerry up two. ARG did offset these misses by nailing tight outcomes in IA and WI giving them an excellent score in overall average error.
8) FOX News/Opinion Dynamics
Failed to Project Winner: 50% | Average Error = 4.5
It stands to reason that if your national numbers are way off then at least some of your state numbers are going to be bad as well. This is certainly true of FOX's final poll in FL where they showed Kerry ahead by five points. In fact, that 10-point miss stands out as the worst among the final battleground polls we looked at. Another miss in Wisconsin put FOX News/Opinion Dynamics at 50/50 in battlegrounds, with an average error rate of 4.5%
9) Strategic Vision
Failed to Project Winner: 44% | Average Error = 2.4
We were assaulted by some people for labeling Strategic Vision a Republican polling firm. For the record, the reason we did this was twofold: 1) they had a history of polling for Republican clients and 2) their polls were not sponsored by any independent media outlets like newspapers and television stations.
Strategic Vision's projections for a Bush win in MI and a tie in NJ, seems to provide evidence that, at least in these two states, they were skewing toward President Bush. Their polling in the other seven battleground states was reasonable, though except for Florida and Ohio, they consistently underestimated support for Senator Kerry.
10) CNN/USA Today/Gallup
Failed to Project Winner: 67% | Average Error = 5.8
To find Gallup's name at the bottom of the list is nothing short of shocking. In four of the biggest, most important states in the election this year (FL, OH, PA, and WI) CNN/USA Today/Gallup wasn't even close. In fact, they got it exactly backwards calling for Kerry wins in Florida and Ohio by 3 and 4 points, and Bush wins in PA and WI by 4 and 8.
With most of the polls correctly predicting the winner, why were news outlets so quick to believe exit polls which are known not to have anywhere near the reliability? Even when actual vote totals came in, they were reluctant to get off the exit polls story. Is there any data out there whether the exit polls affected voting; maybe cost us some states?
I'm not surprised about M-D. They had a stellar record from 2002, as well. When M-D results come in, people listen.
Zogby and Gallup state polls were the pits. Gallup should get out of state polling completely.
I am surprised about Strategic Vision being off. Sometimes I relied on their results for optimism in what I thought would be a tight election. They were over optimistic for President Bush and need to balance their polling a little more.
I'm happy for Rasmussen after his 2000 Portrait of America disaster. With Ras and Survey USA automated polling now proven, looks like this is the way the polling will end up. It's cheaper and seems just as effective as human polling.
I don't disagree. Many of the pollsters were correct on the final result, within the margin of error. To try to declare that any one of that group was more or less accurate than another, based on only a single data point, is kind of silly, IMO.
Mason Dixon being top dog is no suprise, but the performance of the phonebots -- Rasmussen and SurveyUSA -- certainly was.
bump
Based on one bad year? Coming into this cycle, Gallup had a pretty good track record in statewide polling, based on the comparison data on SurveyUSA's website. They were in the top quartile in terms of accuracy before this year.
I agree that I prematurely wrote off Ras and SUSA. But Ras was right about the race being within MOE. Nothing wrong with publishing surveys everyday. That is how he makes his money.
I suppose. They do gold-standard national polls. I'm surprised that they flounder a bit with state polls. Rasmussen straightened up within 4 years so I guess they can to.
Don't forget the kids' vote - Weekly Reader!!
Zogby's behavior was disgraceful. He needs to pack it up after blowing 2 elections in a row.
Glad to see that the FOX/Opn Dyn was SOOO wrong.
Fox polls is a joke! I hope to never hear those bozos in ever of our news casts again.
They didn't claim W when it was 98% slam dunk until the very end.
What in the hell were they thinking?
Why is that?
The irony of it was that they were part of the cabal tht actually paid those libs to conduct those polls.
Will they learn to stop playing footsies with the left? I doubt it.
Don't know if it's true but NYPost reported today that CNN won the election night and day after ratings.
On the whole, Bush had a three percent lead for most of the polling season, except for the brief jump after the convention. That seems to have been more stable than most of the actual numbers.
It is interesting to note that a three percent lead in the popular vote doesn't necessarily win the election. You gotta have the swing states.
Failed to Project Winner: 44% | Average Error = 2.4
Huh? They polled 11 states (PA, OH, FL, MI, IA, WI, WA, NJ, GA, MN, CA) and got 7 right, a 36% error rate.
They missed MI and MN by calling a tie, and WI and MN by calling a slight Bush lead trending towards Kerry.
Strategic Vision's projections for a Bush win in MI and a tie in NJ, seems to provide evidence that, at least in these two states, they were skewing toward President Bush. Their polling in the other seven battleground states was reasonable, though except for Florida and Ohio, they consistently underestimated support for Senator Kerry.
They only called a tie in MI very late. They had Kerry consistently ahead most of the time, including 1 of their last 3 polls in the final 10 days. Both NJ and MI were also polls with lots of undecideds still left by the SV methodlogy.
The "underestimating" of Kerry support is debateable, sicne the obvious answer is that they could simply say the undecideds in their sample broke for Kerry 2 to 1, as in MI, NJ, and MN. Mostly, the understatement was by 1 point, as in IA, WA, and PA, while they overstated Kerry 2 pts in FL and 1 pt in OH.
Its wishful thinking of RCP to call Strategic Vision biased, but to take Research2000 and ARG as not being severly left-leaning in their bias.
Zogby should be horse-whipped.
How does one correctly pronounce "Goeas"?
Or, in Zogby's case, goat-whipped.
Is Zogby officially out of business now? How could anyone take his "work" seriously now? Just another DNC plant.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.