One was Zogby. The other was the combined efforts of the Tarrence Group and Lake, Snell, Perry & Associates (aka the Battleground poll).
In 2000, Zogby ended up having Gore up by two, which was right on. Battleground had Bush by 5. Believe it or not, both were right on, even though this is counter-intuitive. When one considers the effects of the natural fuzziness of samples (reflected in MOE or margin of error), two perfectly conducted polls can and would differ by this much, especially if each is making a guess as to how to apportion the undecided vote.
I wrote this about 10 months ago on the same subject. It has more detail:
Contrary to common perception, Zogby was not particularly more accurate than other pollsters last November. Quite the contrary, actually.Let me share with you a letter I wrote to Fred Barnes and Mort Kondrake after hearing them basically say the same thing that you just did:
The reason I am writing is because I believe that you both missed the ball, however, during the discussion of Zogby's polling. As a hobby, I analyze polls and polling data. Although nobody pays me to do my work, I am confident enough of my experience and abilities in this realm that I consider myself more of an expert on the field than most experts who appear on TV.FRegardsSpecifically, my attention was drawn to this exchange:
HUME: Well, he was -- he did very well in '96. He did not do so well in 2000.I have to agree with Mr. Hume on this one. Mr. Zogby gets a lot of credit for his work on the 2000 election due to his accuracy on the national popular vote numbers. Unfortunately for Mr. Zogby, a more detailed analysis of his polling during the 2000 election cycle shows that his work was extremely erratic, and not deserving of the accolades he gets; certainly, he does not merit compliments on his work from knowledgeable folks such as yourself.KONDRACKE: Oh, yes. Oh, he did...
BARNES: No, no. He's the guy -- he is the guy who caught the Gore surge at the end.
KONDRACKE: He did the...
HUME: And Fox News caught it, too, because we had them dead even on the last time out.
BARNES: Well, he caught it before anybody else. He had it in the beginning the week, before, better than anybody -- look, Fox -- I don't mean to disparage Fox. The Fox poll wound up very well, but Zogby was on that case first.
Let me provide some details. If his polling methodology was superior to his competitors, this would have translated to success at the state levels (especially since to get a good national picture, one would have to have a good geographical balance in the sample). Mr. Zogby was all over the place on the state battles.
- Mr. Zogby totally had New York, his area of most expertise and experience, wrong. The following was reported on election day:
"Lazio has been closing the gap since last Thursday," Utica, NY pollster John Zogby told Fox News Channel's Bill O'Reilly. Zogby's Monday tracking poll shows Mrs. Clinton with a two point lead. But the polster said she's losing ground steadily in the final hours before the voting begins Tuesday morning.Mrs. Clinton absolutely crushed Mr. Lazio.
- In the largest state, California, Mr. Zogby's poll released Monday, November 6, 2000 showed Mr. Gore with all of a one point edge over Mr. Bush. As Mr. Goeas of the Tarrance Group pointed out, that is particularly nonsensical in light of the fact that Mr. Zogby had Gore up by a few points nationally at that time. This was not just an "outlier" (a poll that is just plain off, which is to be expected statistically about 1 in 20 times). Mr. Zogby had Gore up by only 3 in California on November 3.
Mr. Gore won California by 12 percent.
- In the final Zogby national poll, which is the one which supposedly "nailed" the election, we see the following demographic breakdowns:
"Gore continues to lead in the East (53% - 37%) while Bush is ahead in the Central/Great Lakes (Bush 50%, Gore 46%). Gore and Bush are in a virtual tie in the South (Gore 48% - Bush 47%), while there also continues to be a virtual tie in the West (Bush 47% - Gore 46%)."Mr. Gore did not carry a single southern state, and in most he did not even come close. And while Mr. Gore did win four western states (California, Washington, Oregon, and New Mexico) so as to make it likely that he did cause the west to be a push overall, since Mr. Zogby was overstating Mr. Bush's support by about 11% in California, this means that he was understating Mr. Bush's support by a comparable amount in the rest of the west in order to get to that virtual tie.
- On 11/3, Mr. Zogby had Ohio showing a lead for Mr. Gore. On 11/5, Mr. Zogby had Ohio showing a 10 point lead for Mr. Bush. That is a tremendous swing, one that defies credulity, especially when one considers that nationally, Mr. Zogby was showing the trend moving away from Mr. Bush and towards Mr. Gore. Mr. Bush won Ohio by 4 points.
Not a single other poll showed any major change in voter sentiment in Ohio during this timeframe.
- On October 31, 2000, Mr. Zogby had Mr. Gore leading Florida by 11 points. Mr. Gore campaigned non-stop in the last few days in Florida while Mr. Bush did not. The race in Florida, as we all know all too well, was a dead heat.
Not a single other poll showed the swings in Florida voter sentiments that Mr. Zogby was showing.
- From 10/29 to 10/31, Mr. Zogby showed Mr. Bush increasing his lead in the national level, from 3 points to five points. But what was he showing on the state level? Here is a chart to demonstrate:
In other words, while his national poll was showing movement towards Mr. Bush, nearly every one of his battleground state polls were showing movement towards Mr. Gore. His own polls were inconsistent with each other.State 10/29 10/31 Change Florida Gore +5 Gore +11 Gore +6 Tennessee Bush +11 Bush +5 Gore +6 Pennsylvania Bush +7 Gore +3 Gore +10 Michigan Tied Gore +1 Gore +1 Missouri Gore +2 Bush +1 Bush +3 Ohio Bush +3 Bush +5 Bush +2 Wisconsin Gore +6 Gore +8 Gore +2 Illinois Gore +7 Gore +7 No Change
- Mr. Zogby gets credit for being the first to show the late movement towards Mr. Gore. In reality, Mr. Zogby's polls showed a big change from Mr. Bush leading on 11/5 to Mr. Gore leading on 11/6. Since his poll was a rolling sample, to get such a marked change in one day, either a very good day for Mr. Bush had to fall off the rotating sample, or a very good day for Mr. Gore had to be added. Since Mr. Zogby's numbers had been pretty steady for the week preceding, we can infer that the former was not the case. In order for a four day tracking poll to make a 4 point swing in one day when the day falling out of the sample was not an outlier, the day coming into the sample must have been tremendous for Mr. Gore, on the order of a 10 point leap for that day's sample. Unlike the Gallup poll, in which the one day sample sizes were so small that days where the numbers would jump that much were statistically feasible, Mr. Zogby was sampling over 400 people per day, which makes such swings difficult to explain by statistical chance.
Did Mr. Zogby get the correct gap between Mr. Bush and Mr. Gore on the national level? Yes. But a review of his polling during the days leading up to the election shows that he was wrong, and in some cases tremendously so, as often as he was correct, and that his own polls were inconsistent with each other from day to day and even on the same day.
Side note on Kondracke: the weekend before the GOP landslide, I saw him on the McLaughlin Group, and he called a 40 to 50 seat swing and Republican control of the House for the first time in 40 years. Not sure what polls he was using, because I didn't see anyone else make that call, nor any other commentator.
He got very lucky in getting close to the final national vote breakdown, but his detailed analysis "stunk on ice."