Posted on 11/05/2012 7:34:52 AM PST by smoothsailing
November 5, 2012 7:18 A.M.
From the final Morning Jolt before Election Day:
A Quick Trip Down the Memory Lane of Recent Polling
So a lot of people who don’t read me that closely are going to look at what follows and interpret it as “Jim’s saying the polls are always wrong.” That’s not what I’m saying, but I’m prefacing all of this with that prediction, because we’ve all seen that when people don’t like what you have to say, they attempt to cut off discussion by calling you insane or silly. Sneering “truther” in response to a disagreement from the conventional wisdom is almost as worn out as “racist.”
At the heart of the entire point of polling in political races is that supposition that the people in the sample are a realistic representation of the folks who will vote in the election. Now that the response rate for polls has plummeted all the way down to 9 percent — that is, out of every 100 calls the pollster makes, only 9 are completed — getting a sample that looks like the likely electorate in Election Day is tougher and tougher.
So pollsters adjust, they make extra calls and make sure they have a sample that is properly balanced by gender, by race, by age, and often times, by geography of the nation or state that they’re polling. They do this based on this fairly simple conclusion — the makeup of the kind of people who will answer questions from a pollster for ten or twenty minutes may not accurately represent the makeup of who will vote in the election. So if one gender, racial group, age group, or region may be more likely to take the time to answer questions than another, why not one party?
Folks like me have been wondering for a while whether folks on the Right — with distrust and suspicion of the media fueled by decades’ worth of stories and examples and anecdotes of what they deem media bias — are more likely to hang up on the pollster, and/or urge him to do anatomically difficult things to himself, than folks on the Left. Think of this as an American version of the “Shy Tory” factor.
In 2002, Democrats argued, and the media largely agreed, that President George W. Bush was “selected, not elected” and contended that despite the events of 9/11, and the talk of war with Iraq, Democrats would thrive in the midterm elections.
I found this article describing the difference between the late polls and the final results on a lefty site charging massive voter fraud in favor of the Republicans. He summarizes:
– 14 races showed a post opinion poll swing towards the Republican Party (by between 3 and 16 points);
– 2 races showed a post opinion poll swing towards the Democratic Party (by 2 and 4 points);
– In three races the pollsters were close to correct;
– The largest post opinion poll vote swings occurred in Minnesota and Georgia where pollsters got the final result wrong
2004: Bob Shrum was calling John Kerry “Mr. President” after seeing the first round of exit polls. Think about it, this wasn’t just guessing who would actually vote; everybody coming out of a polling place was a definite voter. Even then, it got thrown off because Kerry voters were much more willing to talk to the exit pollsters than Bush voters:
Interviewing for the 2004 exit polls was the most inaccurate of any in the past five presidential elections as procedural problems compounded by the refusal of large numbers of Republican voters to be surveyed led to inflated estimates of support for John F. Kerry, according to a report released yesterday by the research firms responsible for the flawed surveys.
The exit pollsters emphasized that the flaws did not produce a single incorrect projection of the winner in a state on election night. But “there were 26 states in which the estimates produced by the exit poll data overstated the vote for John Kerry . . . and there were four states in which the exit poll estimates overstated the vote for George W. Bush,” said Joe Lenski of Edison Media Research and Warren Mitofsky of Mitofsky International.
One other point: the exit pollsters were disproportionately collegiate women. Raise your hand if you think some men might be willing to tell a cute college coed that they voted for Kerry. Yup, me too.
2006: The popular vote in the House of Representatives races came out to 52 percent for the Democrats, 44 percent for Republicans, an 8 point margin. Some institutions came close on the generic ballot question, USA Today/Gallup (7 points), ABC News/Washington Post (6 points) and Pew (4 points). But others overstated it dramatically: Fox News (13 points) CNN (20 points) Newsweek (16 points) and Time (15 points), CBS/New York Times (18 points).
2008: If you’re a pollster who tends to overstate the number of Democrats in your sample, this was your year — fatigue over President Bush and war, a Wall Street collapse and economic meltdown, a drastically underfunded Republican candidate who spent much of his career fighting his own party, the first African-American nominee of a major party . . . and yet, some pollsters still overshot it: Marist, CBS News, NBC/Wall Street Journal had Obama winning by 9, and Reuters had Obama winning by 11, as did Gallup.
2010: Polling wasn’t quite as bad this cycle; everyone seemed to know a GOP wave was coming, and by the time Election Day rolled around, the GOP lead on the generic ballot turned out to overstated in quite a few of the later samples. But what’s interesting is how the polls indicating a GOP tsunami didn’t impact the conventional wisdom within Washington. The GOP’s gain of 63 seats — a final majority of 242 seats — was well beyond the total predicted by Politico’s John Harris and Jim Vandehei (224), NPR’s Ken Rudin (219), Arianna Huffington (228), and CNN’s Candy Crowley (223). This is not to argue a crazy conspiracy among the Washington crowd, just to point out that this year, for some reason, the polls didn’t influence the Beltway expectations — why, it’s almost as if poll results showing good news for Democrats are taken more seriously than ones showing good news for Republicans.
Then of course, you have the individual pollsters who sometimes go . . . well, haywire. This is from my piece about Zogby, who became the liberals’ pollster of choice in 2002 and 2004:
In 2002, his final polls were pretty lousy. In Minnesota, Zogby predicted Democrat Walter Mondale over Republican Norm Coleman by 6 points; Coleman won by 3. In Colorado, Zogby picked Democrat Ted Strickland over GOP incumbent Wayne Allard by 5; Allard won by 5. In Georgia, Zogby picked Democrat Max Cleland over Republican Saxby Chambliss by 2; Chambliss won by 7. In Texas, Zogby’s final poll had Republican John Cornyn over Democrat Ron Kirk by 4 points; Cornyn won by 12. Zogby’s final poll in the Florida gubernatorial race had Jeb Bush winning by 15, but only three weeks earlier he had Bush winning by only 3. Bush won by 13 points.
Late afternoon on Election Day [2004] — awfully late for a final call — Zogby predicted that Kerry would win Florida, Ohio, Iowa, and New Mexico (0 for 4!) and get at least 311 votes in the Electoral College, while Bush was assured of only 213. (The remaining 14 electoral votes were too close to call.)
There’s no other way to say it: The Big Z’s final polls were garbage. His final poll had Colorado too close to call; Bush won by 7 points. He had Florida by a tenth of a percentage point for Kerry and “trending Kerry”; Bush won by 5 points. Zogby had Bush winning North Carolina by 3; the president won John Edwards’s home state by 13. Zogby had Bush leading Tennessee by 4; the president won by 14. Zogby called Virginia a “slight edge” for the GOP; Bush won by 8. In West Virginia, Zogby predicted a Bush win by 4; the president won by 13. And in the vital swing state of Wisconsin, Zogby had Kerry up by 6; the final margin was 1 point.
Zogby’s dramatically far-off results were, I would argue, fueled by a combination of hubristic overconfidence in his own ability to read the mood of the electorate and the desire to tell his biggest fans what they want to hear. I’ll let you conclude if you think that description might apply to any other pundit you see cited a lot these days — including myself.
Besides pollsters seeing what they want to see, we must recall the fairly recent example of Research 2000, which may not have actually conducted the surveys that it announced to the world. Here’s a good summary of that scandal:
It came after Daily Kos published a statistical analysis of Research 2000′s polls that alleged a series of statistical anomalies among the results. That analysis led Moulitsas to conclude that the weekly poll Research 2000 had conducted and run on Daily Kos during 2009 and 2010 “was likely bunk.”
Moulitsas added that Ali had “refused to offer any explanation” for the anomalies or turn over raw data as requested. Daily Kos lawyer Adam Bonin vowed to “file the appropriate discovery requests” in order to determine whether Ali had fabricated data.
In a rambling public response published last July, Ali characterized “every charge” made by the Daily Kos lawsuit as “pure lies, plain and simple.” He promised that “the motives as to why Kos is doing it will be revealed in the legal process.”
But by agreeing to a settlement, Ali leaves open the question of whether his data were in fact fabricated.
The same July statement also included a comment that raised eyebrows among pollsters (typos in original):
Yes we weight heavily and I will, using the margin of error adjust the top line and when adjusted under my discretion as both a pollster and social scientist, therefore all sub groups must be adjusted as well.
After sending that statement, Ali disappeared from public view. Attempts to contact his email account temporarily bounced, his Twitter account went silent and the Research 2000 website started redirecting to a Wikipedia entry on opinion polls. Ali started posting again to his Twitter account two weeks ago, although he has so far not mentioned either the lawsuit or his polling business.
Now, not every pollster is making up their results; probably none of the polls we read about today are made up of whole cloth. But this case suggests that the most paranoid scenario — a pollster not really collecting data, just pretending to and telling the client some combination of what they want to hear and what sounds realistic — can happen.
I mention all of this because I hear from a lot of readers — throughout this past weekend, in fact — with some variation of “EEK! X poll shows my candidate down!”
Well, your candidate may be down. But you should know better than to panic over a poll, and you should know that there’s nothing anyone could or should be telling you to make you stop being as active as you are in these final hours. Also, you should be checking the samples, too see if the partisan breakdown makes sense to you. If the percentage of Democrats in the sample is higher than the percentage of Democrats in the 2008 exit polls, some skepticism is warranted.
That’s how you find CNN releasing a poll Sunday night that has it tied, 49 percent to 49 percent, with Mitt Romney winning independents by 22 points, 59 percent to 37 percent, but “among those likely voters, 41% described themselves as Democrats, 29% described themselves as Independents, and 30% described themselves as Republicans.”
If the electorate is D+11 Tuesday, Romney’s doomed. If Romney’s winning independents by 22, he’s winning in a landslide.
Just this weekend I heard (for the first time ever) one pollster admit that the biggest problem this election was the number of people refusing to participate. (>5% in many polls)
When polls were created many decades ago everyone had a landline phone and they always answered.
Now a high percentage have ONLY a cell phone and many people screen their calls to the answering machine
I know when pollsters call me I have told them no, I didnt want to participate.
I think these non-participaters are goign to be for Romney more likely than not.
I agree with your statement. I have recieved so many calls starting at 8 in the AM and ending at 9PM and not once have I answered the phone (caller ID is wonderful) How many people do the same?
I think you are right on that point.
ON THIS: "Now a high percentage have ONLY a cell phone""..
Consider how many of those Cell phones belong to very young teenagers and under.
If those kids are answering the calls, they would just be playing with the pollster, but yet counted by the Dems. lol
I think you are right on that point.
ON THIS: "Now a high percentage have ONLY a cell phone""..
Consider how many of those Cell phones belong to very young teenagers and under.
If those kids are answering the calls, they would just be playing with the pollster, but yet counted by the Dems. lol
It’s been my general theory that Obama voters have time to waste.
Romney voters are busy and value constructive use of limited time.
If my theory is has any merit, then Romney voters are being underpolled.
Being in Florida, we get polled over and over and over. We have finally just said ‘no’ and hang up or don’t answer the landline, since we have no caller ID.............
While many talk about the problems of getting demographics right, party affiliation right, etc., presidential polling is far easier than survey research due to their being one and only one question and that question being clearly defined.
Do you support Obama or do you support Romeny or do you support Johnson/Goode/etc?
It is a clearly defined choice, so that means you can get a fairly accurate view of sentiment with a relatively small sample of a large population.
I hang up on all political phonecalls, polls, GOTV, just don’t want to deal with them. I made the mistake of doing a poll once, 20 minutes out of my life then apparently I got flagged as “cooperative” because I got more poll phonecalls every night for weeks. Never again.
There’s a linear equation that could define the model.
Something that would involve party affiliation, age, income level, turnout expectation, expectation to respond to polling calls, landline responder, cellphone responder etc.
Pollsters are smart guys. There’s big money to be made. Why can’t they stay ahead of this?
My answering machine speaks out the caller ID. If I don't recognize the number, I let the machine take it.
I've made a habit of answering polling calls the last few weeks, declaring that I'm a young, black female Democrat who is not voting for Obama.
My husband gets upset... But I can feign obamamania. It’s none of anyone’s business who I vote for. I would tell them I am an independent who voted for Obama. Unfortunately, I never get asked. Why would I ever, ever tell the pollsters the truth. I think I am not alone, based on 2008 exit polling.
There are hard demographics like race, sex, age and the like. Then there are soft demographics like party affiliation.
Pollsters will stick to models based on hard demographics because it’s the safe thing to do. Theres a point where the risk of being wrong is outweighed by the risk of other alternatives.
Start adjusting a model based on party ID for instance and you can expect an unpleasant phone call from David Axelrod and an indictment from Attorney General Holder. Better to just stick with the hard variables and eat the loss of rep from a wrong call.
I do.
And there are other dynamics: TIME. Dems are more likely to be government workers, students, unemployed, or on welfare. They literally live lives with more leisure time, and therefore are more likely to take the 15-20 minutes that a real poll (as opposed to a manipulative "push-poll") takes. Conservatives, which include a high mix of small business people and larger families are less likely to find the time.
AND, as a long-suffering battleground-stater (Colorado), I literally cannot be polled, because it is my practice to hang up on ALL calls that even HINT they might be political. At our house, I have received well over 400 calls this fall. I don't have time to hear all the robo-call nonsense (and can't imagine that they could possibly sway anyone's vote). Romney ESPECIALLY has been robo-calling, inoculataing many of his supporters from participating.
If a poll does not actively adjust for party affiliation, I don't see how they could be accurate. The two groups simply live very different lives.
this article is not really true ...he cherry picked exceptions...shabby for the source really
i do think Dem oversampling could make this the first gross error in polling since 1948
not since then have poll aggregates been off much but then they did not run a fraction of what they do now
talking about POTUS national...
1980...polls were slow to catch Magnus debate trounce
2000 ...late polls missed Bush DUI...not polls fault really
this time...I think Dem samplings based on 2008 is an error
and if you dig you can see the proof otherwise...not to mention where Axlerod is spending the money
but polls are not inaccurate much
the 2004 exit polls...another matter..done on purpose to some degree
and I would caution of alphabet polls or hired college polls
but the top 4-6 respected ones...even Gallup...are not that far off
I predict Romney 300 plus EV
and 5-8% pop vote margin and carries most states in play
come and get me if wrong..if I can log in
i mean really...this is just frigging awful here
this article is not really true ...he cherry picked exceptions...shabby for the source really
i do think Dem oversampling could make this the first gross error in polling since 1948
not since then have poll aggregates been off much but then they did not run a fraction of what they do now
talking about POTUS national...
1980...polls were slow to catch Magnus debate trounce
2000 ...late polls missed Bush DUI...not polls fault really
this time...I think Dem samplings based on 2008 is an error
and if you dig you can see the proof otherwise...not to mention where Axlerod is spending the money
but polls are not inaccurate much
the 2004 exit polls...another matter..done on purpose to some degree
and I would caution of alphabet polls or hired college polls
but the top 4-6 respected ones...even Gallup...are not that far off
I predict Romney 300 plus EV
and 5-8% pop vote margin and carries most states in play
come and get me if wrong..if I can log in
i mean really...this is just frigging awful here
this article is not really true ...he cherry picked exceptions...shabby for the source really
i do think Dem oversampling could make this the first gross error in polling since 1948
not since then have poll aggregates been off much but then they did not run a fraction of what they do now
talking about POTUS national...
1980...polls were slow to catch Magnus debate trounce
2000 ...late polls missed Bush DUI...not polls fault really
this time...I think Dem samplings based on 2008 is an error
and if you dig you can see the proof otherwise...not to mention where Axlerod is spending the money
but polls are not inaccurate much
the 2004 exit polls...another matter..done on purpose to some degree
and I would caution of alphabet polls or hired college polls
but the top 4-6 respected ones...even Gallup...are not that far off
I predict Romney 300 plus EV
and 5-8% pop vote margin and carries most states in play
come and get me if wrong..if I can log in
i mean really...this is just frigging awful here
Case in point, I was at the Social Security office this morning. Approximately 25 full-time people work there. The place is open 32 1/2 hours per week. Figure THAT out.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.