Posted on 09/27/2012 7:05:36 PM PDT by SeekAndFind
After yesterday's post on poll trustworthiness, I started wondering whether there's any poll or model that's been consistently accurate over time and therefore worth watching down the stretch as a weathervane of where the race really stands. I e-mailed two experts whom I trust and put that question to them. Is there any steady signal they trust amid the cacophony of statistical noise? Anyone we can look to as a beacon in the darkness when the NYT drops its next D+10 sample of Utah or whatever on us?
Short answer: No, there's no one whom they count on to get it more or less right every time. Polling averages did well in 2008 and 2004 but not so well in 2000 and 1996. The first person I spoke to told me flatly that it’s not worth paying much attention to the numbers now because the assumptions being made about the composition of the electorate on November 6 differ too widely among individual pollsters to distill a truly useful average. That uncertainty is compounded by the fact that, with six weeks left until America votes, there's still an ocean full of potential "black swans" --- wonderful/terrible jobs reports, war with Iran, a new eurozone spasm, etc --- that could send the trendlines fluttering. (Team Romney told Rich Lowry they think their dip in Gallup’s tracker lately is due to one such black-swan moment whose effects are already fading.) Once we get to within a week or two of election day and pollsters’ assumptions finally start to coalesce, the polling averages will become more reliable as an indicator of where the race really stands. As my own addendum to that, I think we’re close enough to the first debate that there’s no point picking through polls until late next week at the earliest. Why worry about this week’s data when there’s a hugely important event that’s bound to affect the race right around the corner?
My other source had less to say about the reliability of polling averages generally than their reliability with respect to specific candidates. He told me that if you look at historical averages, you find that they underestimated Gore in 2000, Dole in 1996, and Bush 41 in 1992 — all of them dull, somewhat stiff candidates whom their respective bases weren’t thrilled about. Why would polls miss the mark on people like that? His theory is that pollsters pay lots of attention to voter enthusiasm but less attention to whether voters say they’re “certain” to vote, and in the case of candidates who aren’t beloved by their base, those two variables don’t match up especially well. There were plenty of Republicans who weren’t enthusiastic about Bush and Dole but who were nonetheless certain to vote for them in hopes of defeating the Democrat. Ditto for Gore vis-a-vis the GOP. (Kerry and McCain were also dim lights to their bases and the polls gauged their support pretty well, but in McCain’s case he had a huge shot of enthusiasm late from adding Palin to the ticket.) He thinks the same thing could be happening this year — essentially, pollsters are keying off of the Dems’ slight edge in “enthusiasm” and missing the fact that plenty of unenthusiastic Republicans will be at the polls anyway to vote for a guy who’s taken to citing RomneyCare lately as proof of his empathy. If that’s the case, then they’re lowballing Romney’s support. And in a tight race, that’s potentially a decisive error.
See? I am capable of writing a poll post that’s not hopelessly eeyorish. Although I’d be lying if I said this wasn’t how I felt when I saw those Gallup numbers yesterday. Oof.
After you do all the math and the calculating, don’t forget to add in the commons sense factor that tells you whether democrat turnout will be like it was in 2008, or much smaller than in 2008.
Dirty secret - no way to independently verify the accuracy of a poll 2 months from an election.
They can say any crazy thing they want. Who’se to gain say ‘em? Me,
http://www.unskewedpolls.com/
Unskewed polls is very funny.... it tries to balance the other polls that skew Democrat by a wide margin by SKEWING their polls to favor Republicans by 4.3% in 2012.
Not sure if this assumption is correct, but they are assuming that Rasmussen’s national sample is correct.
The only concern I have is this — does this 4.3% GOP advantage in 2012 reflect what you see in the BATTLEGROUND STATES?
Rasmussen’s poll is a NATIONAL POLL. However we are a nation that does not depend on popular votes, we depend on the ELECTORAL COLLEGE.
So, the next logical question is this — Does unskewedpolls.com assume that this 4.3% GOP national advantage in 2012 also reflect what is on the ground in Ohio, Florida, Virginia, Pennsylvania, Colorado and Iowa?
These states are where the ultimate decision will be made.
There is much work to do. Romney needs to talk details of his plans. I dont want to hear him say one more time that he knows what to do. Please tell us what that is!!!
Good plan of how Romney could turn things around can be found here.
http://www.realitybatslast.com/2012/09/27/how-romney-can-win-with-a-mandate/#comments
AS I have posted before, the real polls, that is the ones that are highly accurate, are something you will not see. At least not for many years.
I trained as a pollster in college, worked as one afterwords.
The polls you see in the media are all about either rallying the faithful or demoralizing the opposition or selling eyeballs.
The actual polls, that is the ones where the outcome actually resembles reality as closely as (lots of ) money and (woefully limited and stressed out) time permit. These polls are the ones the campaigns (or their ancillary staffs, usually funded by PACs) use to determine where to buy ads or otherwise apply resources. As Willard pointed out it’s the small percent in the middle that “matter” (in terms of where you spend money, and that, my friend is the only thing that matters in a campaign). You, me. Jim Rob, Anderson Cooper, John Sterwart, Rush... no one sees those ‘real’ polls except a very very small group around the candidate, they are crucial info. If you have $400K in a tri-county area, NOW do you spend it? That is what the $$$ and the genuine expertise are chasing right now. News organizations and the campaighns release “polls” all the time, but it’s all about psych ops and selling papers/website hits, the numbers are all about what will generate excitement (or despair).
My old buds are making the mortgage, college and boat payments for the next few years over the course of the next few weeks. And they care far more about being right than who wins.
I don’t know much about polling, but I do have experience with data analysis.
If your raw data is not random and you have to apply some modeling formula to correct your data, then the data is pretty much worthless. If polling data represented a random sample of the population it was representing, no correction would be necessary.
If you have a known quantifiable systematic error then you can correct for that error (i.e. you have a bad measuring sensor and you know it produces a constant error).
I think a good guess with good intuition can give as good a result as most of the pollsters. You know Obama is not going to do as well as 2008 and it is likely Romney will do better than McCain.
If Obama wins it will be by a closer margin than 2008. At this point, there is nothing other than questionable polling that says Romney will lose.
This fact makes a mockery of the claim their methodology is based on science.
The other day Rush had a list rating the accuracy of the various pollsters compared to the results of given elections.
I believe FOXNEWS was 13th. Most of the often quoted polls were way down the food chain.
He also said that many of them were using the RAT “weighting” bias pertinent to the 2008 election, and ignoring 2010.
How in the hell did they judge accuracy?
My old buds are making the mortgage, college and boat payments for the next few years over the course of the next few weeks.
And what are your old buds saying about this election...?
“And what are your old buds saying about this election...?”
“Those who speak don’t know, those who know don’t speak”, as Confucius said (maybe it was Lat Tzu).
That stuff is akin to the top secret marketing research done by Ford or Proctor and Gamble...
We would be focused on getting to the nitty gritty of which groups would vote for “our” candidate, regardless, and which groups would NEVER vote for our candidate no matter how much we spent. Identify those groups so you don’t spend an unnecessary dime on them, take the remainder and find out what media they consume, correlate that with likelihood of voting and cost of ad buys.... now you know how to spend precious dollars; believe me (and if you can’t trust an anonymous source on an internet discussion page who CAN you trust?) that data is guarded like any highly valuable data, you or I are not seeing it unless we are responsible for deploying resources!
The camps will release whatever numbers will either inspire people to vote or hopefully discourage people more likely to vote for the opponent to vote. The media will release whatever numbers will increase traffic... in no case the numbers ‘we’ see having anything to do with a result attained at great expense.
Boring and completely mercenary stuff. As to any ethos, more akin to a pilot who would strive to make the best landing possible regardless of their feelings for the candidate, or the surgeon who would of course do the best possible operation even if they would as soon the person they are operating on be hung immediately afterward :-)
They get information.
The classic standard is that with a 1500 response (that is all of those 1500 have answered EVERY question and a goodly number have been called back to verify their responses) you have a plus or minus three percent margin of error in the best possible case scenario. That means that your sample actually represents the population you intend (presumably, but not necessarily ‘likely”, or “undecided” voters’. The math behind that is pretty provable and verifiable; if you have a hundred thousand, or a million or a billion jelly beans *IF* your sample is representative of the whole THEN a sample of 1500 is the point of diminishing returns, you would have to have a MUCH larger survey to gain even a fraction of accuracy... Since a ‘perfect’ sample is pretty much impossible to take (or more accurately it is impossible to KNOW you’ve taken a perfect sample), interpreting the data, gauging it’s accuracy well enough to know where to throw the money, that is the skill. That’s what makes folks like Rove, Atwater, Morris and such the influences they are or were (actually it was as often wonks on their staffs).
By definition, if they got “no information” from them, they were not part of the sample. IF they got incomplete demographic data then there are the techniques I’ve alluded to above to turn the noise and static into the daily racing form :-)
The people who get paid the big bucks are the ones who can extract the information from the noise.
NONE of them are legitimate they are all propaganda.. or worse agitprop..
Even the ones that favor your “issue(s)”..
Even simple questions are not so simple..
Ditzes are manipulated by Polls, to everyone else they are entertainment..
Just vote your heart, ignore the polls. The polls are rigged to influence your vote by making use of the “herding instinct”.
Your post is a classic example of the experts fallacy.
You begin with a few cursory sentences about the fantastically complex subject of statistical sampling theory. This establishes you as an expert. Do no argue with me, I will bury you with my expertise.
Next you assert the expertise of the pollsters.
“The people who get paid the big bucks are the ones who can extract the information from the noise.”
You can’t “independently prove and verify” how they “extract the information from the noise” because unlike scientific sampling theory how they do it is a proprietary secret. But they are experts so we must assume they know what they are doing.
As further proof that their work is valid you give us the fact that highly paid campaign consultants - Rove, Atwater, Morris and such - spend vast amounts of money on poling and they wouldn’t do that if they didn’t get value for their money. Take that for fact since they are experts in their field.
Well here’s a statistic that is provable and verifiable about Rove, Atwater, Morris and such - in every election 50% of them fail to get their candidates elected, despite all the cash they waste on polling.
Thank you, I will do my own thinking about the subject.
Personally, I think the only polls that are relavant are those that translate their numbers to Electoral votes.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.