Posted on 09/26/2012 12:18:38 PM PDT by smoothsailing
September 26, 2012
Matthew Sheffield
With no manufactured outrage to hammer Mitt Romney at the moment, liberal journalists are now eagerly touting a series of polls which appear to show President Obama pulling away from the GOP nominee in several key states.
Unfortunately, these polls are relying on sample sizes which are skewed tremendously leftward with far more Democrats than Republicans and as such, they are unlikely to be good predictors of actual Election Day turnout. Do the pollsters themselves actually believe in their own sample sizes though? At least one appears not to.
Interviewed last month by conservative talk show host Hugh Hewitt, Peter Brown, assistant director of the Quinnipiac polling operation was particularly squeamish about sampling under tough questioning from Hewitt about a poll which Quinnipiac had released showing Democrats with a 9 percentage point advantage in the state of Florida.
In the conversation, Brown defended Quinnipiac’s sampling techniques but admitted that he did not believe that Democrats would outnumber Republicans to that degree in Florida come November. Pressed by Hewitt, the pollster said he believed that was a “probably unlikely” scenario. Instead, Brown kept saying that he thought his poll was an accurate snapshot of reality at the time.
“What I believe is what we found,” he insisted while also touting his organization's record of polls closer to actual elections.
Unfortunately, this cavalier attitude toward accuracy is actually widespread throughout the entire polling industry. As NewsBusters noted in June, exit polls, which rely on far larger sample sizes than those conducted by Quinnipiac and others have long been known to oversample Democrats, sometimes even drastically. Sadly, the awful record that many pollsters have is something that most people barely know anything about. As such, it is one of the media’s “dirty little secrets” since Americans certainly won’t hear about it from the press.
Despite not believing that Democrats would have a 9-point advantage, Brown defended his organization, claiming that he and his colleagues were not intentionally trying to skew their sample size:
“We didn’t set out to oversample Democrats,” he protested. “We did our normal, random digit dial way of calling people. And there were, these are likely voters. They had to pass a screen.”
But what if that screen is simply not enough? The 2012 presidential election is unlikely to have an electorate which is similar to the ones before it. In the 2008 election, young and black voters turned out in record numbers and voted in even higher percentages for Obama. As specific surveys of these two voter groups have shown, however, both are dispirited this time around and are less likely to turn out for Democrats.
This point is particularly crucial given that the electorates in the years following 2008 have been much more Republican skewed. It could be argued that these were off-year elections and thus less likely to have blue-collar and college kid Democrats turn out to vote but ultimately no one knows today what the party breakdown will be November 6.
That’s why it’d be best for pollsters like Peter Brown to double-check their work the way that Scott Rasmussen does against a running party ID poll, especially considering by Brown’s own admission that Quinnipiac’s process for determining who will actually vote is “not a particularly heavy screen.”
A partial transcript of this highly illuminative interview is provided below courtesy of Hewitt show. Please see this link for the complete discussion. (Hat tip to Da Tech Guy who has more on the sampling controversy.)
HUGH HEWITT: Why would guys run a poll with nine percent more Democrats than Republicans when that percentage advantage, I mean, if you’re trying to tell people how the state is going to go, I don’t think this is particularly helpful, because you’ve oversampled Democrats, right?
PETER BROWN: But we didn’t set out to oversample Democrats. We did our normal, random digit dial way of calling people. And there were, these are likely voters. They had to pass a screen. Because it’s a presidential year, it’s not a particularly heavy screen.
HEWITT: And so if, in fact, you had gotten a hundred Democrats out of a hundred respondents that answered, would you think that poll was reliable?
BROWN: Probably not at 100 out of 100.
HEWITT: Okay, so if it was 75 out of 100…
BROWN: Well, I mean…
HEWITT: I mean, when does it become unreliable? You know you’ve just put your foot on the slope, so I’m going to push you down it. When does it become unreliable?
BROWN: Like the Supreme Court and pornography, you know it when you see it.
HEWITT: Well, a lot of us look at a nine point advantage in Florida, and we say we know that to be the polling equivalent of pornography. Why am I wrong?
BROWN: Because what we found when we made the actual calls is this kind of party ID.
HEWITT: Do you expect Democrats, this is a different question, do you, Peter Brown, expect Democrats to have a nine point registration advantage when the polls close on November 6th in Florida?
BROWN: Well, first, you don’t mean registration.
HEWITT: I mean, yeah, turnout.
BROWN: Do I think…I think it is probably unlikely.
HEWITT: And so what value is this poll if in fact it doesn’t weight for the turnout that’s going to be approximated?
BROWN: Well, you’ll have to judge that. I mean, you know, our record is very good. You know, we do independent polling. We use random digit dial. We use human beings to make our calls. We call cell phones as well as land lines. We follow the protocol that is the professional standard.
HEWITT: As we say, that might be the case, but I don’t know it’s responsive to my question. My question is, should we trust this as an accurate predictor of what will happen? You’ve already told me there…
BROWN: It’s an accurate predictor of what would happen is the election were today.
HEWITT: But that’s, again, I don’t believe that, because today, Democrats wouldn’t turn out by a nine point advantage. I don’t think anyone believes today, if you held the election today, do you think Democrats would turn out nine percentage points higher than Republicans?
BROWN: If the election were today, yeah. What we found is obviously a large Democratic advantage.
HEWITT: I mean, you really think that’s true? I mean, as a professional, you believe that Democrats have a nine point turnout advantage in Florida?
BROWN: Our record has been very good. You know, Hugh, I…
HEWITT: That’s not responsive. It’s just a question. Do you personally, Peter, believe that Democrats enjoy a nine point turnout advantage right now?
BROWN: What I believe is what we found.
We are speaking of the difference in vote totals of each party ~ not the total percent of the total vote.
Thank you for your insight.
No, my math is not wrong. I simply reported the comparable statistics from the last Presidential election.
Simply dividing the difference in the two vote totals by the total number of votes don't tell you anything about the difference!
Yes, it does. The difference is simply expressed in a proportion of the entire sample.
Look, I see your point. You can compare the two subsets to each other, instead the entire group, and come up with a different percentages. But, that's not how this difference was reported. It is the percentage of the entire sample, not a percentage of any subset.
You can't calculate it your way, then say that it means that a percentage calculated the other way (and the way everyone else is doing it) is comparable. If you know as much about statistics as you claim, you should know better.
Personally, I don't think that you can just adjust a poll according to self-reported political affiliations. However, I think it's both intellectually and ethically dishonest to report a poll as "correct" or "accurate" when there is such a clear imbalance in the sample.
Pollsters are just not willing to admit their sampling methodologies aren't really random. If they did, they would be out of business. These days, telephone poll respondents are effectively self-selecting. Between caller ID and call screening, most poll responders know exactly who they will be talking to before they even pick up the phone.
When you add in the different behaviors of the major voting groups (especially on the weekends), the possibility of intimidation (do I really know who is calling me for my opinion?), and telemarketers using the poll/survey exclusion to get their foot in the door, there is simply no way to get a genuinely random sample.
I'll also note that the party identification that I cited above is also suspect. It was derived from exit polls, and those are also self-selecting.
Pardon me but I got a couple of thoughts on this matter.
Your reputation precedes you as far as I’m concerned. You are always involved in some dispute with another Freeper or some such.
HOWEVER....this time I agree with you. I think to properly reflect the populace that the polls should reflect , oh I dunno, maybe the % Dem/pub turnout from the prior election.
HOWEVER again, jeez louise, I once worked for five years on a job requiring specific knowledge of samples and sample sizes....quality control to be exact.
I learned then and still now that sample sizes are fairly accurate. Gotta find the coveted “mean” and all that.
But come on, today all these polls come out showing Obamer running away with this thing in swing states. That’s just not believable I say based on my experience with statistics. It’s not likely at all that polls would suddenly change so dramatically overnight.
Still and so, hey, if the American people re-elect Obama I have decided to accept their choice and not grouse about it.
Do I see it as the end of America as I know it? Yes.
But we have a majority rule and so the majority shall rule.
I believe with all my heart America will make the right decision as regards this election.
But if not I will stoicly accept the rule of the majority.
Unhappily, to be sure. But what shall be, shall be.
“Personally, I don’t think that you can just adjust a poll according to self-reported political affiliations.”
__________________________________
That might be singularly the stupidest thing I’ve ever read on FR regarding polls. Voter turnout is NOT random!! One more time invade you we’re unable to let it sink in the first time. Voter turnout is NOT random.
Usually, I ignore personal attacks like this. You should know better. And if you don't dial it back a notch, this will be my last response to you.
No, voter turnout is not random. But, you can't expect a telephone poll to predict it reliably, especially when the poll results are well within the margin of error.
Instead of taking my comment out of context (like the Obama Media Group does for Romney), maybe you should read the rest of my response.
Poll responders are not randomly selected, either. The "opportunity" may be random, but the choice to respond is not random. There's too many factors that essentially make a poll respondent self-selecting. So, you can't adjust the results of one self-selecting poll with the results of another self-selecting poll and say that it is "better".
However, as I noted in the part that you deleted, I think that it is ethically and intellectually dishonest to claim that a poll is "accurate" when it is clear that your samples have a significant number of outliers.
Either:
1) There’s a vast leftwing polling conspiracy
or
2) We are part of the slant.
Maybe the left has a weak turnout for BO, but also, Romney has a weaker base demonstrated daily on FR by people who’d rather teach the GOPe a lesson than send Obama packing.
#2 seems more and more likely.
We assume the left won’t turn out as well as 2008. But I think a lot of spite is going to materialize in the Laz-e-boys and popcorn sales when the puritanicals sit home and passively protest against republicans.
....”a guide to where a politician might want to spend scarce campaign resources.”
Please explain to me why Obamy is campaigning today in Ohio, while enjoying a 53% to 43% lead over Romney?
I have my own theory about what the polls are telling us and why even the pollsters don't know what they mean.
It's not all bad news, just confusing news.
As you know all political parties attempt to stir their followers to go vote for their candidates. Billions have been spent based on the idea that this stuff works ~ that you can campaign and raise your candidate's popularity, and more people will come out to vote for him. BILLIONS!
I have no doubt that's the way it is almost all the time. But usually the parties and their campaigns start with candidates of more than uncommon popularity. For example, Eisenhower was very popular. He had a large edge with the populace and could draw on Democrats for votes as well. Truman had even tried to get him to run as a Democrat. He'd won either way.
The others we know about were not in the Ike category of popularity, but they weren't like Charlie Manson either. Sure he had his following, but you get tired of the glow in the dark eyeball trick in a really short time.
Obama was highly popular before folks got to know him. Now he's down in the dumper.
What about the other guy? Romney ~ is he popular, or is he just the guy who managed to snag the nomination? What if BOTH major party candidates are perceived by most voters as not being all that popular, or even someone to respect?
Then, as the campaign progresses and we get to know both of these guys better their existing popularity could drop leaving more and more voters confused, upset and disturbed that they don't have a candidate in this race.
The pollsters should be detecting this in the number of non-respondents they encounter ~ those are the folks who hang up on the pollster right off, or refuse to answer questions, or who hector the poll caller, or even preach to them before hanging up.
None of these guys will tell you, but if it's going on, and there are fewer and fewer actual probable voters in the pool within which the sample is being taken, the results should, at some point, come to a resting point where any number of additional samples (calls to people) will not change the results beyond random chance ~
I think we have begun hearing that with ever increasing incidences of pollsters reporting ties, and once one guy does it the others will bravely come forward and they'll tell us the same thing.
This before the debates and without any real campaigning having taken place.
Does not look good
It is what it is. Compare A to B, not A to A+B
I’ll take #1 sprinkled with just a tiny pinch of #2.
Good point. We need to keep our principles, but take the long view. All this hand wringing isn’t going to produce a conservative majority.
Several reasons ~ it's on the way to Chicago. Michelle has friends in Cleveland. ........ just hundreds of things like that.
Then I suggest we name the gulags after you.
“It’s in a symposium i reffed the other day ~ the lead lecture was by Zogby ~ and he went right to comparing his approach to Rassmussen’s approach.”
Good grief are you freaking kidding me? Is this the same John ‘the Retard’ Zogby that just this week made the claim that Obama (a black, Marxist, pro gay marriage, failed president) has a significant lead among NASCAR fans??? WOW I say!! If you are stupid enough to follow this dipshit then you really have ZERO credibility.
Can't argue with that. They wanted to find Obama in the lead and did..
Everybody has a bad poll or two lying about ~ but say, tell us how you’d go about sampling specifically for NASCAR fans. Bet that problem is a bit more difficult to crack than you imagine.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.