‘Right Side of History,’ or Primed to Say Yes? Same-Sex Marriage? It matters how question is asked

National Review ^ | 08/20/2013 | Mark Regnerus

[SeekAndFind]

Reports of recent nationwide polls about same-sex marriage would seem to put those who oppose the idea squarely in the minority, with many who formerly opposed it apparently fearing that they would find themselves socially on the “wrong side of history,” akin to Bull Connor and his Birmingham police force or the spectators who jeered at James Meredith as he walked to class at Ole Miss in 1962.

Emotional slogans are no doubt effective, but they muddy social-scientific attempts to figure out just how popular the idea of same-sex marriage (SSM) really is in the American mind. Polling data certainly suggest that public support for SSM is increasing, and I affirm that that perception is accurate. But discerning exactly what people think about SSM — and how many of them think that way — is not as simple as a sound bite.

A recently released Rice University study on attitudes about same-sex marriage — and the absence of media attention accorded it — made me wonder about the science, and possible politics, behind the most commonly cited polls.

Rice sociologists Michael Emerson and Laura Essenburg analyzed data from a poll that asked a random sample of nearly 1,300 American adults — on two different occasions, in 2006 and 2012 — whether they agreed or disagreed with this statement: “The only legal marriage should be between one man and one woman” (see Figure 1). What’s the advantage to querying the same people six years later? Rather than simply mapping trends in the overall population — which is what most polls on the subject do — you can discern internal movement within people. That is, they change their minds, and the results show that they don’t always move in the direction of greater openness to same-sex marriage.

Here is what the Rice study’s authors say they discovered: First, they found less support for same-sex marriage than polls like Gallup and CNN tend to find. In fact, in 2012, 53 percent of those surveyed agreed that the only legal marriage should be between a man and a woman, while 13 percent sat on the fence, and 33 percent disagreed with the statement. Second, they detected no statistically significant change in overall sentiment on same-sex marriage over those six years. Third, some things did change — minds — and not all of them toward favoring same-sex marriage (see Figure 2). The authors write:

. . . when we look behind the overall numbers, we find that many people did indeed change their minds over the 6-year period. The most stable category was among Americans who agreed in 2006 that the only legal marriage should be between one man and one woman. About three-quarters (74%) who agreed with the statement in 2006 also agreed with it in 2012. Among those who disagreed with the statement in 2006, 61% also disagreed in 2012. What is surprising in light of other polls and the dominant media reports that Americans are moving in droves from defining marriage as one man and one woman to an expanded definition is the movement of people in the other direction as well, a fact missed by surveys that do not follow the same people over time.

The uncommon results of this study, when contrasted with most media reports on the matter, may be to blame for the silence observed about this release. It simply didn’t jibe with the dominant narrative of majority — and growing — support for same-sex marriage. It’s possible that the Rice study’s sample is more religious than average, given that religion was one of several topics the investigators were most interested in. (Topical interest, however, need not bias a sample if the survey contacts are conducted smartly.) Moreover, its youngest respondents were 18 in 2006 and 24 in 2012, so this survey misses out — just a bit — on the youngest adults in 2012, more of whom are no doubt on board with the shift in marital meaning. Yet what about the psychology of giving positive versus negative responses to surveys? The Rice survey is unique in that the positive response is one of support for traditional marriage rather than for same-sex marriage.

What do other surveys show, and how do they ask their questions? Gallup, the granddaddy of such organizations, regularly asks Americans, “Do you think marriages between same-sex couples should or should not be recognized by the law as valid, with the same rights as traditional marriages?” When Gallup did so just last month, 54 percent of those polled said “should” and 43 percent said “should not,” while 3 percent remained unsure. The results may be skewed by the fact that the negative response is the one favoring traditional marriage. Nevertheless, it would seem that SSM has solid support.

But Gallup continues to ask a question about the legality of “homosexual relations” before it asks about same-sex marriage, a technique known as “priming,” or preparing survey-takers for subsequent questions. In their book News That Matters, political psychologists Donald Kinder and Shanto Iyengar document how priming shapes respondents’ answers to subsequent questions, particularly where sentiments about a previous question spill over. Gallup asks whether respondents “think gay or lesbian relations between consenting adults should or should not be legal,” a question that most observers would assume is not even asked any more.

It turns out that Gallup did not always prime with a question on the legality of homosexual relations before asking about same-sex marriage. Back when it varied its practice — priming on some surveys and not others — support for same-sex marriage varied. When Gallup did not prime, support for SSM totaled, on average, 6 to 7 percentage points less than when it did. A few percentage points may not seem like much, until we recall last month’s Gallup survey: Swing 6 or 7 points in the other direction and you would bring the poll to near-equilibrium between supporters and opposers. Thus a majority of Americans might not — or at least not yet — actually support same-sex marriage. The Rice study did not prime its respondents, and it asked the question differently; the results show notably greater opposition than support.

It’s impossible to know if such priming continues to affect Gallup’s numbers today, because it no longer varies its practice of priming — it now always primes — even though the wisdom of asking about the legality of “homosexual relations” makes little sense in our post–Lawrence v. Texas era. So why does Gallup still prime its survey respondents in this way? Consistency? Perhaps, but varying the practice is a methodological safety mechanism.

The lack of clarity about polling extends to actual voting behavior as well. In 2010 Patrick Egan, assistant professor of politics and public policy at New York University, compiled ten years of polling data about same-sex marriage in states that had voted on same-sex-marriage ballot initiatives. He found that public-opinion polls consistently underestimated ballot-box opposition to SSM. Egan noted that “the share of voters in pre-election surveys saying they will vote to ban same-sex marriage is typically seven percentage points lower than the actual vote on election day.” Why? Egan doesn’t know.

One might suspect something akin to the Bradley Effect at work in polling on this issue. Los Angeles mayor Tom Bradley, an African American, lost the 1982 California governor’s race despite being consistently ahead in the polls going into the election. Scholarly assessments of why Bradley lost focused on social-desirability bias on the sensitive issue of race. In particular, a minority of white voters was thought to have offered inaccurate polling responses for fear that, if they stated their true preference — which emerged at the ballot box — pollsters would perceive them as racially motivated.

In other words, when sensitive issues are at stake, people may feel pressure to give pollsters answers that sound enlightened, politically correct, or free of any trace of “bigotry” — a term that has reemerged as a club in the debate over same-sex marriage. Egan, however, claims that social-desirability bias is not responsible for the gap between polling and ballot-box results in this case, a gap that remained squarely in place in the May 2012 marriage and civil-union referendum in North Carolina. However, the phenomenon was mostly absent in the four ballot initiatives last November, when the pro-SSM side was likely aided by the ballot initiatives’ being attached to a presidential election.

Other suspects are the words with which survey questions are constructed. When polling organizations include the term “rights” in their question — as do Gallup, USA Today, and CNN/ORC — support for same-sex marriage is elevated: Each found 54 to 55 percent in favor. Survey respondents appear to react positively to words like “rights,” “freedom,” and “benefits,” and negatively to words like “ban.”

Recognizing this, Quinnipiac University’s pollsters stick to a very generic and brief question: “In general, do you support or oppose same-sex marriage?” The last time they asked it, in late April 2013 — about 30 days after the High Court’s twin decisions — 45 percent of respondents reported support and 47 percent said they opposed. Eight percent were unsure.

And yet there are polls — such as the ABC News/Washington Post one last conducted in early March — in which, for no obvious reason, support for SSM runs 8 to 10 percentage points above where other polls seem to. Sampling, in the end, is a science, but a very human one.

What to conclude? First, American public opinion seems split nearly down the middle on same-sex marriage, once we account for priming, question-wording “bonuses,” and Egan’s observations of systematic underreporting of opposition. Second, the bad news for those who oppose legal recognition of same-sex marriage is that the overall, decades-long trend lines do not favor them, individual surveys aside. Battering one’s opposition with catchy memes and claims about right and wrong sides of history may be annoying, but it has been effective. Finally, many minds have not been made up. In the 2011 population-based New Family Structures Study survey, respondents were offered an “unsure” option when asked whether they agreed or disagreed with the statement “It should be legal for gays and lesbians to marry in America.” Almost one in four 18-to-39-year-olds took it. If nothing else, the Rice study reveals that such fence-sitters can move in either direction.

— Mark Regnerus is an associate professor of sociology at the University of Texas at Austin and a senior fellow of the Austin Institute.

[traderrob6]

Just ask: “are you for or against same sex marriage” period.

[tbpiper]

fearing that they would find themselves socially on the “wrong side of history,”

Apparently they're not afraid of finding themselves on the wrong side of God.

[SkyDancer]

My grandma told one once that she and granddad have been having the same sex for the past twenty years and she said frankly it’s no big deal.

[tbpiper]

fearing that they would find themselves socially on the “wrong side of history,”

Apparently they're not afraid of finding themselves on the wrong side of God.

[Travis McGee]

[driftless2]

I’ll never be for validating with the allowance of marriage a mental disorder like homosexuality. Sorry.

[Romulus]

If you are for contraception and/or divorce-and-remarriage, you already accept a definition of marriage as a union based entirely on personal taste and private sentiment, unmoored from questions of procreation or other natural law concepts that would make the state a stakeholder in the issue. Since this is what a majority of Americans have accepted for generations, there is no question they will happily accept a definition of same-sex “marriage”. It’s baked in the cake.

[Richard from IL]

Thanks for posting this.