Replies

Here is the pdf copied over -- for ref. It's easier to read in in pdf -- see link in my earlier post.

====

Field Research Corporation is an Equal Opportunity / Affirmative Action Employer
Embargoed for publication: Tuesday, September 16, 2003
A Different Take on “Why Polls Differ”
By Mark DiCamillo and Mervin Field
The Los Angeles Times on September 12 in reporting on its latest statewide Times Poll in the recall
election of Governor Gray Davis took the additional step of trying to explain to its readers why its
poll results differed from those of a Field Poll reported earlier in the week. David Lauter bylined
the story, “Why Polls Differ.”
The Times Poll, conducted September 6-10, reported that 50% of voters said they intended to vote
Yes to recall the Governor, while 47% were taking a No position. By contrast , The Field Poll
survey conducted September 3-7 found 55% intending to vote Yes and 40% on the No side.
There was basic agreement between the two polls with regard to their measures in the candidate
race to replace Davis should he be recalled. Each poll showed Democrat Cruz Bustamante leading
Republican Arnold Schwarzenegger by an identical 30% to 25% margin, although the two polls
differed in respect to the size of the vote for Republican Tom McClintock, who placed third in
both polls. The Times Poll pegged McClintock’s support at 18%, while The Field Poll had it at
13%.
In attempting to explain the reasons for the differences in the two polls in the recall election,
Lauter stated that due to each poll’s margin of error (+/-3 points applicable to the Times Poll and
+/-4.5 points for The Field Poll), it is possible the lowest range estimate of Yes side support for
the recall from The Field Poll could be overlapping with the highest range estimate of Yes side
support in the Times Poll, inferring that the findings of the two polls were not really so dissimilar.
He also cites the fact that while there was some overlap in the interviewing dates of the two
surveys, The Field Poll started earlier and the Times Poll ended later, and that this also could have
been responsible for some of the variation.
When comparing the results of two polls, the difference in sample size and timing of interviewing
can sometimes account for differences. However, in this case, it appears that other aspects of each
poll need to be considered before invoking these standard explanations.
2
Differences in ideological classifications
Instead of dwelling on these matters, Lauter focuses his attention on the fact that 38% of the voters
in the Times Poll identified themselves as conservatives, compared to 32% in The Field Poll. He
also points out that 34% of the Times Poll likely voter sample identified themselves as liberal,
compared to 25% in The Field Poll. In respect to the proportion of voters classifying themselves
as moderates (or middle-of the-road) in politics, it was 26% in the Times Poll and 43% in The
Field Poll.
He then infers the differences between the polls as being due to differences in the ideological
make-up of each poll’s likely voter sample. The problem with this argument is that the two polls
go about measuring voters’ political ideology in very different ways. For over twenty years, The
Field Poll has employed a question sequence devised by the University of Michigan in its National
Election Surveys. The approach asks voters the following set of questions to assess their political
ideology:
1. “Generally speaking, in politics do you consider yourself as conservative, liberal,
middle-of-the-road, or don’t you think of yourself in these terms.”
2. (IF CONSERVATIVE OR LIBERAL) “Do you consider yourself a strong or not very
strong (conservative) (liberal)?”
3. (IF DON’T THINK IN THESE TERMS OR NO OPINION) “If you had to choose would you
consider yourself as being conservative, liberal or middle-of-the-road?”
The results from these questions are then combined to place voters on a five-point political
ideology scale: strongly conservative, moderately conservative, middle-of-road, moderately liberal
and strongly liberal.
By contrast, it is our understanding that the Times Poll derives the political ideology of voters
through a single question, asking voters the following question, “How would you describe your
views on most matters having to do with politics? Do you generally think of yourself as very
liberal or somewhat liberal or middle-of-the-road or somewhat conservative or very conservative?”
The results of these two very different lines of questions apparently produce very different
characterizations of the political ideology of California voters. From our perspective, we are
inclined to believe that The Field Poll’s procedure produces a closer approximation of the actual
ideological leanings of Californians than the one employed by the Times Poll.
However, more germane to this issue is that differences between the Times Poll and The Field Poll
in the recall election can not be explained by differences in each poll’s estimates of the number of
conservatives and liberals, since each poll arrives at these percentages very differently. And, if
this were truly the explanation for differences in each poll’s recall election results, they would also
have been expected to produce diverging results in each poll’s findings in the replacement
election, and this was clearly not the case.
Racial/ethnic subgroup differences
The Times Poll reported that 71% of its sample were white non-Hispanics, and on the Davis recall
they divided 54% Yes and 43% No, an eleven point plurality on the Yes side. Among Latinos who
accounted for 11% of the Times Poll sample, the split was 53% Yes and 41% No, a twelve point
plurality on the Yes side. Thus, its poll showed that when the preferences of white non-Hispanics
and Latinos were combined, accounting for 82% of its likely voters, they divided approximately
54% Yes and 43 No, an eleven percentage point plurality on the Yes side.
When the state’s two major racial/ethnic subgroups combined produce an eleven-point plurality on
the Yes side, it would mean that the remaining 18% of the Times Poll sample had to have had
dramatically different preferences for the overall total to be 50% Yes and 47% No.
It is a simple arithmetic exercise to calculate that the unreported 18% in the Times Poll had to be
against the recall approximately two to one (66% No to 33% Yes) to justify the overall finding of
the Times Poll.
While it is not unusual for a small voting segment in a poll to divide very differently from larger
majority segments, such a result would have been a significant poll finding.
While the sample sizes of the individual components of this 18%, which presumably comprised
black/African-Americans, Asians and voters of other racial and ethnic groups, were probably too
small to produce reliable poll estimates, the voting preferences of the combined 18% could have
and probably should have been reported, particularly since they apparently had such a dramatic
impact on the overall poll results. The Times made no reference to this fact.
In the absence of any reporting by the Times Poll as to the relative sizes of their black/African-
American and Asian subgroups and how they divided in their voting preferences on the recall, we
offer these observations for consideration. The September Field Poll found that 6% of the likely
voters in its sample were black/Africans-American voters and 7% were Asians/others. Last year,
in a statewide exit poll conducted by the Times, it was found that 4% of all voters in that election
were black/African-Americans and 6% were Asians.
Why did the size of the unreported racial/ethnic subgroups in latest Times Poll amount to 18%,
when according to its own exit poll, blacks and Asian voters combined comprised just 10% of all
voters in the last general election? Did the Times Poll sample include a proportionate number of
black/African-Americans or a disproportionately large number, whose inclusion, due to their
strong opposition to the recall, could have skewed their poll results? And, what about the
preferences of Asian voters, who historically tend to be more divided in their voting preferences on
partisan matters? If there were imbalances in these minority group cells, were they weighted to
bring them into conformity with historical voting patterns? If not, why not?