Upload
jakob-solomons
View
219
Download
0
Embed Size (px)
Citation preview
A Roper Poll on PollingSussman, 1985
• “Can interviews with 1,500 or 2,000 people accurately reflect the views of the nation’s population or is it not possible with so few people?”– Possible: 28%– Not possible: 56%– No opinion: 15%
For in-class use
• Write down questions you have about polling Consult your “neighbors” if you wish Bring along to the next class• Any questions except: how is it possible to represent a
big country with only a thousand people (I’ll cover that) How polling is done (statistical questions) How polling is carried out (practical questions) Questions about the questions used in surveys About the use of polls About their honesty/meaningfulness About pres. polls; exit polls; questionable polls
Why take (sample) surveys?
• Can’t afford to get information on entire population
• Timeliness—survey data can be gathered quickly
• Survey data is sometimes better than “hard” data (e.g., crime data)
• Can get survey data on attitudes (and behavior that would be hard to observe)
• A survey was even proposed to improve the decennial Census
Survey limited areas for very-hard-to-
reach individuals, extrapolate to other
areas to arrive at more complete
Census data
• Scuttled by the Reps; it probably would have increased the count in Dem areas
Why study surveys(in this course)?
• They are ubiquitous in politics Over 100 pre-election polls about the pres election (in one election year) Many more in gub, Sen, House races Constant barrage of poll data about issues Presidential approval is a standard news item (has been for 50 years)
Why study surveys (cont.)
• Surveys/polls are not confined to politics
More in marketing than elsewhere
Mall intercept studies
• Often well done; often not. You should be able to distinguish good from bad.
• Even good surveys are hard to interpret; you should be able to do to
Basic terminology
• Population: individuals that we wish to speak about
Adults in the United States
Likely voters in one state
UR graduates, 1995-2004
• Sample: individuals we gather info from
Individuals referred to as respondents
• Basic idea of sampling: We extrapolate from a sample to a population We take a sample—usually much smaller than the population—and make inferences about the population• Primary issues: How small or large a sample do we need? How do we sample (what individuals do we get information from)?
Nonprobability samples
• Haphazard sample
Sometimes called “random”
• Purposive sample
Select “representative” units
• Quota sample
Match population proportions (e.g., half
women, half men in a national sample)
• Problem with nonprobability samples:
Can’t assess accuracy (no statistical basis)• Lots of bias likely to occur in sampling:
Avoid certain parts of town
Avoid hard-to-reach respondents
British ex—interviews near tube stations
Select cooperative respondents (big no-no)• Solution: use probability (random) samples
Probability samples
• Simple random samples
• Stratified random samples
• Clustered random samples
• Systematic random samples
• Multi-stage random samples (combine elements of one or more of above)
Simple random samples
• Definition
Every element and every combination of
elements has an equal chance of being
selected; equivalently, every element has
an equal chance of being selected and
the selections are made independently.• Not true of all random samples We may
have time to discuss; otherwise, see text.
Simple random samples
• How can we represent so many with so few?
• Intuitive idea (explained in class)
• Formula for proportions (explained in class)
The great simplification
Confidence intervals, different sized samples
Confidence interval
Sample size (+ or – this percent)
2000 2.2
1500 2.5
1000 3.1
500 4.4
200 6.9
Based on the assumption of srs; proportion of .50.
Other random samples
• Most samples of people are not srs• One example only Stratify: divide the population in groups or areas; specify quotas; sample randomly within• Major reasons for using more complex samples: They are feasible, cheaper Significantly, and surprisingly, they are more accurate in some respects
A Famous FailureThe 1936 Literary Digest “Straw Poll”
• Postcard “ballots” succeeded in predicting the winner in 1920, 24, 28, and 32
• 10m ballots sent out to addresses drawn from every phonebook in US, rosters of clubs (e.g., Auto), city directories, lists of voters, and mail order & occupational data.
• 2.2m respondents predicting:– Alf Landon 57%– FDR 43%
• On election day, FDR won by a landslide (62.5%, all but MN & VT) What went wrong??
Literary Digest v. George Gallup:“straw” meets “science”
Some problems with the Literary Digest Straw Poll
• Sampling bias.• Response Bias.• Time. Early September.• New Deal Coalition: class polarization• DO NOT LET RESPONDENTS SELF-
SELECT!
George Gallup got it right in 1936 using “quota sampling”– Interview a quota from each demographic
group
Gallup Poll ResultsFinal poll (for winner) vs election
• 1936 FDR -4.81972 RMN 1.3• 1940 FDR -2.71976 JC -2.1 • 1944 FDR -2.11980 RR -3.8• 1948 HST -5.1 1984 RR 0.2• 1952 IKE -4.1 1988 GB 2.6• 1956 IKE 1.9 1992 WJC 6.0• 1960 JFK 1.11996 WJC 2.8 • 1964 LBJ 2.7 2000 GWB 0.1• 1968 RMN -0.42004 GWB c2.0
Gallup Poll Results (cont.)
• Averages
• 1936-1948: 3.68
• 1952-1978: 1.94
• 1980-2004: 2.50
• Source: Election Polls: Elections 1936-2000 (Gallup Poll report) http://institution.gallup.com/content/default.aspx?ci=9448
Surveys today
• Most are telephone surveys (since c1980) Use random digit dialing (RDD), not numbers from phone directories• Small, but increasing number are internet
surveys Biggest promoter is Harris Interactive (Roch) Methods, validity still not fully accepted• Focus groups supplement surveys• Some nonscientific polls survive (be wary)
Problems with Public Opinion Polls
• Occasionally people lie Rs don’t want to admit “unacceptable” behavior/attitudes Ex (1989): Gov. Wilder’s race in Virginia• Attitudes don’t always match behavior Example: 1930s study of hotels• Individuals often give ill-informed, inconsistent responses (next slides)
Ex: R’s are ill-informed
• 1948 Socialist party’s presidential cand. 21%• 1966 Incum. House cand. in their district 46%• 1970 Know the pres is limited to two terms 67%• 1972 Know China to be communist 63%• 1977 Know meaning of no-fault insurance 31%• 1987 Can name Chief Justice of the SCt 8%• 1987 Know Bill of Rights is part of Const. 41%• The list could go on—and on• New study (1/31/05): 75% of high school
students think flag-burning is illegal
R’s give inconsistent responses
Problems (cont.)Question wording matters
• Do you think the United States should allow public speeches against democracy?
• Do you think the United States should forbid public speeches against democracy?
One might expect:
% “allow” = % “don’t forbid”
Forbid/Don’t allow example (cont.)
• Results of three surveys using these Q’s
Public speeches against democracy
Don’t
Allow Forbid
1940 25 46
1974 56 72
1976 55 80
Problems (cont.)Question order/context can matter
• Survey asked whether Soviet journalists should be allowed to move freely, report on U.S. (Result: low % yes)
• Then asked, first, a question about U.S. journalists in the Soviet Union. (Result:
Much higher % say Soviet journalists should be free in U.S.
Problems specifically of election polls
• Predicting turnout
Biggest problem
• Undecideds
A problem for low profile elections
• Timing
Early polls can be quite inaccurate
Pollsters now work until election day.
Why believe polls?
• With all these sources of error, how can we believe polls actually work?
• With respect to technical matters: Polls match population data quite well. Repeated polls give consistent results. Polls do a good job—generally—of predicting elections (if not over- interpreted). Surely better than nonscientific methods
Why believe polls? (cont.)
• With respect to substantive matters:
Aggregate opinion is highly meaningful
even if individual responses are
problematic
We can often identify, overcome biases
Polls do a good job—generally—of
predicting elections
Some “solutions” toproblems with polls
• Look at multiple questions Attitudes: A chord, not a note• Be esp careful with controversial issues• Look at responses over time Changes are significant even if absolute level is not fully meaningful• Don’t over-interpret results (easy with
presidential polls because a few % matter)