Embed Size (px)
Citation preview
#EUref 2016The challenges of telephone polling
Ben Page, Chief Executive
Ipsos MORI
@benatipsosmori
Our 2015 final poll – all parties less than 2% points away from actual: but Labour overestimatedHOW WOULD YOU VOTE IF THERE WERE A GENERAL ELECTION TOMORROW?
Source: Ipsos MORI Political Monitor/Ipsos MORI calculations from BBC reports
36%
35%
11%
5%8% 5%
Ipsos MORI 2015 final poll GB final result
Conservative lead = +1 Conservative lead = +6.5
CONSERVATIVE
LABOUR
UKIP
GREEN
LIB DEM
OTHER
37.7%
31.2%
12.9%
3.8%8.1%
6.4%
Base: 862 British adults who are registered and certain to vote 5th -6th May 2015
Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) 3
1. Newspaper weights (reduced claimed voters by c.3%, increased Con lead by c.4%)
We made two changes post-2015:
2. Stricter turnout filter, taking into account past behaviour (during campaign increased Con lead by 2-4%)
Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) 4
The EU referendum A “low information environment”?
Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) 5
Sampling in 2016Made a further adjustment to
sampling procedures in campaign polls to improve representation of politically-disengaged
• Added education as a quota (along with age, gender, region, social class, work status) to reduce impact of heavy weights on effective base size
• Not possible to exactly compare with previous polls, but estimate it resulted in a c3% swing to Leave.
EU referendum voting intention – base allHow will you vote on the question “Should the United Kingdom remain a member
of the European Union or should it leave the European Union”?
Source: Ipsos MORI Politial Monitor
Base: c1,000-1,600 British adults aged 18+
6055 56
60
49 50
4045 44
40
51 50
0%
10%
20%
30%
40%
50%
60%
Feb-16 Mar-16 Apr-16 May-16 Jun-16 Final
Remain Leave
Sods Law – our final pre-campaign poll
probably a rogue (but we have to publish
anyway)
EU referendum voting intention – using stricter post-2015 turnout filter
How will you vote on the question “Should the United Kingdom remain a member of the European Union or should it leave the European Union”?
Source: Ipsos MORI Politial Monitor
Base: c1,000-1,600 British adults aged 18+. June and Final polls based on all registered to vote
5551 53
58
4751
4549 47
42
5349
0%
10%
20%
30%
40%
50%
60%
Feb-16 Mar-16 Apr-16 May-16 Jun-16 Final
Remain Leave
EU referendum voting intention – with new turnout filter, and education quota
How will you vote on the question “Should the United Kingdom remain a member of the European Union or should it leave the European Union”?
Source: Ipsos MORI Politial Monitor
Base: c1,000-1,600 British adults aged 18+. June and Final polls based on all registered to vote
5248 50
57
4751
4852 50
43
5349
0%
10%
20%
30%
40%
50%
60%
Feb-16 Mar-16 Apr-16 May-16 Jun-16 Final
Remain Leave
Two other points – past vote profile continues to be same as BES one year on….
2015 recall %
Actual IM June-July 2015
IM Sept ’15 –May ’16
IM June –Nov 2016
BES May –Sept 2016
Conservative 37.7 38 39 41 41Labour 31.2 33 31 32 33LibDem 8.1 9 9 7 7UKIP 12.9 9 10 10 11Other 10.1 11 11 10 8
Perception of telephone volatility: actual average monthly vote share change since June 2015 = 1.7 points
Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) 10
The final pollThe choices we had to make –Scotland and 2015 error was prominent in our minds
Studies on referendums suggested opinion usually shifts towards ‘no change’, which we also saw in Scotland
Opinion change during referendum campaigns since 2000
‘No change’ option
Source: Alan Renwick, Reading University
The choice we had to make in the final pollReferendum voting intention (squeezed), including only those registered to vote
Unfiltered(all expressing an opinion)
49%51% 49%51% 51%49%
Old 2015 turnout filter(absolutely certain to vote)
New turnout filter(9/10 certain and habit of voting)
First time thechoice
between filters had
made more than a 1-point
difference
Remain Leave
Base: 1,592 GB adults aged 18+ interviewed by telephone, 21-22 June 2016.
12
The choice we had to make in the final pollReferendum voting intention (squeezed), including only those registered to vote
Unweighted
Remain Leave
50%50% 49%51% 51%49%
Old 2105 turnout filter(absolutely certain to vote)
New turnout filter(9/10 certain and habit of voting)
79% turnout
74% turnout
Base: 1,592 GB adults aged 18+ interviewed by telephone, 21-22 June 2016.
13
The choice we had to make in the final pollReferendum voting intention (squeezed), including only those registered to vote
Unweighted
Remain Leave
50%50% 52%48% 51%49%
FINAL PUBLISHEDHEADLINE FIGURES
New turnout filter(9/10 certain and habit of voting)
Adding “result is important” filter made projected turnout level more plausible (we thought) 70%
turnout74%
turnout
Base: 1,592 GB adults aged 18+ interviewed by telephone, 21-22 June 2016.
14
Document Name Here | Month 2016 | Version 1 | Public | Internal Use Only | Confidential | Strictly Confidential (DELETE CLASSIFICATION) 15
In summaryAfter 2015, we knew we needed to fix our samples and the way we interpreted the data.We think we dealt with under representation of disengaged But remain elements of judgement!Hindsight is great - but we couldn’t tell at the time –past referendums and the 2015 GE errors pointed in the other direction..c.50% of time pollsters make polls better (Scotland) –this was one of the other 50%!
