Embed Size (px)
Citation preview
4/15/2015
British Politics and Policyblogs.lse.ac.uk /politicsandpolicy/predicting-the-polls-april/
Each month, the team from electionforecast.co.uk compare new constituency polls as they arereleased to their estimates of what polls would show in those constituencies. This allows foran assessment of the accuracy of their modeling approach. More information about theoverall model can be found here. The results presented in this post use all constituencypolls released by Lord Ashcroft in April so far.
For the past few months, we have been comparing Lord Ashcroft’s constituency polls as they arereleased to our estimates of what such polls would show from the previous day. These comparisonsoffer a test of our model’s ability to accurately predict what polls would show in constituencieswhere there has not yet been a poll. In April, so far, Lord Ashcroft has released three separatebatches of constituency polls – the first on April 1st, the second on the 8th and a third batchyesterday (April 14th).
In the graphs below, we plot our estimates from the day before each release of the polls on the x-axis and the Ashcroft estimates on the y-axis. Note that, as for previous months, our model uses thedata before the application of the “spiral of silence” adjustment which reallocates surveyrespondents failing to give a preference to the parties they voted for at the last election.
Both the polls and our estimates include uncertainty; in the figures below we break these resultsout by constituency and plot ellipses capturing the joint margin of error around each point.Whenever these ellipses cross the dashed 45-degree line, we can say that our predicted voteshare range for a given party, in a given constituency, coincides with the range implied by LordAshcroft’s polls.
1st April
Camborne and Redruth
Cambridge
North Cornwall
North Devon
Sheffield, Hallam
St Austell and Newquay
St Ives
Torbay
1/32
8th April
Blackpool North and Cleveleys
Gloucester
Harrow East
Hove
Kingswood
Loughborough
Morecambe and Lunesdale
Pendle9/32
Pudsey
Stockton South
10/32
14th April
Cleethorpes
Crewe and Nantwich
Dover
Dudley South
Finchley and Golders Green
Harlow
Milton Keynes South
North East Somerset19/32
Rossendale and Darwen
South Ribble
20/32
Navigating through the plots suggests that we continue to accurately recover the current state of play in many of theconstituencies polled during April. We predict the “winning” party in 17 of 28 cases, with our erroneous predictionsconcentrated in the constituencies where the race is particularly close (for example, in Crewe and Nantwich wepredict a Conservative victory while the Ashcroft poll suggests a narrow Labour win).
To measure the magnitude of our prediction errors systematically, we compute a Euclidean distance between ourestimates and Ashcroft’s polls. Averaging across all constituencies polled in April thus far, the mean distancebetween our estimates and Ashcroft’s polls is 6.8 percentage points. Given the size of the polls and the uncertaintywe were calculating for our estimates, we would have expected it to be 4.6 percentage points. This suggests that inthese constituencies, as in others we have evaluated, we are somewhat overconfident in our estimates.
We can also compare the performance of our model to a simpler forecasting model such as uniform national swing.As we have mentioned before, there is no single correct method for calculating uniform swing, and we use three
29/32
versions that give UNS a good chance to perform as well as our own model. First, we calculate the England-onlyswing from our poll estimates for all of England, calculate the average swing implied by those, and then apply thataverage swing to each constituency’s 2010 results. Second, we can be kinder to the UNS model by calculating theaverage swing from our poll estimates using only the constituencies that Ashcroft polled this round. Finally, we cancalculate the average swing across the Ashcroft constituency polling itself, and use that to apply a uniform swing.The table below provides the average Euclidean error from each of these UNS-variants for each batch of polls inApril.
Note: Table measures Euclidean error between model predictions and measured Ashcroft vote shares in percentagepoints. The model with the lowest error for each batch is emphasized.
As the table demonstrates, the error associated with our predictions is significantly lower than either the national orconstituency-specific UNS models in all three batches. Our model also out-performs the Ashcroft-specific UNSpredictions in 2 out of 3 batches. The comparison with the Ashcroft-only swing is a stern test for any forecastingmodel, as it requires knowing exactly the average swing of the constituencies of interest.
Nevertheless, we can go a step further and investigate the structure of the errors we are making. In the plot belowwe investigate whether our model systematically over- or under-predicts the vote shares of particular parties. The x-axis of the plot measures the difference between the electionforecast.co.uk predictions and the Ashcroft poll voteshares for each party. This number is positive when our model predicts a higher share of the vote for a party in aconstituency than suggested by the Ashcroft poll. Negative numbers represent cases where our model under-predicts a party’s vote share in a constituency. The y-axis orders the parties by their absolute mean error such thatparties at the top of the plot are subject to greater average error than those at the bottom.
The graph reinforces our previous evaluations: on average, our model over-estimates UKIP vote share, and slightly
30/32
underestimates the vote shares of Labour and the Conservatives. While our predictions for the Liberal Democratsalso have a tendency to diverge from the Ashcroft polls, there is little evidence to suggest that these errors aresystematically pointing in one direction or the other.
Overall, these results largely agree with our previous analyses. We correctly predict the winning party in mostconstituencies but we remain a little overconfident in our estimates. Furthermore, there is some evidence to suggestthat we are systematically over-predicting UKIP at the expense of Labour and the Conservatives.
As a final point, and a reminder of why this is a difficult (but fun) election to predict , the map below shows all of theconstituencies that have been polled over the past year. Constituencies coloured in green have received at leastone individual poll, and those in grey have not. There is a reasonable amount of green on this map, but thereremains a lot of grey! With just 21 days until the election, we await further constituency polls and will continue toinclude new polling information as it is released.
Note: Figures accurate as of April 15 th.31/32
About the Authors
Jack Blumenau is a PhD candidate in Government at the London School of Economics.
Chris Hanretty is a Reader in Politics at the University of East Anglia.
Benjamin Lauderdale is an Associate Professor in Methodology at the London School ofEconomics.
Nick Vivyan is a Lecturer in Quantitative Social Research at the Durham University.
32/32
