Online experiment:

Data and Method

Media in Context and the 2015 General Election:

How Traditional and Social Media Shape Elections and Governing

(ES/M010775/1)

University of Exeter

What has an online experiment done for us?

Survey questions help estimating exposure to the specific media

sources, which can then be linked to the content of that media

Online experiment help examining causal linkages between media

exposure, information seeking and attitudes in constituencies with

varying contexts

Together they provide a robustness check on the findings from

observational data and allow detecting media effects and

estimating their size, as well as under what conditions they emerge.

11/09/2015

2

Design of the experiment

Online panel

Competitive (marginal) constituency

Guardian Telegraph Local/regional Control

Safe constituency

Guardian Telegraph Local/regional Control

11/09/2015

3

Wave 1-2 – 3 exposures a week followed by a survey & W3 – one shot exposure + a survey

Seat type by Wave

11/09/2015

4

All

Wave 1 Wave 2 Wave 3

2010 GE 2015 GE 2010 GE 2015 GE 2010 GE 2015 GE

Marginal 1297 32.0% 20.4% 32.8% 21.2% 33.1% 21.6%

Safe 2729 68.0% 79.6% 67.2% 78.8% 66.9% 78.4%

4026 3024 3279 1830

Treatment groups be Wave

Wave 1 Wave 2 Wave 3

Guardian marginal 227 7.5% 259 7.9% 136 7.4%

Telegraph marginal 247 8.2% 263 8.0% 146 8.0%

Regional marginal 246 8.1% 281 8.6% 163 8.9%

Control marginal 248 8.2% 272 8.3% 160 8.7%

Guardian safe 516 17.1% 546 16.7% 300 16.4%

Telegraph safe 496 16.4% 558 17.0% 297 16.2%

Regional safe 526 17.4% 553 16.9% 307 16.8%

Control safe 518 17.1% 547 16.7% 321 17.5%

3024 3279 1830

11/09/2015

5

Treatments

Panel members were randomly assigned to four treatment groups

the week before the General Election.

Three of them were exposed to different political news stories –

covering “horse race” and policy issues – from different newspapers

– The Guardian, The Telegraph, or relevant regional newspapers - 3

times a week treatment (before and after the election) and a ‘one

shot’ treatment (after the election). The fourth – control – group will

be exposed to “placebo” (non-political) news stories.

All treatment groups completed post-treatment online surveys

(Waves 1-3).

11/09/2015

6

Examples of treatments for W1-2

11/09/2015

7

Treatments for Wave 3

11/09/2015

8

Sample vs. UK population: Demographics

Variable Experimental subjects General population

Average age (in years) 53.22 47.78

Female (%) 48.31 50.89

White (%) 95.44 86.84

Education: GSCE (or equivalent) or below (%) 26.11 52.34

Education: A-levels or equivalent (%) 13.25 12.12

Education: University degree or professional

qualification (%)49.04 27.12

Employed full-time (%) 32.87 38.56

Employed part-time (%) 14.03 13.70

Unemployed (%) 5.48 4.41

Household income: < £20,000 31.76 30.00

Household income: > £80,000 4.26 10.00

11/09/2015

9

Sample vs. UK population: Regional distribution

Variable Experimental subjects General population

Region: East (%) 9.02 9.53

Region: East Midlands (%) 8.36 7.39

Region: London (%) 8.85 13.32

Region: North East (%) 5.14 4.23

Region: North West (%) 11.69 11.49

Region: South East (%) 15.03 14.07

Region: South West (%) 10.44 8.62

Region: West Midlands (%) 9.02 9.13

Region: Yorkshire and The Humber (%) 9.51 8.61

Wales (%) 5.03 4.99

Scotland (%) 7.92 8.63

11/09/2015

10

Sample vs. UK population: Voter preferences

Variable Experimental subjects General population

Election: Conservative voters (%) 36.94 36.81

Election: Labour voters (%) 29.14 30.44

Election: Lib. Dem. voters (%) 9.34 7.88

Election: UKIP voters (%) 14.91 12.65

11/09/2015

11

Party holding the participant's seat after the 2010 GE

Con Lab Lib Dem SNP

Plaid

Cymru Green Other

Wave 1 % 51.8 38.1 8.7 0.7 0.3 0.3 0.1

Wave 2 % 51.5 38.5 8.4 0.9 0.2 0.3 0.2

Wave 3 % 51.0 38.8 8.9 0.6 0.3 0.3 0.2

11/09/2015

Party holding the participant's seat after the 2015 GE

Con Lab Lib Dem SNP

Plaid

Cymru UKIP Green

Wave 1 % 56.9 33.8 1.3 7.3 0.3 0.1 0.3

Wave 2 % 56.4 34.5 1.3 7.3 0.2 0.1 0.3

Wave 3 % 56.6 34.3 1.0 7.5 0.3 0.1 0.3

Survey questions (all waves)

Socio-demographics

Political engagement

Political preferences (voter preferences, like/dislike of

parties/party leaders, closeness to political parties)

Political efficacy and interest in politics

Engagement with and recall of political information

Issues positions

11/09/2015

13

Survey questions – altering W1-3

Political knowledge (e.g., % of women MPs) – W1

Expectation of the GE outcome (nationally and in the constituency) – W1-2

Expectation of the closeness of the GE and its outcomes – W1

Likelihood to vote – W1

Media endorsements – W2-3

Trust in media – W2

Voted/not voted in the 2015 GE and party choice – W2-3

Assessment of the election outcomes – W2-3

Assessment of the previous/newly elected Government performance – W3

Assessment of the legitimacy of the electoral outcome and the decisiveness of the Conservative majority in 2015 – W3

11/09/2015

14

Linking the data to other data sources

Statistical matching - integrating two or more data sources (usually data from sample surveys) referred to the same target population (D’Orazio, 2015).

Statistical matching implies the matching of similar rather than identical units is not only acceptable but is expected.

Statistical matches are made on the basis of similarity of characteristics rather than uniquely identifying information as is the case with exact and probability matching (ONS, 2004).

See D’Orazio et al., 2006b; Singh et al., 1993 for more details11/09/2015

15