In Focus 2018: Campaign Evaluations in West Virginia ... Focus/In Focus 2018 Executive Summary.… · 14/05/2019 · Illinois, and Nevada had an energizing, neutral, or de-energizing

E X E C U T I V E S U M M A R Y

In Focus 2018: Campaign Evaluations in

West Virginia, Illinois, and Nevada

DATE

May 14, 2019

PRESENTED TO:

Susan Thompson Buffett Foundation

PRESENTED BY:

The Public Affairs and Media Research

Department and The Social Data

Collaboratory

NORC | IN FOCUS 2018: CAMPAIGN EVALUATIONS IN WEST VIRGINIA, ILLINOIS, AND NEVADA

EXECUTIVE SUMMARY | I

Table of Contents

Overview of Study Objectives and Approach ......................................................................... 4

Campaign Monitoring and Survey Research ....................................................................... 5

Twitter Analysis ................................................................................................................... 6

Key Findings ............................................................................................................................. 7

Survey and Campaign Monitoring: Summary of Findings ..................................................... 9

Research Question 1 ............................................................................................... 10

Similarities across states ....................................................................................10

Differences across states ...................................................................................11 West Virginia ..............................................................................................11 Illinois .........................................................................................................11 Nevada .......................................................................................................12








Twitter: Summary of Findings ................................................................................................ 19

Twitter: West Virginia Findings .......................................................................................... 19

Twitter: Illinois Findings ..................................................................................................... 20

Twitter: Nevada Findings .................................................................................................. 21

Twitter: Overall Abortion Narrative .................................................................................... 21

Overview of Twitter Data – Abortion Topic – Illinois, Nevada, West Virginia...................... 22

Characterizing Message Source .............................................................................. 25

Amount of Tweets by Source ................................................................................... 25

Average Twitter Activity by Source .......................................................................... 26

Engagement Overall and by State .................................................................................... 27


EXECUTIVE SUMMARY | II

Twitter Messaging Source: Takeaways .................................................................... 27

Twitter: Characterizing Message Sentiment ............................................................. 27

Twitter Message Sentiment: Takeaways .................................................................. 28

Twitter: Sentiment over time ............................................................................................. 28

Twitter Message Sentiment over Time: Takeaways ................................................. 29

Twitter: State-level Trends in Pro-choice Versus Pro-life Sentiment ........................ 30

Twitter Message Sentiment by State over Time: Takeaways ................................... 31

Twitter: Top National Hashtags ................................................................................ 32

Methodology............................................................................................................................ 34

Qualitative Campaign Monitoring ...................................................................................... 34

Collection and Analysis Processes .......................................................................... 34

Campaign and relevant organizations’ websites and Facebook pages ...............35

Campaign and relevant organizations’ emails ....................................................35

Debate transcripts ..............................................................................................36

Political advertisements .....................................................................................36

Campaign, candidate and relevant organization tweets .....................................36

Media coverage .................................................................................................37

Campaign contributions .....................................................................................37

Twitter Analysis ................................................................................................................. 38

Data Collection ........................................................................................................ 38

Comprehensiveness of the keywords.................................................................40

Twitter: Sentiment Classification .............................................................................. 41

Source Coding ...................................................................................................42

Source Sample ..................................................................................................43

Bot Filter ............................................................................................................43

Top terms and hashtags by source ....................................................................44

Identifying West Virginia Amendment 1 Predictive Terms ........................................ 44

State-Level Surveys .......................................................................................................... 44

AmeriSpeak Sample ................................................................................................ 44

TargetSmart Sample ................................................................................................ 44

Field Period, Respondent Screening and Weighting ................................................ 47

Analysis ................................................................................................................... 49

Voter Validation ....................................................................................................... 49

Appendix ................................................................................................................................. 50


EXECUTIVE SUMMARY | III

List of Exhibits

Percentage of candidates’ campaign content that was related to abortion .................................. 9

Considerable inconsistency among voters in party identification and abortion attitudes ............ 10

Republican pro-choice voters were more likely to cross party lines than Democratic pro-life voters ........................................................................................................................................ 11

Abortion ranked lower on the issue priority list and was more important for Democrats in Nevada and Illinois .................................................................................................................... 13

Almost no difference between pro-choice and pro-life voters in candidate personality traits they value ......................................................................................................................................... 14

Abortion was a higher priority for pro-life than pro-choice voters in West Virginia and Illinois .... 15

No difference between pro-choice and pro-life voters in campaign engagement ....................... 18

Weekly volume of abortion-related tweets in the United States ................................................. 22

Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia ....................... 22

Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia (per 100,000 population) ................................................................................................................................ 24

National and state-level trends of tweets on abortion (with Pearson correlations between the states and national level) .......................................................................................................... 24

Amount of abortion-related tweets by source type (among the tweets posted from Illinois, West Virginia, and Nevada)................................................................................................................ 26

Average number of abortion-related tweets per account by source type ................................... 26

Engagement: amount of tweets among other individual/person that are retweeting, quoting, or replying to a verified or influential user ...................................................................................... 27

Number of tweets categorized as having pro-choice sentiment, pro-life sentiment, or neither for each state ................................................................................................................................. 28

Valence of state-specific abortion-relevant tweets at each week in the study period ................. 29

Valence of abortion-relevant tweets by state at each week in the study period ......................... 31

Top 50 national hashtags (larger font indicates greater number of posts using that word as a hashtag) .................................................................................................................................... 32

Example Recruitment Brochure: Illinois ..................................................................................... 46


EXECUTIVE SUMMARY | 4

Overview of Study Objectives and Approach

In this study, The Susan Thompson Buffett Foundation funded NORC at the University of Chicago to

comprehensively assess campaigns in three states in 2018. The research examines the conditions under

which the issue of abortion can impact a variety of campaign and electoral outcomes. The goals of this

study were to understand the campaign process, how the issue of abortion is framed within that context,

and how that might affect election dynamics and outcomes. Empirical research to date shows mixed

results on whether the content of campaigns can influence voters, especially in this era of deep political

polarization.

As summarized in a recent meta-analysis of nearly 50 campaign field experiments, there is little to no

evidence of candidate campaigns’ ability to persuade voters (neither undecided voters nor supporters of

the other candidate) to vote for their candidate in general election campaigns. However, it would be short-

sighted to say that campaigns do not matter. The research assessed in the meta-analysis was focused on a

particular impact of campaigns—changing minds in general elections. While this is arguably an important

outcome, it would be a mistake to generalize the research too broadly. Other empirical studies show that

campaigns do matter in a variety of contexts.

First, campaigns are dynamic and candidates are strategic. Candidates and their campaigns are successful

at changing the focus of the debate in ways that differentiate them from the other candidate and align

them with majority segments of the electorate. In other words, campaigns can elevate or depress the

salience of issues that the public is considering as they make decisions about whether to vote and for

whom they will vote. If abortion is seen by a candidate as a salient issue likely to raise engagement from

their base, it can become a strategic issue for the campaign. The success of a candidate in competitive

races often depends on the ability of campaigns to convert supporters to voters by identifying, energizing,

and getting their supporters to the polls. Assessments of campaign experiments focused on voter turnout

do show that campaigns can be successful.

Second, candidates can persuade their own supporters to care about, and sometimes change, their attitudes

on issues. And, as even the meta-analysis shows, campaigns can be successful in persuading undecided

and opposition voters in both primary and ballot initiative campaigns. Other research has also found that

campaigns can be effective in changing voters’ minds in some unique circumstances.

Given this research, we focused this study on three races to better understand the role of abortion in

campaign dynamics in different political contexts. This enabled us to evaluate the following research

questions:

1. Under what conditions does a candidate’s pro-choice or pro-life position, and communication around

that position, help, hurt, or have no effect on their electoral chances?

2. How do ballot initiatives on the issue of abortion interact with candidates’ campaigns?

3. What motivates or demotivates pro-choice and pro-life voters? How does the issue of abortion

specifically interact with other issues or partisanship for these voters?

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3042867

https://journals.sagepub.com/doi/abs/10.1177/1532440011413085

https://www.tandfonline.com/doi/abs/10.1080/17457289.2012.728223

https://gabriellenz.me/

http://scholar.harvard.edu/files/todd_rogers/files/can_inaccurate_beliefs_about_incumbents_be_changed_ssrn.pdf



4. Does the issue of abortion play a role in persuading voters, mobilizing them, both, or neither? If it

impacts persuasion or mobilization, which voters are most affected?

To answer these questions, we pursued a multi-method research design in three states. A comprehensive

description of the study methods appears in the final section of this executive summary. The states

included are: 1) West Virginia, where there was a highly competitive U.S. Senate election with two pro-

life candidates and a ballot initiative to restrict access to and funding for abortion, 2) Illinois, where there

was a governor’s race with two pro-choice candidates, and 3) Nevada, where there was a highly

competitive U.S. Senate election with a pro-life Republican and a pro-choice Democrat.

Race Candidate Vote share Party Abortion Stance

Illinois: Governor

Winner J.B Pritzker 54.5 Democrat Pro-choice

Bruce Rauner 38.8 Republican Pro-choice

Nevada: Senate

Winner Jacky Rosen 50.4 Democrat Pro-choice

Dean Heller 45.4 Republican Pro-life

West Virginia: Senate

Winner Joe Manchin 49.6 Democrat Pro-life

Patrick Morrisey 46.3 Republican Pro-life

West Virginia Amendment 1

Approved Yes 51.7 - Pro-life stance

No 48.3 - Pro-choice stance

Campaign Monitoring and Survey Research

In each state, we conducted three waves of surveys during the 2018 midterm election campaigns. The first

was conducted at the beginning of the midterm election campaign cycle in late August to early

September. The second was conducted in the final week before the November 6 election. And the third

was conducted shortly after the general election. These three surveys in each state allowed us to evaluate

how opinions about and support for the candidates and the ballot initiative changed throughout the

election cycle.

We also gathered extensive information about the campaign context to enable us to contextualize the

public opinion data and to understand any changes or patterns in the survey data. The data for this part of

the project included the content of websites, emails, and social media posts from the campaigns and

relevant organizations, general Twitter conversation about the campaigns, political advertisements, debate

transcripts, and newspaper coverage of the election within each state.

With this comprehensive evaluation of the election in three states, we are able to evaluate the research

questions outlined above as well as recommend avenues for further research.



Twitter Analysis

In addition to the survey and campaign monitoring research, NORC’s Social Data Collaboratory

conducted a complementary analysis of abortion-related content on Twitter, spanning a five-month period

from July 1–December 10, 2018. The broad goal of the work was to explore and understand the nature of

public and political discourse around abortion prior to the 2018 midterm elections. We examined the

general conversation about abortion, unrestricted by geography, and also aimed to assess whether and to

what extent political campaigns and candidates’ positions on abortion in the states of West Virginia,

Illinois, and Nevada had an energizing, neutral, or de-energizing impact on voting behavior.

We aimed to map our approach to the key research questions, data collection and analyses conducted by

NORC’s survey research team, providing a detailed description of the campaign context with a focus

(narratively and visually) on the timeline of key events with the potential to impact attitudes and other

outcomes being measured. To accomplish this, we analyzed the number, sources, content, and diffusion

of abortion-related messaging overall and in the three sentinel states. We conducted a comprehensive

social media inventory of all messages posted by accounts from major candidates for governor or senate

and containing Twitter handles of those candidates, ballot organizations, advocacy groups, and PACs

supporting either side to insure that all major voices were included in our database and analysis. We used

a combination of supervised and unsupervised machine learning to extract themes and user types from the

retrieved data.

Social media is a key component of modern political discourse analysis. It has emerged as an important

place for stakeholders, advocates, and regular people to express their opinions and react to events both

within their community and in the context of a larger conversation. Political candidates use Twitter to

make public announcements, interact with the public and the media, and promote awareness of their

policy positions. One of the key functions of Twitter is that it also supports two-way communication:

advocacy organizations and regular people can engage with each other about political topics, as well as

with candidates and campaigns. This engagement between candidates, stakeholders, and the general

public represents a dynamic that was rare before the age of social media.

From its inception, Twitter has been characterized as a public sphere,1 where debate and discourse can

flow freely. At its best, Twitter can empower voters by enhancing deliberative democracy as users refine

their own views, gain exposure to different opinions, and identify common means and ends. However,

research has also shown that online discussions can amplify division among social groups holding

different views, rather than forming consensus among them.2 Further, Twitter’s algorithms and features,

such as its hashtags and retweet functions, influence the discourse itself. Indeed, these algorithms

determine the flow of information, enabling prominent topics and content posted by prominent users to

achieve higher exposure and engagement.3 Hashtags serve as a strategic communication device, enabling

1 Fuchs C. 2014. Twitter and democracy: a new public sphere? In Social Media: A Critical Introduction, pp. 176-209. London:

Sage.

2 Yardi, S., and Boyd, D. 2010. Dynamic debates: an analysis of group polarization over time on Twitter. Bulletin of Science,

Technology & Society, 30(5), 316–327.

3

http://www.slate.com/articles/technology/cover_story/2017/03/twitter_s_timeline_algorithm_and_its_effect_on_us_explained.ht

ml

http://www.slate.com/articles/technology/cover_story/2017/03/twitter_s_timeline_algorithm_and_its_effect_on_us_explained.html

http://www.slate.com/articles/technology/cover_story/2017/03/twitter_s_timeline_algorithm_and_its_effect_on_us_explained.html



individuals to find content that interests them, join a larger conversation, and amplify their own voice.

Indeed, research has shown that posts that feature hashtags achieve greater influence compared to

untagged posts.4

Social media research does not represent public opinion. In fact, in recent years, numerous studies have

documented differences between Twitter users and the general public. Rather, the advantage of using

Twitter as a source of information is that it functions as a platform for agenda-setting by groups often

classified as “elites,” including journalists and politicians. At the same time, it also supports engaged

activism: expression by the most opinionated members of the public,5 who are also more likely to vote,

compared with the general public.6 Twitter does not fully represent the average citizen’s opinions.

However, what it does represent extends beyond messages from the traditional elite to include how citizen

influencers and the engaged public react to those messages and how their opinions and disagreements are

framed in the public sphere. In this way it affords users (and researchers) exposure to voices and

stakeholders who are not part of the traditional elite.7

Whereas survey research can provide detailed insight into the attitudes and beliefs held by individuals in a

representative sample of the study population, social media analysis can provide important contextual

information about the amount and content of messages posted by agenda-setters and other stakeholders:

how and to what extent are these actors influencing and reflecting the opinions of the general public?

In addition, analysis of Twitter conversations can place a regional discourse in the context of a broader

national or international conversation—identifying key players outside of a particular geographic

community. Finally, analysis of Twitter discourse can uncover unexpected actors, ideas, and language

used by both sides of a political issue.8 Thus, Twitter analysis can help inform the interpretation of

survey-based public opinion research, offer insights about campaign strategy, reveal key stakeholders,

and suggest opportunities and potential risks for developing public engagement and issue-oriented

strategies.

Key Findings

Although the social media conversation is not representative of public opinion, the abortion discourse on

Twitter provided some confirmation of and context for the survey findings. The surveys found that

overall, abortion was not a strongly motivating issue in the election cycle and not among the most

important issues in most people’s voting behavior. The Kavanaugh Supreme Court justice nomination and

4 Davis, Bud. 2013. “Hashtag politics: the polyphonic revolution of #Twitter.” Pepperdine Journal of Communication Research:

Vol. 1, Article 4. Available at: http://digitalcommons.pepperdine.edu/pjcr/vol1/iss1/4.

5 https://www.nytimes.com/interactive/2019/04/08/upshot/democratic-electorate-twitter-real-life.html

6 https://www.pewinternet.org/2019/04/24/sizing-up-twitter-users/

7 Freelon D, Karpf D. 2015. Of big birds and bayonets: hybrid Twitter interactivity in the 2012 presidential debates. Information,

Communication & Society, 18: 390–406.

Papacharissi Z. 2013. A networked self: identity performance and sociability on social network sites. In Frontiers in New Media

Research, eds. F Lee et al., pp. 207–21. New York, NY: Routledge.

8 Sedereviciute K, Valentini C. 2011. Towards a more holistic stakeholder analysis approach. Mapping known and undiscovered

stakeholders from social media. International Journal of Strategic Communication, 5: 221–39.

http://digitalcommons.pepperdine.edu/pjcr/vol1/iss1/4

https://www.nytimes.com/interactive/2019/04/08/upshot/democratic-electorate-twitter-real-life.html

https://www.pewinternet.org/2019/04/24/sizing-up-twitter-users/



confirmation hearings drove much of the Twitter discourse across all three states, and election-related

Tweets largely avoided the issue of abortion. Analyzing the ratio of pro-choice to pro-life tweets across

the three states showed that the Twitter discourse largely mirrored abortion sentiment in those states.

There was little evidence that social media influenced individual voting behavior, but strong evidence that

it reflected elite viewpoints about abortion and revealed ways in which the elite groups engage with the

public. Generally, the abortion discourse on Twitter involved expected voices: advocacy groups,

celebrities, political parties and candidates, and regular people who hold a strong opinion about abortion

politics.

The unique contribution of the Twitter analysis was to provide context, identify key stakeholders, and

uncover rhetoric used in the political discourse about abortion overall and in each of the three states. Our

analyses confirmed that the ballot initiative added a unique dimension to the state-level conversations on

Twitter. In West Virginia and Nevada, where the political races were more highly contested, celebrity

accounts weighed in on the issue of abortion and received significant exposure and engagement. The

biggest missing piece of information in our Twitter analysis was whether and how abortion co-occurred

with other issues that motivate voters, such as gun control and immigration. The surveys, however,

provide several insights into the relative position of abortion among other key election issues. We also did

not look for misinformation overall or by user type.

Abortion did play a significant role in the Twitter conversation in West Virginia; the per capita rate of

tweeting about abortion was significantly higher in West Virginia than in the other states or overall. The

ballot initiative accounted for a significant portion of the abortion discourse on Twitter in West Virginia,

and many of the messages reflected confusion over what the Amendment was about. Similarly, survey

respondents indicated confusion about the ballot initiative. Tweets posted by candidates, political and

advocacy organizations in West Virginia leaned strongly pro-life and Tweets about Amendment 1 were

largely supportive. While the surveys found that pro-life voters were motivated when messages were

framed as moral, religious, or personal choice, the rhetoric of tax-payer responsibility was commonly

used in pro-life advocacy for Amendment 1 on Twitter.

The abortion-related Twitter discourse in Illinois was relatively contained and largely reflected the

national discourse, following the trends in volume around Kavanaugh’s nomination and confirmation. In

a state where most voters indicated a pro-choice stance and where both major candidates were pro-choice,

abortion was simply not a priority topic in the context of the political campaigns. Those posts which were

campaign-related largely focused on a vocal pro-life minority trying to elevate the issue, with very limited

effect. These findings are consistent with survey results showing that abortion was not one of the top

issues motivating Illinoisans to vote.

The abortion discourse on Twitter in Nevada reflected that the Democratic Senate candidate was a vocal

pro-choice candidate on Twitter. While her campaign messaging on broadcast ads and in debates largely

avoided the topic of abortion, her Twitter messages highlighted her pro-choice stance and achieved high

exposure and engagement. This finding supports the survey results that showed the Democratic Senate

candidate was able to garner more support from opponents even though she was pro-choice, and voters

more accurately identified her position.



Survey and Campaign Monitoring: Summary of Findings

Coupled with the Twitter analysis, the survey and campaign monitoring research provide a wealth of data

to answer the key research questions of interest. While several facets of the data provide insights across

several questions, the findings below are what we consider the high-level results. Each case study section

walks through the findings in further detail.

One task of this research was to determine the role abortion plays in campaigns with differing contexts.

Both the Twitter analysis and campaign monitoring analysis show that abortion was most salient in West

Virginia, and least salient in Illinois. In the Nevada Senate race, the Democratic Senate candidate brought

up the issue more than the Republican incumbent and embraced the issue directly.

Percentage of candidates’ campaign content that was related to abortion

Illinois Nevada West Virginia

Democrat % Republican% Democrat% Republican %

Democrat % Republican %

Ads 4 - - - - 21

Twitter <1 - 2 - <1 5

Web content 10 - 6 5 - 23

Debates - - - - 5 6

Emails 2 - 7 1 - <1

Funding <1 - 1 <1 <1 -

Articles 3 2 4 8 13 10

Broadcast 7 4 1 1 2 2

The West Virginia ballot measure drove a lot of the Twitter traffic coming from pro-choice messengers

early in the campaign, and late in the campaign, pro-life conversations dominated. At the same time, the

U.S. Senate race featured two pro-life candidates, however, the Republican candidate was much more

likely to discuss the topic of abortion than the Democrat. For his part, the Democratic incumbent

reiterated his pro-life position, but at the same time opposed the ballot measure late in the campaign,

lining up with pro-choice rather than pro-life interests. This campaign dynamic shaped voters’ level of

knowledge about the issue as it relates to the campaign. Voters were actually not very familiar with

candidates’ issue positions generally, and abortion was one of the least familiar issues.

A majority of voters in West Virginia considered themselves pro-life, while majorities in Illinois and

Nevada considered themselves pro-choice. Across parties, identification as pro-choice or pro-life varied

by state. This context is important for understanding the detailed survey findings that follow.



Considerable inconsistency among voters in party identification and abortion attitudes

West Virginia Illinois Nevada

% Pro-choice

% Pro- life

% Pro-choice

% Pro- life

% Pro- choice

% Pro- life

Overall 44 55 64 35 66 34

Democrats 67 32 83 17 84 16

Independents 47 51 61 38 70 30

Republicans 25 75 36 63 44 56

The Illinois gubernatorial election featured two pro-choice candidates, yet neither focused on the issue of

abortion much at all. In both West Virginia and Illinois, voters were confused about candidates’ positions

on abortion when they didn’t line up with traditional party opinions. Most voters thought that Democrat

Senate incumbent was pro-choice despite his pro-life stance, and most viewed the incumbent Republican

Governor of Illinois as pro-life rather than pro-choice.

In Nevada, a competitive race for Senate resulted in a Democratic win, all while the Democrat embraced

the issue of abortion directly. The Twitter analysis shows that the Democrat was one of the top

communicators about the issue during the campaign, retweeting interest groups’ content and calling

herself pro-choice.

It is under this context that we observe several key findings related to the research questions below. It is

important to note that across the research questions highlighted, the findings hold up when examining

respondents determined to have actually voted in the election, based on a voter match.9

Research Question 1

Under what conditions does a pro-choice or pro-life position, and communication around that position,

from a candidate help, hurt, or have no effect on their electoral chances?

Similarities across states

■ Context matters for knowledge on issue positions when candidates hold the same views and when one

holds a position contrary to the majority of their party. In Illinois and West Virginia, majorities could

not accurately describe the candidate with views discordant with their party’s conventional view on

abortion.

■ Crossover voting was more likely for Republican pro-choice voters than Democratic pro-life voters.

Democratic candidates received more support from pro-life Democrats and independents than

Republican candidates received from pro-choice Republicans. And Democratic candidates also did

better among independent voters. Democratic candidates were more likely to get the support of pro-

choice independents than Republicans were to get the support of pro-life independents.

9 See methodology section for voter match procedure.



Republican pro-choice voters were more likely to cross party lines than Democratic pro-life voters

% of __ voting for__

Illinois Nevada West Virginia

Pritzker Rauner Rosen Heller Manchin Morrisey

Democrats, Pro-choice 88 8 94 3 91 5

Democrats, Pro-life 77 17 88 7 82 16

Indeps, Pro-choice 48 35 60 28 68 20

Indeps, Pro-life 40 41 32 46 39 49

Republicans, Pro-choice 20 74 16 78 40 54

Republicans, Pro-life 7 81 5 92 19 76

Question: With respect to the abortion issue, would you consider yourself to be pro-choice or pro-life? Source: In Focus Wave 3 poll conducted by NORC at the University of Chicago, November 7-November 21, 2018, with 2,525 registered voters in Illinois, 2,049 registered voters in Nevada, and 2,831 registered voters in West Virginia

Differences across states

■ Despite relatively low familiarity with candidates’ positions on the issue of abortion observed across

the campaigns, Rosen was able to garner more support from opponents. Rosen did so as a pro-choice

candidate, with most voters accurately identifing her position.

■ Voters were more confused in West Virginia and Illinois, where candidates aligned on the issue of

abortion despite being from opposing parties.

■ In West Virginia and Nevada, the Democratic candidates were considered better able to handle the

issue of abortion, while in Illinois the candidates were rated similarly.

West Virginia

■ Manchin discussed his position on abortion far less than Morrisey. Manchin also held a position

inconsistent with the majority of his party. Voters were confused about Manchin’s position, with most

viewing him as pro-choice rather than pro-life even before he came out against the ballot measure.

Within this context, Manchin was still considered better able to handle the issue of abortion than

Morrisey.

■ Likely driven by his incumbency advantage, Manchin received a larger share of pro-life voters than

Morrisey received of the pro-choice vote, as well as a larger share of Republicans than Morrisey

received of the Democratic vote, which pushed Manchin to victory in a tight election. He also earned

the support or more independents.

Illinois

■ Very little information was communicated by either campaign in the race on the issue of abortion.

Pritzker got support, however, from more pro-life voters than Rauner did from pro-choice voters. This

occurred even though there was substantial confusion around Rauner’s position—most thought he

was pro-life.



■ At the same time, abortion ranked lower as a salient issue to vote choice when compared to other

issues, so there is a lack of evidence that this issue would have been pivotal in this race—especially

given the wide spread in the election result.

Nevada

■ While neither candidate in the Nevada race focused on the issue significantly, Rosen did so more than

Heller. At the same time, she was able to consolidate more pro-choice independents and Republicans

than Heller was able to obtain from pro-life Democrats or independents. This finding holds up among

voters determined to have actually voted in the election.10

■ Voter contact mattered a lot for Rosen. Those who had any form of political or other organizational

contact during the campaign were more likely to vote for her, as well as understand her position on

abortion.11

■ Despite being pro-choice, over a quarter of Republicans said Rosen would do a better job on abortion

than Heller. And more Democrats thought Rosen could better handle the issue than did Republicans

for Heller.

■ Despite being contextually different, the overall implications of the role of abortion in Illinois and

Nevada were quite similar.

Research Question 2

What motivates or demotivates pro-choice and pro-life voters? How does the issue of abortion

specifically interact with other issues or partisanship for these voters?


■ Across states, abortion was predictive of vote choice even when controlling for partisanship and other

demographics. After controlling for these other factors, individuals’ who were pro-choice were more

likely to vote for Democrats. The Democratic candidate in each of these elections won the race. 12

However, other issues including immigration and gun control/gun rights were also associated with

vote choice and, in some cases, had a larger effect in the model than abortion. In all cases,

partisanship is more predictive of vote choice than issue attitudes.

■ In each state, abortion ranked relatively low on issue importance, and there are few single issue

voters.

10 As part of the research study, NORC conducted a voter match to validate which survey respondents actually voted in the

midterm election based on official records. NORC used a vendor to conduct this analysis. See the Methodology section for a full

description of the procedure.

11 Among actual voters, the latter finding is not statistically significant.

12 See multivariate models from Appendix.



Abortion ranked lower on the issue priority list and was more important for Democrats in Nevada and Illinois

Question: How important were each of the following issues to you in deciding how to vote in this year’s

election for [governor/senator] between…? …Abortion

Source: In Focus Wave 3 poll conducted by NORC at the University of Chicago, November 7-November 21, 2018, with 2,525 registered voters in Illinois, 2,049 registered voters in Nevada, and 2,831 registered voters in West Virginia

■ Motivations for being pro-choice or pro-life were nearly identical across states, and opinions about

the roles of gender and race in society also reflect clear patterns.

► Pro-life voters were motivated when the issue was framed as moral, religious, and a personal

health issue.

► Pro-choice voters were motivated when an issue was framed as one of freedom, an individual’s

personal health, and privacy.

► Voters across states also ranked candidate traits important to their vote choice similarly, with

alignment on issues between the voter and a candidate among the most important, as well as the

candidates’ values.

65 (#7)

41 (#7)

41 (#8)

60 (#6)

43 (#9)

48 (#8)

59 (#7)

48 (#9)

61 (#8 of 9)

0 10 20 30 40 50 60 70 80 90 100

West Virginia

Illinois

Nevada

Percent extremely/very (rank among nine issues asked about)

Democrats Independents Republicans



Almost no difference between pro-choice and pro-life voters in candidate personality traits they value

Question: Thinking about your vote for [governor/Senate], how important was each of the following when

deciding who to vote for? That the candidate cares about people that need help, That the candidate was

honest, That the candidate is moral, That the candidate has integrity


76

84

82

83

80

84

85

87

84

84

88

89

81

78

82

85

83

76

90

89

86

77

89

88

0 10 20 30 40 50 60 70 80 90 100

Moral

Cares about people

Integrity

Honest

Moral

Cares about people

Integrity

Honest

Moral

Cares about people

Integrity

Honest

Percent extremely/very important

Pro-life Pro-choice

Nevada

Illinois

West Virginia




■ Importance of abortion as an issue varies by state and context among pro-choice and pro-life voters. It

was most important for pro-life voters in West Virginia.

Abortion was a higher priority for pro-life than pro-choice voters in West Virginia and Illinois

Question: How important were each of the following issues to you in deciding how to vote in this year’s

election for [governor/senator] between…? …Abortion


West Virginia

■ In an election where both candidates claimed the same position, abortion position was a significant

predictor of vote choice, but so were other factors, such as partisanship, position on gun rights or gun

control, immigration, and the economy. These other factors are more predictive of vote choice than is

position on abortion.13

■ Pro-life voters are motivated by the issue when was framed as a moral, religious, and personal health

issue. Pro-life voters also preferred candidates who are honest, have integrity, and seen as moral.

■ Pro-choice voters were motivated by the issue when it was framed as an issue of freedom, personal

health, and privacy. Pro-choice voters were motivated by similar candidate traits as pro-life voters;

they were slightly more likely to prefer a candidate who cares about people who need help.


51

40

52

50

52

70

0 10 20 30 40 50 60 70 80 90 100

Nevada

Illinois

West Virginia

Percent extremely/very important

Pro-life Pro-Choice



Illinois

■ Being pro-choice or pro-life was a significant indicator of one’s vote choice, but similar to the other

states, it’s not the only predictor. Partisanship and issue position on guns, immigration, and the

economy mattered more.14

■ Voters were also more polarized in their responses about whether guns and immigration were critical

to their vote choice—while sizable proportions did consider abortion, there are not significant

disparities between pro-choice and pro-life voters in considering it.

■ Pro-choice and pro-life voters alike preferred candidates who share their positions on the issues

generally and on those most important to them, as well as their values.

■ Pro-life voters were motivated when the issue is framed as moral, religious, and a personal health

issue.

■ Pro-choice voters were motivated when an issue is framed as one of freedom, an individual’s personal

health, and privacy.

Nevada

■ Healthcare was a major mobilizing factor for pro-choice voters and Democrats in Nevada. For

Republicans and pro-choice voters, the importance was placed on the economy.

■ Being pro-choice or pro-life is a significant indicator of one’s vote choice, but similar to the other

states, it’s not the only predictor. Partisanship and issue position on guns, immigration, and the

economy mattered more.15

■ Similar to the other states, voters were also more polarized in their responses about whether guns and

immigration were critical to their vote choice. Pro-choice and pro-life voters placed similar levels of

importance on the issue of abortion when voting.

■ Pro-choice and pro-life voters alike preferred candidates who share their positions on the issues

generally and on those most important to them, as well as their values.

■ Pro-life voters are motivated when the issue is framed as moral, religious, and a personal health issue.

■ Pro-choice voters are motivated when an issue is framed as one of freedom, an individual’s personal

health, and privacy.

Research Question 3

Does the issue of abortion play a role in persuading voters, mobilizing them, both, or neither? If it

impacts persuasion or mobilization, which voters are most affected?


■ Voter turnout and campaign participation/engagement is similar between pro-life and pro-choice

voters despite varying election contexts and issue positions of candidates.





■ In every candidate race, pro-choice Republicans were more likely to support the Democratic

candidate than pro-life Democrats were to support the Republican candidate.


■ Respondents across states heard relatively less about the issue of abortion than other issues. Yet

voters who heard more information about the issue were more likely to vote for the Democratic

candidates in Illinois and Nevada, and the Republican candidate in West Virginia.

West Virginia

■ Abortion did not impact propensity to vote in any state, including West Virginia. Regardless of issue

position, voters were equally likely to participate in the election even when pro-choice voters didn’t

have a candidate aligned with their position on the issue.

■ Voters across issue positions were also equally likely to be interested in the election and follow the

news about it.

■ Sizable proportions of voters aligned themselves with abortion opinions that are discordant with their

party, and these voters are more moveable on the issue. A quarter of Republicans identified as pro-

choice and a third of Democrats identified as pro-life.

■ There is no direct evidence that the issue of abortion mobilized voters to engage more or less in the

campaign.

Illinois

■ Illinois voters across issue positions were equally motivated to participate in the election.

■ The issue of abortion itself did not impact reported voter turnout.

■ Voters were also equally likely to say they were interested in the election and paid attention to news

about it, across issue positions.

■ Pro-choice Republicans tended to be upper-income, older, white, or married. Pro-life Democrats

tended to be black, Hispanic, or lower-income.

Nevada

■ Nevada voters across issue position were equally motivated to participate in the election, and did so at

a similar rate, with over 90 percent each engaging in the campaign either actively or passively.

■ As in the other states, abortion is not a “single issue to vote on” for many people. Voters do label it as

very important, but not as important as some of the other issues tested in the survey.

■ The issue of abortion itself did not impact reported voter turnout.

■ Voters were also equally likely to say they were interested in the election and paid attention to news

about it, across abortion issue positions.

■ Pro-choice Republicans tended to be upper-income, older, white, have no religion or identify as

Protestant, or be married. Pro-life Democrats tended to be black, Hispanic, or lower-income.



Research Question 4

How do ballot initiatives on the issue of abortion interact with candidates’ campaigns?

■ There was confusion over wording and what the ballot measure would do. Confusion was highest

among those who voted for the measure.

■ Candidate endorsements did not play a significant role in support for the ballot measure, though they

mattered for some key demographic groups, like lower-income voters and women.

■ Despite the ballot measure being solely about abortion, the candidates did not spend relatively much

time discussing the issue during the campaign, and the issue was not a top priority for voters.

■ Voter contact mattered significantly on the ballot measure. Those who were contacted by a political,

organization or PAC, or a civil society group about the ballot measure were more likely to vote for

the measure, understsand it, and accurately say what it would do. However, when looking just at pro-

choice respondents, contact did not impact propensity to vote for Amendment 1.

■ Campaign engagement—particularly passive engagement—was similar in the ballot campaign when

compared to the candidate campaigns in Illinois and Nevada. However, in the latter two states, people

were more likely to talk about the campaign with friends and family.

No difference between pro-choice and pro-life voters in campaign engagement16

Question: We find that many people participate in politics in other ways besides voting. For the November 6

election in [state], how many times did you do each of the following to support a candidate in the

campaign…? If you [have not done/did not do] any of the items you may tell us that too.

How many times did you talk with family or friends about the campaign…? If you did not do this you may

tell us that too.

16 Note: West Virginia data refer to engagement on the Amendment 1 campaign.

45

66

56

79

51

73

41

68

56

78

52

76

0 10 20 30 40 50 60 70 80 90 100

Wore buttons orwatched debates

Talked to family orfriends





Percent once/more than once

Pro-life Pro-Choice

Nevada

Illinois

West Virginia




Twitter: Summary of Findings

Overall, the social media analyses showed that state-level discourse around abortion generally reflected

the national discourse. The Kavanaugh nomination and confirmation accounted for major peaks in the

overall conversation, nationally and in each state. Controlling for population size, the amount of Twitter

discourse in each state reflected the contentiousness of the races. With both a senate race and the ballot

initiative to restrict access to abortion, West Virginia generated the most abortion-related tweets per

capita. While individuals (regular users) accounted for 95 percent of all tweets, elite accounts generally

tweeted most actively and achieved the highest reach and engagement in each state. Indeed, retweets of

elite’s messages account for much of the volume of tweets coming from regular people. This points to the

important role of elites/influencers in shaping the political discourse. Nationally and across each state,

sentiment on Twitter was mostly pro-choice as measured by tweet volume; if measured by reach,

sentiment was more evenly divided. The volume of pro-life tweets increased steadily between September

and November, and then dropped off, suggesting organized mobilization within the pro-life movement.

■ During the study timeframe from July 1, 2018, through December 10, 2018, overall 16,064,399

tweets were collected with an average of 669,350 per week.

■ During the study timeframe from July 1st, 2018 through December 10th, 2018, overall 16,064,399

tweets were collected with an average of 669,350 per week.

■ Illinois, Nevada, and West Virginia followed national trends with the exception of the West Virginia

election. Other individuals/regular users accounted for 95 percent of tweets about abortion during the

study period.

■ Celebrity accounts were the most active sub-group of users among the elite accounts that tweeted

about abortion during study period.

■ Overall, pro-choice sentiment was more common than pro-life sentiment.

■ The top 10 most active accounts overall were mainly pro-life or right leaning.

■ While pro-choice tweets outnumbered pro-life tweets during most of the study period, by late

October, pro-life tweets became more common than pro-choice. Following the election, pro-choice

was again the dominant sentiment.

■ The top five national hashtags included “prolife” (239,743 tweets), “abortion” (95,548 tweets),

“Kavanaugh” (91,753 tweets), “MAGA” (66,238 tweets), and “StopKavanaugh” (66,139 tweets).

Twitter: West Virginia Findings

Abortion was a heated topic in West Virginia leading up to the midterm election. Political campaigns

actively tweeted about the issue, including messages about the national discourse, political candidates,

and the ballot initiative. Pro-life messages dominated the discourse in West Virginia and achieved the

greatest reach, due to the role of elite accounts. Bots played a larger role in the West Virginia discourse,

compared to other states, and were used more to communicate a pro-life message. The pro-life movement



used the rhetoric of protecting taxpayers in its messaging supporting Amendment 1; the pro-choice

movement used language about protecting women and providing health care.

■ Of the three states, West Virginia had the most campaign-related Twitter activity.

■ Engagement was high for the West Virginia campaign-related tweets, likely because they included

tweets from celebrities like Donald Trump Jr., John Cusak, John Leguizamo, and Holly Figueroa

O'Reilly.

■ Pro-choice groups tweeted much more than pro-life groups, but pro-life groups had a significantly

higher proportion of tweets relevant to abortion.

■ Pro-life organizations and individuals were potentially more influential because those accounts had

more followers.

■ Celebrities participated in the conversation in West Virginia, increasing the visibility of the abortion-

related messages on Twitter.

■ Pro-life sentiment was dominant across most of the user-type categories in West Virginia, in a way

that it was not in either of the other states.

■ Pro-choice sentiment was much higher among non-affiliated groups and organic users on Twitter, but

the pro-life sentiment was higher among other elite user categories, leading to a high exposure of pro-

life messages over pro-choice messages.

■ Bots were more likely to be used by and for pro-life groups, indicated by the sentiment and hashtags.

■ West Virginia Amendment 1 was a major topic of the abortion-related discussion in the state.

■ Posts about the Amendment often reflected incorrect information about the circumstances under

which the new law would cover abortion services.

■ Support for the amendment was expressed in terms of protecting taxpayers, while opposition for the

amendment used language about providing access to healthcare and protecting women.

Twitter: Illinois Findings

In contrast to West Virginia, abortion was not discussed much on Twitter in Illinois. Neither campaign

mentioned abortion in its campaign messaging, nor were the major candidates mentioned in the discourse

about abortion. Advocacy groups posted messages mainly related to the national discourse. Illinois

Democratic organizations were more active than Republican groups, but pro-life and pro-choice groups

posted at similar rates. Pro-life groups used more inflammatory language in their posts and achieved

higher exposure because the accounts had more followers. Because both candidates were pro-choice and

the race was not close, the pro-life activity on Twitter neither reflected nor influenced the overall

discourse or voting behavior.

■ There was little conversation about abortion from either Democratic or Republican campaigns or the

Illinois Democratic and Republican Parties. Also, Twitter mentions of each candidate contained little

mention of abortion keywords.

■ Both Planned Parenthood Illinois Action and the Illinois Family Institute tweeted about abortion

equally and their accounts had the most tweets out of any state-based advocacy organization.



■ Some smaller political players from Illinois tried to emphasize abortion, but the issue was not adopted

by the major candidates.

■ The official Democratic Party account tweeted more overall than the Republican Party account, but

neither tweeted significantly about abortion.

■ Pro-choice and pro-life groups tweeted at similar rates overall, and had similar proportions of tweets

relevant to abortion.

■ Overall, pro-life and pro-choice groups sent out a similar number of relevant tweets, but pro-life

groups used explicit language such as “murder” and “abort.”

■ Top influencer accounts were pro-life.

■ Bots generated a relatively small portion of tweets about abortion from Illinois.

■ Tweets by news organizations were relatively neutral.

■ Illinois advocates, other groups, and bots had a pro-life sentiment.

■ All other Illinois sources, including political accounts and regular people, were pro-choice.

Twitter: Nevada Findings

As in Illinois, the abortion-related conversation in Nevada tracked with the national discourse, with a

focus on the Kavanaugh confirmation. In contrast to Illinois, the Democratic candidate for Senate took on

abortion as a campaign issue and was vocal on Twitter about her pro-choice stance. Her tweets and tweets

about her pro-choice position achieved high exposure and engagement. Republican voices were largely

absent in the abortion-related discourse in Nevada. Regular people posted the majority of tweets about

abortion in Nevada, and strongly leaned pro-choice. In Nevada, abortion rights was a rallying call for

Democratic politicians, advocacy organizations, and pro-choice individuals.

■ While the Nevada Democratic Party, the Democratic candidate and her campaign account posted

messages on abortion, the Republican Party and the Republican candidate avoided the issue entirely.

■ Tweets from the Democratic candidate campaign account were retweeted and mentioned the most

during the data collection time period.

■ Celebrity account users were the most vocal group.

Twitter: Overall Abortion Narrative

During the study timeframe from July 1, 2018, through December 10, 2018, the total amount of abortion-

related tweets was 16,064,399, with an average of 669,350 tweets per week.



Weekly volume of abortion-related tweets in the United States

Overview of Twitter Data – Abortion Topic – Illinois, Nevada, West Virginia

Specifically for each state, the total amount of abortion-related tweets was 211,336 from Illinois, 59,508

from Nevada, and 24,433 from West Virginia, with weekly averages of 9,076, 2,556, and 1,049

respectively.

Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia

0

200000

400000

600000

800000

1000000

1200000

1400000

1600000

1800000

7/2/2018 8/2/2018 9/2/2018 10/2/2018 11/2/2018 12/2/2018

National Weekly Volume of Tweets

0

5000

10000

15000

20000

25000

30000

7/2/2018 8/2/2018 9/2/2018 10/2/2018 11/2/2018 12/2/2018

Weekly Tweets

IL NV WV



When adjusted for population, West Virginia had a disproportionate amount of tweets about abortion. The

number of abortion-related tweets per 100,000 people was 1,657 from Illinois, 2,226 from Nevada, and

2,525 from West Virginia.



Weekly volume of abortion-related tweets in Illinois, Nevada, and West Virginia (per 100,000 population)

National and state-level trends of tweets on abortion (with Pearson correlations between the states and national level)

The trends in Illinois and Nevada are strongly correlated with the national trend, and the trend in West

Virginia is moderately correlated with the national trend.

0

50

100

150

200

250

300

350

7/2/2018 8/2/2018 9/2/2018 10/2/2018 11/2/2018 12/2/2018

IL NV WV

USA

Illinois

Nevada

West

Virginia

July 10: President Trump nominates Kavanaugh for SCOTUS

Sept 6: Kavanaugh hearing

Oct 5: Kavanaugh sworn in

Nov 7: West Virginia Amendment 1 passes

Pearson Correlations

(0.98)

(0.95)

(0.66)



Characterizing Message Source

The American states have often been described as “laboratories of democracy” because of their ability to

experiment with and adopt new policies. Building on this perception, prior research investigated how

policy innovations are formulated, promoted, implemented, and spread across the states and jurisdictions.

Recently, however, journalists have documented that interest groups exert considerable influence over

policy adoption and emulation.17 This indicates a prominent role for interest groups in policy adoption,

particularly at the state- and local- level.

Political and social institutions, as well as leaders of opinion on the abortion rights issue are widely

present on social media platforms. Social media allow us to identify and prioritize these known and

undiscovered stakeholders based on their online level of activity, engagement, and demonstrated concern

with the issue (and to potentially engage the “movers and shakers”/influencers by directly communicating

with them, bypassing the traditional mass media channels).18

Account holders posting about abortion-related issues were categorized as members of one of the

following groups:19

■ news media

■ advocates

■ political organizations

■ politicians

■ celebrity

■ other individuals/regular people

Together, the news media, advocacy groups, political organizations, politicians, and celebrities are

referred to as elite social media accounts throughout this report. We juxtapose this elite group from other

individuals/regular people because of their cultural influence.

Amount of Tweets by Source

The table below shows that within the larger group of elite accounts, celebrity users accounted for the

largest subgroup of elites. Celebrity accounts included any users who indicated that they were actors,

musicians, performers, writers, or entertainers, and who had more than 8,000 followers. Other

individuals/regular users accounted for 95 percent of the tweets about abortion during the study period.

The second table also shows that advocates and celebrities were the most vocal elite groups on the issue

of abortion rights nationally and across the three states, with celebrity users posting the largest number of

tweets nationally (3,450 tweets), followed by advocacy groups (3,388 tweets). It is worth noting that,

besides the other individuals (~95 percent), advocates (1.2 percent) posted the highest number of tweets

17 Greenblatt, Alan. 2011. “Right-minded.” Governing 25 (3): 32–36.

18 Kristina Sedereviciute and Chiara Valentini. 2011. Towards a more holistic stakeholder analysis approach. Mapping known

and undiscovered stakeholders from social media. International Journal of Strategic Communication. 5:4, 221-239.

19 See methodology section for definitions.



in Illinois, celebrities (1.9 percent) tweeted the most about abortion in Nevada, and bots (2.1 percent)

posted the highest number of tweets in West Virginia.

Amount of abortion-related tweets by source type (among the tweets posted from Illinois, West Virginia, and Nevada)

Source Type Total Illinois Nevada West Virginia

News media 2,205 (0.7%) 1,643 (0.8%) 350 (0.5%) 212 (0.5%)

Advocate/advocacy group 3,388 (1.0%) 2,470 (1.2%) 323 (0.5%) 615 (1.3%)

Political organization 585 (0.2%) 116 (0.1%) 164 (0.2%) 305 (0.7%)

Other group/organization 898 (0.3%) 741 (0.3%) 66 (0.1%) 92 (0.2%)

Celebrity 3,450 (1.1%) 1,758 (0.8%) 1,288 (1.9%) 424 (0.9%)

Politician 1,073 (0.3%) 766 (0.4%) 235 (0.4%) 77 (0.2%)

Other individual/person 308,999 (95.4%) 202,663 (95.6%) 63,755 (95.6%) 43,148 (94.2%)

Bot 3,194 (1.0%) 1,791 (0.8%) 495 (0.7%) 948 (2.1%)

Average Twitter Activity by Source

The following table shows that while celebrities accounted for the most tweets among the elite subgroups,

politicians and advocacy groups on average posted more tweets per account, with politicians posting the

greatest average number of tweets nationally (11.7 tweets per account) and advocacy groups being

particularly vocal on abortion-related issues in the state of Illinois (32.9 tweets per account).

Average number of abortion-related tweets per account by source type

Source Type Total Illinois Nevada West Virginia

News media 5.7 6.3 6.1 2.8

Advocate/advocacy group 11.6 32.9 4.1 3.5

Political organization 10.6 8.9 7.8 13.3

Other group/organization 6.4 7.8 3.3 3.2

Celebrity 7.8 8.0 10.1 3.9

Politician 11.7 19.2 7.8 2.9

Other individual/person 4.6 5.6 4.7 2.4

Bot 3.1 3.5 2.3 2.6

It is noteworthy that exploratory analyses of account users helped uncover another, less prominent,

influencer group—religious institutions and leaders—which predominantly fell under the “other

group/organization” category. Religious organizations are well established as opinion leaders and

stakeholders on the issue of abortion rights and supporters of the pro-life position.



Engagement Overall and by State

Engagement with abortion-related tweets is defined as any retweet, quote, or reply to an original post, and

reflects a greater interest in and wider exposure with the content than is achieved by tweets that do not

generate engagement. The table below shows that the vast majority of posts overall (77 percent) were not

retweeted, quoted, or replied to. Abortion content posted in Illinois received the least engagement,

followed by Nevada. In contrast, over half of the abortion-related tweets in West Virginia received some

engagement.

Engagement: amount of tweets among other individual/person that are retweeting, quoting, or replying to a verified or influential user

Total Illinois Nevada West Virginia

Engagement 70,131 (22.7%) 31,747 (15.7%) 17,871 (28.0%) 20,970 (48.6%)

Retweet 50,646 24,028 12,702 14,223

Quote 15,689 6,234 4,309 5,273

Reply 3,796 1,485 860 1,474

Not engagement 238,868 (77.3%) 170,916 (84.3%) 45,884 (72.0%) 22,178 (51.4%)

Twitter Messaging Source: Takeaways

■ Politician and advocacy groups were the most vocal; they posted an average of 11-12 tweets per

account.

■ While there were more tweets from individuals, on average advocates, political organizations, and

politicians were more active on the topic.

■ The level of engagement was the highest in West Virginia (48.6 percent), followed by Nevada (28

percent) among the “other individual/person” category.

■ Hypothesis: the difference in level of engagement with gatekeepers in West Virginia versus Nevada

and Illinois was due to the presence of the West Virginia ballot measure as well as the Democratic

candidate’s vote on Kavanaugh. These issues were missing in Nevada and Illinois. However, there

was still a large amount of organic conversation about the topic but less engagement.

Twitter: Characterizing Message Sentiment

This section summarizes key trends in the pro-choice/pro-life sentiment of abortion-related tweets in

West Virginia, Illinois, and Nevada from July to December 2018.

We developed models to predict whether individual tweets advocated for pro-choice positions, for pro-life

positions, or neither. Prediction models were trained using information about the account authoring the

tweet and the combination of words and phrases in the tweet text (see Methodology section for more

detail). Because we did not perform source coding of the non-geolocated tweets, our sentiment analyses

were conducted on the subgroup of tweets that were geolocated for the three states. Thus, the trends

reported below reflect overall sentiment and sentiment by state for only the subset of approximately

300,000 geolocated tweets, rather than the full archive of over 16 million.



Number of tweets categorized as having pro-choice sentiment, pro-life sentiment, or neither for each state

Sentiment Category Illinois Nevada West Virginia Three-State

Total

Pro-choice 75,858 (36%) 19,536 (33%) 7,866 (32%) 103,260 (35%)

Pro-life 38,108 (18%) 13,764 (23%) 6,932 (28%) 58,804 (20%)

Neither (neutral or irrelevant) 97,370 (46%) 26,208 (44%) 9,635 (39%) 133,213 (45%)

Pro-choice to pro-life ratio 2.0 1.4 1.1 1.8

Twitter Message Sentiment: Takeaways

■ Pro-choice sentiments were more common than pro-life sentiments across all three states; however,

the magnitude of the difference differed substantially across states.

■ The ratio of supporting to opposing sentiment was the highest in Illinois (2:1), followed by Nevada

(1.4:1), and lowest in West Virginia (1:1).

■ West Virginia had the lowest proportion of neutral tweets, suggesting that the issue was most divisive

in that state.

Twitter: Sentiment over time

There was an apparent rise in pro-life content in the weeks leading up to the election in West Virginia,

Illinois, and Nevada. Immediately following the election, the share of pro-life content decreased. This

trend is most pronounced in West Virginia, which is also the only state with an abortion-specific ballot

measure—and the state in which abortion featured most prominently as a campaign issue (see the Survey

campaign monitoring section for additional background about the West Virginia races). These facts

provide preliminary evidence for the hypothesis that there was a coordinated effort—most concentrated in

West Virginia—to use Twitter to influence citizens to vote in favor of pro-life policies during the 2018

midterm elections. Some of the most influential pro-life accounts certainly played a role, but pro-life

advocacy in West Virginia also took place at the level of individual voters voicing their opinions on

Twitter. Further targeted analyses could characterize the nature of pro-life advocacy on social media more

precisely.



Valence of state-specific abortion-relevant tweets at each week in the study period

Twitter Message Sentiment over Time: Takeaways

■ The overall pro-choice/pro-life trend:

► For much of August and early September, the proportion of pro-choice tweets was much higher

than that of pro-life.

► Starting in early/mid-September, there was a steady rise in the proportion of pro-life tweets. By

late October, pro-life tweets became slightly more common than pro-choice tweets.

► Starting in the days immediately following the midterm elections (November 6), the proportion of

pro-life tweets decreased substantially, and the proportion of pro-choice tweets returned to its

previous peak from the first week of September.

■ What explains the trend?

► Hypothesis 1 (Kavanaugh): In September, the Brett Kavanaugh affair generated a large amount

of abortion-related content on Twitter, sparked largely by suspicion among Democrats that

Kavanaugh was sympathetic to the idea of repealing Roe v. Wade (while also being clueless about

the medical details of abortion). Much of this content consisted of outcries from the political left

about the importance of Roe v. Wade.

● This theory could partially explain the rise in pro-choice content during late August and early

September.

● However, the Kavanaugh hearing did not take place until September 24, so the timeline is not

exactly as one would expect under this theory.



► Hypothesis 2 (pro-life activism): For most of September and October, the proportion of pro-life

content grew steadily, as the proportion of pro-choice content shrunk at a similar (albeit less

consistent) rate. This trend could be the result of a targeted strategy from religious and pro-life

advocacy groups and individuals to spread pro-life messaging across Twitter, in an effort to

influence the results of abortion-relevant races and ballot measures across the country in the

direction of pro-life policies.

● This theory would explain why the proportion of pro-life content grew in the weeks leading

up to the election (the period during which voting decisions are finalized).

● It would also explain why the proportion of pro-life content dropped immediately following

the election (after which point votes had been cast and there was no strategic political reason

to push partisan content).

Next we’ll inspect the state-level trend of pro-choice/pro-life sentiment, which will give us finer-grained

information to adjudicate between the two explanations of the trends offered here. While the Kavanaugh

confirmation hearing and associated news cycle was largely responsible for the growth in pro-choice

sentiment during September, examination of the state-level trends suggests that the rise of pro-life

sentiment in October was due at least in part to advocacy for pro-life political causes proliferated on

Twitter.

Twitter: State-level Trends in Pro-choice Versus Pro-life Sentiment

The following plot shows the weekly percentage of tweets with pro-choice and pro-life sentiment, for

each state of interest. This is equivalent to the overall plot above, but broken down by state.



Valence of abortion-relevant tweets by state at each week in the study period

Twitter Message Sentiment by State over Time: Takeaways

■ The rise in pro-life sentiment leading up to the election was strongest in West Virginia.

► At its September-November peak, pro-life sentiment in West Virginia was above 40 percent—a

percentage never reached in the entire study period in either of the other states.

► For six consecutive pre-election weeks (weeks of September 24 through October 29), pro-life

sentiment was more prevalent than pro-choice sentiment in West Virginia. No other state

experienced more than three such consecutive weeks.

► The drop-off in pro-life sentiment after the election was also the most dramatic in West Virginia,

where the pro-life sentiment rate is cut nearly in half in the week following the election.

■ West Virginia is the only one of the three states that had an explicitly abortion-related ballot measure

during the midterm election (West Virginia Amendment 1). Therefore, it is likely the state in which

abortion was the most salient as a political issue (see population-adjusted by-state volume graph

above for further evidence of this).

■ The theory of coordinated pro-life campaigning on Twitter neatly explains why the rise in pro-life

sentiment was most extreme in West Virginia: groups would have concentrated efforts in the place

where there was a specific ballot measure they could encourage voters to take a position on.



■ This theory also explains why the immediately post-election drop in pro-life sentiment was most

extreme in West Virginia: after the election, ballots had been cast and the strategic reason to push

pro-life messaging disappears.

Twitter: Top National Hashtags

Hashtags represent a signpost for conversational threads on Twitter. They enable people to find content of

interest by searching a hashtag and indicate to others that they are participating in a larger discourse by

flagging their post with a hashtag.

The top five national hashtags included “prolife” (239,743 tweets), “abortion” (95,548 tweets),

“Kavanaugh” (91,753 tweets), “MAGA” (66,238 tweets), and “StopKavanaugh” (66,139 tweets).

Top 50 national hashtags (larger font indicates greater number of posts using that word as a hashtag)



Study Summary

Across this three-state case study, there is mixed evidence that a candidate’s pro-life or pro-choice

position and communication of this position affected electoral chances. In Illinois and West Virginia,

majorities could not accurately describe the candidate with views discordant with their party’s

conventional view on abortion. That being said, Republican pro-choice voters were more likely to vote

across party lines than Democratic pro-life voters in each state. In Illinois and Nevada, pro-choice

attitudes were associated with pro-choice, Democratic candidates winning office, despite controlling for

other factors. And while the Democrat in West Virginia was pro-life, he did eventually communicate his

position against the ballot inititive and held his seat against a staunchly pro-life Republican challenger.

The evidence suggests that communication of abortion position did not hurt pro-choice candidates.

There is also mixed evidence that abortion position motivated vote choice. Across states, abortion was

predictive of vote choice even when controlling for partisanship and other demographics, but so are other

issues, such as immigration and gun control/gun rights. Partisanship was always more predictive of vote

choice. In addition, gun control/gun rights and immigration were more highly associated with vote choice

in a number of cases. In each state however, abortion ranked relatively low on issue importance, and there

were few single issue voters. A conglomerate of issue positions, including abortion position, affected vote

choice.

The issue of abortion did not seem to mobilize pro-choice and pro-life voters differentially. Voter turnout

and campaign participation were similar between pro-life and pro-choice voters. Still, in each candidate

race, pro-choice Republicans were more likely to support the Democratic candidate than pro-life

Democrats were to support the Republican candidate.

The ballot initiative on the issue of abortion had little interaction with the candidates’ positions. Despite

the ballot measure in West Virginia being solely about abortion, the candidates did not spend much time

discussing it during the campaign relative to other issues, and the abortion issue was not a top priority for

voters. Voter contact mattered significantly in voting for Amendment 1, but not among a subset of pro-

choice voters.

This study has many strengths in the comprehensive nature of the data and the research design. The multi-

method research design allowed for a complex contextualization of public opinion surrounding abortion

and the ways candidates and campaigns directly acted and communicated positions. It also allowed us to

assess attitudes before the campaign was fully underway, during the height of the campaign, and after

voters cast their ballots. The unusual case of having two pro-choice candidates head-to-head, two pro-life

candidates head-to-head, and a third race with one pro-life and one pro-choice candidate, also allowed us

to compare results with varied electoral contexts.

The study was somewhat limited by the fact that abortion was not a very salient issue in the election. In

two states, Illinois and Nevada, candidate communication about abortion was relatively low, and in the

third, West Virginia, a majority of voters did not accurately understand candidate positions. Individual

state- and candidate-level dynamics across the races studied also limits our findings—in a perfect

experimental setting, we would run each of these races in the same state, with the same political context,

and with the same demographic composition of the electorate.



There are interesting opportunities for future research. We believe these data can be leveraged to

understand policy discordant voters. In addition to questions on knowledge of candidate position, the

survey also asked questions on voters’ issue positions. We believe there is a large opportunity to profile

voters who voted against their policy preferences and understand if misunderstanding of candidates,

contact with campaigns, or demographics drove their decision to vote. The large variety of data sources

also offers a great chance to conduct more complex analyses using data linkage to understand if there

were temporal relationships between spending and advertising and beliefs and positions in survey data

and on social media.

Overall, the results show differences in motivations between pro-choice and pro-life voters that are

consistent across the three states and campaigns. Several multi-item questions revealed underlying values

and pre-dispositions around race, ethnicity, and feminism. Even controlling for demographics and

partisanship, pro-choice voters tend to hold more liberal attitudes on race/ethnicity and women. These

data suggest that additional issue framing and message testing research could identify abortion messages

that resonate with different types of voters.

Methodology

This study was funded by the Susan Thompson Buffett Foundation and conducted by the Public Affairs

and Media Research Department and Social Data Collaboratory at NORC at the University of Chicago.

Qualitative Campaign Monitoring

The goal of the campaign monitoring process was to have a deep, qualitative understanding of the

campaign context as it related to the topic of abortion. The data collected provides a descriptive summary

of how central abortion was to the campaign and the extent to which it was discussed over time.

We collected a range of materials meant to capture the main ways that an election campaign is framed for

and communicated to voters. Specifically, we collected the content of campaigns’ and relevant

organizations’ websites, Facebook pages, and emails; debate transcripts; political advertisements; Tweets

from the candidates’ accounts; and media coverage of the campaign. We also evaluated the source of

campaign contributions for the candidates and recorded a timeline of major occurrences in each of the

campaigns to provide interpretative context. For a significant portion of the material collected, we worked

with a data analytics firm to collect the data.

Using content analysis, we evaluated the volume of campaign communications, ads, and media coverage

that covered the topic of abortion and whether the focus on the topic was sustained throughout the

campaign.

Collection and Analysis Processes

The sections below describe the collection and analysis processes for each type of data collected. In many

of the datasets, we searched the content for mentions of abortion with word searches. To be classified as

having mentioned the topic, a material had to include one of the following words: abort* as a word stem,



pro-life, pro-choice, Planned Parenthood, Emily’s list, Hyde Amendment, reproductive, 20 week ban,

right to life, SJR12 and Amendment 1 (in West Virginia), and HB40 (in Illinois).

Campaign and relevant organizations’ websites and Facebook pages

The Virtual Tracker data were collected by a data analytics firm. The timeframe for collection of the data

was September 11, 2018 until November 6, 2018, Election Day. We followed the websites and Facebook

pages of candidates and other important players in the campaigns. The Virtual Tracker monitored these

websites by taking screenshots each time the websites were updated or changed in any way. The data

show each change of additional text or deletion of text in the sites over time. This allowed us to monitor if

candidates mentioned abortion on their websites and see when this content was added or changed. Each

website and its pages were recorded in separate datasets.

For each unique page of the websites we tracked, we analyzed whether it mentioned abortion at any point

during the time we were tracking changes. We then calculated the percentage of unique pages of the

candidate’s website and Facebook profile that mentioned abortion.

Campaign and relevant organizations’ emails

We collected emails from each campaign and from relevant organizations working on the issue of

abortion in the state. The timeframe for collection of the data was August 24, 2018 until November 6,

2018, Election Day. We received the emails by signing up for the respective campaign or organization

distribution list. For each email sent, we analyzed whether it mentioned abortion. We then calculated the

percentage of emails sent that mentioned abortion.

Below is a list organizations and the number of emails collected.

■ West Virginia

► Manchin: 38

► Morrisey: 33

► West Virginia for life: 15

► West Virginia Free: 12

► Vote no on 1: 7

► West Virginia Dems: 2

► Planned Parenthood Action local West Virginia: 2

■ Illinois

► Rauner: 62

► Pritzker: 41

► Illinois Family: 51

► Personal PAC: 41

► Planned Parenthood Action Illinois: 19

► ICE PAC: 10

► Democratic party of Illinois: 1



► Illinois GOP: 1

■ Nevada

► Rosen: 490

► Heller: 140

► Nevada Democrats: 81

► Nevada GOP: 1

Debate transcripts

Debates in all three states were transcribed by a data analytics firm. The West Virginia U.S. Senate debate

between Manchin and Morrisey is 10,969 words transcribed and occurred on November 1, 2018. The

Nevada U.S. Senate debate is 10,892 words and occurred on October 19, 2018. The three Illinois

gubernatorial debates between Pritzker and Rauner totaled 32,357 words and occurred on September 20,

October 3 and October 11, 2018.

For analysis, the debate transcripts were read into R with the textreadr package and analyzed for word

count and frequency of abortion mentions with the tidytext library. Transcripts were read through to

ensure accuracy of the search.

The debate transcripts were also analyzed for frequency of mentions of three other topics – economic

issues, guns, and immigration. The words used to identify mentions of these topics are listed below.

■ Economic issues: the word stems econo*, unemploy*, tax*, and wage*, as well as budget, jobs, class,

inequality, deficit, debt

■ Guns: the word stems gun* and background check*, as well as second amendment and NRA

■ Immigration: the word stems immigr* and caravan*, as well as citizenship, family separations,

separating families, families separated, dreamers, and borders

Political advertisements

Political advertisements run on broadcast TV were collected and analyzed by Kantar Media. The

timeframe for collection of the data was August 1, 2018 until November 6, 2018, Election Day. Kantar

tracks every single ad run, whether it was a unique ad or a rerun of an earlier ad, who sponsored the ad,

and what topics were covered in the ad, among other information.

The data provided were analyzed in R and visualized in dygraphs javascript charting library. Based on

Kantar’s content coding of the ads, we calculated the percentage of ads run and of ad spending that

addressed the issue of abortion, broken down by the sponsor of the ad. We also summarized the number

of ads run that focused on abortion over time.

Campaign, candidate and relevant organization tweets

A data analytics firm provided tweets for all of the candidate and organization Twitter accounts we

requested for Illinois, Nevada and West Virginia. The timeframe for collection of the data was July 1,



2018 until November 6, 2018, Election Day. The tweets provided were analyzed in R for mentions of

abortion-related words.

We also used the Twitter REST API and rtweet package to scrape tweets for all major party Senate

candidates for the 2018 election campaign across all states. We searched these 150,000 tweets for

mentions of abortion-related words to provide context for the candidates’ frequency of tweeting about the

topic.

Media coverage

We used Cisionpoint and Critical Mention to collect information on the news media coverage around the

campaigns. The timeframe for collection of the data was August 1, 2018 until November 6, 2018,

Election Day.

Cisionpoint was used to collect print and online articles and media related to the campaigns. Short

sections of articles were used for content analysis. In Illinois there were 49,064 mentions of Rauner or

Pritzker in print and online content. Nevada Senate candidates had 142,796 mentions. West Virginia

Senate candidates had 61,648 mentions. All searches through Cision were conducted on the last name of

each candidate and then further filtered to include only mentions of the candidate’s full name. Cision data

were analyzed via R with the tidytext library.

Critical Mention was used to collect mentions of the candidates on television and radio broadcasts. Short

transcripts of broadcasts were used for content analysis. For the Illinois Gubernatorial candidates, there

were 24,845 mentions, 43,913 mentions for Nevada U.S. Senate candidates, and 24,849 mentions for

West Virginia U.S. Senate candidates. Critical Mention data was collected for all mentions of both the

candidate’s first and last name. The data included common misspellings of candidates’ last names, such

as “Morrissey” and “Morissey.” Critical mention data were analyzed via R with the tidytext library.

We collected mentions of the West Virginia Amendment 1 in the Critical Mention database, which allows

for more complex word searches. This search was conducted with the following Boolean search terms:

“west virginia” AND (abort* OR “hyde amendment” OR “pro life” OR “pro choice” OR

reprod* OR “planned parenthood” OR “pro-life” OR “pro-choice”) OR “West virginia free”

OR “West Virginians for Life”

This search was designed to represent any mentions of West Virginia and abortion-related terminology or

direct mentions of Amendment-1-related organizations.

Media clips were then analyzed for mentions of abortion-related terms. This generated a percentage of

news media mentions of the candidates that focused on the topic of abortion.

Campaign contributions

A data analytics firm collected all contributions to the federal campaign candidates from the Federal

Election Commission. The timeframe for collection of the data was Q1 2018 through the end of the Q4

reporting period. Data on contributions for the Illinois governor’s race came from the Illinois State Board

of Elections. In the campaign contributions data, we coded whether the source of the contribution was an



organization that advocates or lobbies on the issue of abortion, and then calculated the percentage of all

contributions to each candidate that came from abortion-related organizations.

Twitter Analysis

Data Collection

Given the vast diversity of users and messages on social media platforms, finding a relevant signal in the

noise can be a challenge. Also, with rate limits and other platform restrictions on data collection,

researchers need to be careful to ensure that the data collected and analyzed are useful to answer the types

of questions being asked. The Twitter platform, being principally regarded as a public forum, provides

data access options that facilitate targeted, yet thorough, collection for research. The Twitter Historical

Powertrack was used to collect the data for this project, which applies a collection of targeted queries,

called search rules, retrospectively on the entire archive of public tweets during a specified time period.

The data collection strategy for this project was conceptualized in two primary themes: abortion-related

content, and specific political contest content. These two themes were operationalized across four

archives and 59 individual search rules, which were understood to have some overlap. The time period

was 7/1/2018 – 12/10/18 in order to capture the run up to voting day as well as 1 month post-election. We

created four Archives of relevant data for this project. The table below illustrates the structure of the data

sets.

The first dataset type, covering general abortion-related content, included 4 datasets. The primary body of

each search rule included multiple terms which were specifically related to abortion, such as “abortion,”

“pro-life,” and “pro-choice.” Each term that was included in the search rules was first entered into Twitter

search to review the tweets returned, confirm the term has high relevance, and scan for related terms in

the tweet content. Several terms were tested, but not included, due to mixed relevance. These terms, such

as “women’s health” and “planned parenthood,” often related to abortion, also introduced themes which

were not abortion-related, such as maternal mortality. Prior experience with the challenges of cleaning

data after collection incentivized the adoption of search rules that were specifically targeted and highly

relevant to the theme at the step of data collection.



Archive 1: We constructed a corpus of tweets containing covered general abortion-related content. The

primary body of each search rule included multiple terms which were specifically related to abortion,

such as “abortion,” “pro-life,” and “pro-choice.” Each

term that was included in the search rules was first entered

into Twitter search to review the tweets returned, confirm

the term has high relevance, and scan for related terms in

the tweet content. Several terms were tested, but not

included, due to mixed relevance. These terms, such as

“women’s health” and “planned parenthood,” often related

to abortion, also introduced themes which were not

abortion-related, such as maternal mortality. Prior

experience with the challenges of cleaning data after

collection incentivized the adoption of search rules that

were specifically targeted and highly relevant to the theme

at the step of data collection. We also created subsets of

Archive 1, using geography filters for each study state.

This allowed us to explore what portion of abortion-related

content in each state intersected with each campaign, as

well as a comparison of general attitudes towards abortion

across states.

Archive 2 focused on specific campaign accounts of the campaigns of interest for this research. Specific

twitter accounts, including official, campaign, and personal accounts which were verified by Twitter, for

each candidate in each race of interest were identified and the search rules specifically pulled tweets

posted by those accounts. Additionally, to the extent that official twitter accounts existed for ballot

measures in West Virginia, tweets posted from those accounts were also collected.

Archive 3 included campaign-related posts beyond those coming specifically from the candidates

themselves. Again, given the possibility of so much noise coming from general discussion of politics, the

rules focused on capturing posts mentioning the specific campaign accounts in the contests of interest,

those posts which included a link to an official campaign website, and posts that used campaign-related

hashtags used by candidate accounts.

Archive 4 focused on state-specific political organizations and abortion-related organizations. These

search rules included mentions of official accounts representing state-based political parties or abortion

advocacy groups in the states-of-interest, and also posts which include links to the official websites of

these state-based advocacy groups.

Search Rule Strategy by Archive

Archive 1: Terms testing for specific abortion-related content; subsets of Archive 1 used operators to filter users from each study state

Archive 3: Official campaign accounts

Archive 3: State politics, including tweets that contained campaign hashtags and URLs

Archive 4: Advocacy group inventory to identify official, verified state-based abortion advocacy groups and political party accounts and pages



Comprehensiveness of the keywords

As an additional quality control check regarding the comprehensiveness of the search terms which were

adopted for this research, some terms which were not adopted, yet could be relevant, were tested more

thoroughly. This was done by scanning the public 1 percent twitter stream (a 1 percent sample of all

tweets across Twitter), and cross checking tweets found there both for relevance and also against the

archive created for this research. The proportion found from the 1 percent which was also present in the

project archive allowed an estimate of the volume that would have been added, had these terms been

adopted. The additional terms tested for this including variations with and without a space of “planned

parenthood,” “my body my choice,” “women health,” and “reproductive right.”

The table below suggests that, of these terms, the only concept which would have provided a meaningful

amount of relevant content was “planned parenthood” (or “plannedparenthood”). However, due to the fact

that only a fraction of these data were clearly relevant to abortion, this additional volume would have also

introduced non-relevant noise. Without an adequate relevance-cleaning process, which can be quite

cumbersome, this extra noise may have distorted some results.

Dataset Type Dataset Tweet Count

d01_abortion_allgeo 16,064,399

d01_abortion_IL 211,555

d01_abortion_NV 59,576

d01_abortion_WV 24,456

d02_accounts_IL_gov 1,527

d02_accounts_NV_gov 2,159

d02_accounts_NV_sen 4,151

d02_accounts_WV_amendment1 3,580

d02_accounts_WV_sen 983

d03_hashtags_keywords_IL_gov 214,329

d03_hashtags_keywords_NV_gov 150,123

d03_hashtags_keywords_NV_sen 1,055,647

d03_hashtags_keywords_WV_amendment1 27,371

d03_hashtags_keywords_WV_sen 1,295,858

d03_websites_IL_gov 4,261

d03_websites_NV_gov 5,008

d03_websites_NV_sen 24,291

d03_websites_WV_amendment1 1,010

d03_websites_WV_sen 4,296

d04_groups_IL 3,893

d04_groups_NV 2,432

d04_groups_WV 4,126

d04_groupsites_IL 2,999

d04_groupsites_NV 26,060

d04_groupsites_WV 548

General

Abortion

Keywords

Specific

Campaign

Accounts

State

Campaign

Politics

State

Abortion

Advocacy

Groups



Key term quality control questions:

Do the tweets that match these terms were retrieved from Historical PowerTrack?

Are they abortion-related?

Keywords Match in

Streaming Data Match in PowerTrack

(our DB) Relevance in

sample Projected Amount of

Relevant Tweets

Planned parenthood 16,488 3,169 50% 665,950 (4.1%)

My body my choice 338 79 68% 17,742 (0.1%)

Women health 483 4 0% 0

Reproductive right 50 28 100% 2,200 (0.01%)

The tweets that match “planned parenthood” would likely have influenced the amount of tweets we collected. We observed that a tweet is often relevant to abortion when the term is mentioned in the perspective of pro-life or anti-abortion although it is tricky otherwise.

Data Collection Limitations

Data collection has several limitations. Twitter users are not representative of the general population, or

of registered voters. Due to the company’s protective policies it can be difficult to know precise

demographics of any subset of the Twittersphere and therefore adjust outcomes to represent general

populations. Additionally, the company’s diligence in purging bots and bad actors (those who violate

Twitter terms & conditions) may result in older portions of the historic timeline containing fewer bots and

bad actors than more recent portions of a timeline. However, Twitter, remains a leading social media

network for thought leaders and influencers who have the potential to sway the opinion of their followers.

Further, the network allows full access to researchers unobstructed by privacy settings.

Data collection was defined by the keywords and search filters used to retrieve data from Twitter. As

such, search filters are the lens through which we can observe what and how people communicate. This

lens should be appropriately focused, so we can identify the content of interest without the noise of

unrelated conversation. If a search is too narrow, it may miss important data, and conclusions may be

biased. Conversely, if it is too broad, there is a risk collecting irrelevant and potentially misleading

material.

Given the limited budget and time, this project used a pre-defined and restricted set of search terms and

hashtags, which may have missed some abortion-related Twitter content. For instance, under the

campaign datasets, the decision to search based on hashtags which appeared on official campaign posts as

opposed to any hashtag which could be campaign relevant insured a high precision of the campaign

hashtag data, but likely excluded posts featuring more obscene and vitriolic hashtags than those which the

campaigns themselves would publish. In addition, other abortion related terms like “reproductive rights,”

“women’s health”, and “planned parenthood” were excluded because these terms are more expansive and

include health related topics other than abortion. To compensate for the commitment to precise keywords,

a large number of search terms were applied, capturing the data from multiple dimensions (e.g. account

handles, hashtags, and URL links). The abortion terms, specifically, are discussed under the

“comprehensiveness of the keywords” heading.

Twitter: Sentiment Classification

We developed supervised classification models to categorize tweets as expressing a pro-choice sentiment,

a pro-life sentiment, or neither. The model development process consisted of first hand-coding several



thousand tweets, and then training binary classifiers to predict the human-assigned categories as

accurately as possible (one to predict pro-choice sentiment, one for pro-life sentiment, and one for

"neither"). After optimal model parameters were determined (given the available training data), we

generated predictions over all abortion-relevant tweets collected from Illinois, Nevada, and West

Virginia. Predicting the sentiment of a tweet was a two-step procedure: tweets were first screened for

“polarity”, meaning that they express a (subjective) opinion or attitude about the issue of abortion. Tweets

not deemed to express an opinion/attitude were categorized as “neither.” The remaining tweets were then

categorized as either “pro-choice” or “pro-life,” depending on which category's predicted probability was

higher.

The three broad sentiment categories are defined as follows:

■ ‘Pro-choice’ category: A tweet is counted as “pro-choice” if it promotes the concept of (access to)

abortion as a healthcare or women’s rights issue, if it expresses support for pro-choice organizations

or political candidates, or if it disparages pro-life positions, organizations, politicians, etc. Tweets are

also counted as pro-choice if they express support for a political candidate on the grounds of their

pro-choice positions on abortion.

■ ‘Pro-life’ category: A tweet is counted as “pro-life” if it frames abortion as a moral or religious

issue as opposed to a healthcare issue, if it frames abortions as “murder” or fetuses as “tiny lives,” if it

disparages abortion providers or politicians with pro-choice stances, etc. Tweets are also counted as

pro-choice if they express support for a political candidate on the grounds of their pro-choice

positions on abortion.

■ ‘Neither’ category: A tweet is counted as “neither” if it does not clearly express a sentiment,

attitude, or opinion in favor of a pro-choice or pro-life position. About 40% of human-labeled tweets

were categorized as “neither.” A label of “neither” doesn’t mean a tweet is irrelevant to abortion, but

rather that the tweet text itself does not express an attitude or opinion about abortion. For example

news headlines would count as neither, as would jokes or irrelevant passing references to abortion.

Our sentiment model aimed to identify tweets the text of which takes an explicit position on the

political/ethical/medical issue of abortion, and so many abortion-relevant tweets simply do not fall

into the category of “pro-choice” or “pro-life” – such tweets fall into the elsewhere-category of

“neither.”

Source Coding

Source coding was performed by humans, hand-coding several thousand tweets to determine the type of

user that created the tweet. For all individual verified and unverified users, the content of tweets were

assessed in order categorize the user as one of the following types:

■ News organization and news media professionals: Any account that represents a news organization, a

freelance journalist, or an employee, reporter, or correspondent of a news organization.

■ Abortion advocacy group and advocate: Any account that represents an advocacy group,

organization, advocate, or activist that focuses on abortion-related issues.

■ Political organization: Any account that represents a political organization.



■ Other Group/Organization: Any group or organization account that does not fit the categories above,

i.e., an organization that is not a news organization, abortion advocacy group, or political

organization.

■ Celebrity: Any account that belongs to a famous person in the entertainment or culture industry,

including but not limited to: writers, actors, directors, producers, musicians. The account was required

to either be a verified user, or have 8,000+ followers. Note that celebrities working in news media

were not in this category, and instead would be labeled into the category for news organization and

news media professionals.

■ Politician: Any account that belongs to a person who is a politician or a candidate for public office.

■ Other Individual/Person: Any individual that is not celebrity or politician.

Source Sample

A total of 4172 accounts were collected among in the state abortion-related tweets for source coding.

These include 518 verified account, which represent all verified accounts found in the data. Also included

are 3654 influential accounts based on number of followers (at least 3223, the top 10 percentile within the

relevant data) and ratio of follower to following (greater than 1). For each of those account, the account

metadata and all of their tweets were collected into a spreadsheet for coders to perform source coding.

Bot Filter

A total of 69,017 unique users found in our state abortion-related data was sent to Botometer API20

through their official Python client library21 for bot detection.

Botometer is a collaboration between Indiana University Network Science Institute (IUNI) and the Center

for Complex Networks and Systems Research (CNetS).22 It detects bot using a machine learning

algorithm trained on a dataset of 5.6M tweets coming from 15k verified bots and 16k verified human

accounts. The algorithm generates a Complete Automation Probability (CAP), which is the probability

that an account is a bot, based 6 type of characteristics extract from the account:

■ Network: based on retweets, mentions, and hashtag co-occurrence.

■ User: based on Twitter meta-data related to an account, including language, geographic locations, and

account creation time.

■ Friends: based on an account’s social contacts.

■ Temporal: based on timing patterns of content generation and consumption.

■ Content: based on linguistic characteristics of the tweets from the account.

■ Sentiment: based on general sentiment detection of the tweets from the account and usage of

emoticon

After obtaining CAP for all of the accounts, we used a cut-off point of 0.5 to flag an account as bot. This

yield 1030 bots. The number seems low because bot detection algorithm tends to be conservative. As the

20 https://botometer.iuni.iu.edu/#!/api

21 https://github.com/IUNetSci/botometer-python

22 https://arxiv.org/abs/1602.00975

https://botometer.iuni.iu.edu/#!/api

https://github.com/IUNetSci/botometer-python

https://arxiv.org/abs/1602.00975



website of Botometer states: “CAP may look like a conservative estimate, but one must consider that

there are a lot more humans than bots, so you'd better be sure before accusing someone of being a bot!”

Top terms and hashtags by source

All of the tweets coming from each source type are used to count top terms and hashtags by source. For

hashtags we use the meta data field provided by GNIP, which contains a list of all hashtags found in each

tweet, if any. For top terms, we first remove hashtag, mention, URLs, punctuation, and stop words before

conducting the count.

Identifying West Virginia Amendment 1 Predictive Terms

A Support Vector Machine model was fitted for Support vs. Not-Support and Oppose vs. Not-Oppose

using the Unigram, Bigram, and Trigram from the tweets body as features. Both model achieved an

overall F-1 score of 90% on 5-fold cross-validation. The top 10 most predictive features from the model

for Support and Oppose are reported.

State-Level Surveys

The study included three Waves of surveys among adults 18 and older who are registered voters in

Nevada, Illinois, and West Virginia. Respondents came from two samples – the AmeriSpeak panel and

TargetSmart’s VoterBase.

AmeriSpeak Sample

Funded and operated by NORC at the University of Chicago, AmeriSpeak® is a probability-based panel

designed to be representative of the US household population. Randomly selected US households are

sampled using area probability and address-based sampling, with a known, non-zero probability of

selection from the NORC National Sample Frame. These sampled households are then contacted by US

mail, telephone, and field interviewers (face to face). The panel provides sample coverage of

approximately 97 percent of the U.S. household population. Those excluded from the sample include

people with P.O. Box only addresses, some addresses not listed in the USPS Delivery Sequence File, and

some newly constructed dwellings. While most AmeriSpeak households participate in surveys by web,

non-internet households can participate in AmeriSpeak surveys by telephone. Households without

conventional internet access that have web access via smartphones are allowed to participate in

AmeriSpeak surveys by web. For this study, survey respondents were offered a small monetary incentive

per Wave of the survey they completed.

TargetSmart Sample

To supplement the surveys conducted via AmeriSpeak in each state, sample lists of registered voters were

obtained from TargetSmart, a commercial database vendor. For the first Wave, a mail campaign

comprised of an early-notification postcard and a follow-up invitation packet with a $1 bill was used to

recruit registered voters. Respondents were invited to complete the survey either on the web at a study-

specific website or via phone using a study-specific toll-free number. Respondents were given a personal



identification number and offered a monetary incentive as a thank you for completing the survey. An

example of the recruitment brochure is in the table below.

For the second and third Wave, qualified respondents who completed the first survey and gave permission

to be recontacted were mailed an invitation packet with a $2 bill and sent an email invitation with a link to

the survey. The identity of the respondent was confirmed before rescreening for registration status. For

each Wave, respondents were sent multiple reminder emails.



Example Recruitment Brochure: Illinois



Field Period, Respondent Screening and Weighting

Respondents from the AmeriSpeak panel and those recruited from the TargetSmart sample were surveyed

in three Waves. The first Wave was conducted from August 13 to August 24, 2018. The second Wave

was completed just before Election Day, fielding from October 24 to November 5, 2018. And the final

Wave of the survey was a post-election survey conducted from November 7 to November 21, 2018.

Respondents in the first Wave were screened for whether they reported being registered to vote or

planning to register to vote before Election Day. For the second Wave, respondents were reasked whether

they were registered to vote only in Nevada and West Virginia. In Illinois, they were also reasked whether

they intended to register, since it is possible to register up until Election Day in Illinois. For the third

Wave, respondents were screened for self-reported voter registration only.

Surveys were provided in both English and Spanish, depending on the preference of the respondent.

Once the study data had been collected and made final, the two samples were weighted together to create

a sample that is representative of registered voters in each state. The samples were weighted for

probability of selection and poststratified to adjust for any survey nonresponse as well as any

noncoverage or under- and over-sampling resulting from the study-specific sample design.

Poststratification variables included age, gender, race/ethnicity, and education. Weighting targets were

obtained from the 2016 Current Population Survey voter supplement. The weighted data, which reflect

Illinois, West Virginia, and Nevada registered voters, age 18 and older, were used for all analyses.

The tables below report the sample sizes, margins of error and response rates by state for each Wave of

the survey.

Illinois Wave 1 Wave 2 Wave 3

Field dates in 2018 8/8 - 9/4 10/23 - 11/5 11/7 - 11/21

Overall sample size 3,378 2,383 2,525

Online completes 3,202 2,316 2,463

Phone completes 176 67 62

AmeriSpeak completes 309 250 237

TargetSmart completes 3,069 2,133 2,288

Design effect 1.41 1.66 1.60

Margin of Sampling Error (pct. pt) +/- 2.0 +/- 2.6 +/- 2.5

Cumulative response rate (%) 8.8 6.2 6.6

AmeriSpeak: Final stage completion rate (%) 92.0 96.2 97.1

AmeriSpeak: Weighted HH panel response rate (%) 28.6 28.6 28.6

AmeriSpeak: Weighted HH panel retention rate (%) 86.0 86.0 86.0

AmeriSpeak: Cumulative response rate (%) 7.6 6.2 5.9

TargetSmart: Final stage completion rate (%) 91.9 97.0 98.7

TargetSmart: Cumulative response rate (%) 8.9 6.2 6.7



Nevada Wave 1 Wave 2 Wave 3

Field dates in 2018 8/8 - 9/4 10/23 - 11/5 11/7 - 11/21















West Virginia Wave 1 Wave 2 Wave 3

Field dates in 2018 8/8 - 9/4 10/23 - 11/5 11/7 - 11/21

















Analysis

All analyses were conducted using Stata (version 14), which allows for adjustment of standard errors for

complex sample designs. All differences reported between subgroups of the population are at the 95

percent level of statistical significance, meaning that there is only a 5 percent (or less) probability that the

observed differences could be attributed to chance variation in sampling. Additionally, bivariate

differences between subgroups are only reported when they also remain robust in a multivariate model

controlling for other demographic, political, and socioeconomic covariates.

Open ended survey responses were coded in two ways. Short open ends were coded completely by hand

by a single coder. Questions Q10 and Q11 from Wave 1 and Wave 2, which asked respondents to name

the top problems in the country and in their state, were coded using supervised machine learning. Data

from these open ends were trained on 20,000 responses from past AmeriSpeak Omnibus surveys asking

respondents what the top five problems were facing the nation. For Q11, an additional thousand state-

based responses for each state were included in the training data to provide more nuanced classification

for state based issues. The data were used to train a support vector machine to classify responses into

nearly 30 codes. These codes were collapsed into more general categories based on topic.

Voter Validation

For the voter validation, a vendor, L2 attempted to match each respondent’s name and address to records

on the state voter files. If a respondent matched a record on the voter file, the state voter file documents

whether or not that person voted in the 2018 election. This information was appended to the survey data.

Among those who reported voting in Wave 3 of the survey, 83 percent in Nevada, 78 percent in Illinois,

and 78 percent in West Virginia were verified as having voted in the 2018 election via this method. For

every analysis of attitudinal and behavioral differences among self-reported voters, we conducted a

robustness check on the findings by analyzing whether the differences held up among validated voters.

All key findings in the report hold up among validated voters.



Appendix

*P<.05 **P<0.01 ***P<.001

___________________________________________

1 Table represents sample among only respondents validated as voters 2 Religious refers to a binary recode of any indicated religion or an indication of no religion 3 Immigration opinion scaled from 1=strongly favor path to legality to 5=strongly oppose path to legality 4 Healthcare opinion scaled from 1=strongly agree it is the responsibility of the federal government to provide healthcare to 5=strongly disagree 5 Abortion opinion rescaled to 1=legal in all cases to 5=illegal in all cases 6 Economic importance scaled from 1=not at all important to 5=extremely important 7 Gun opinion scaled from 1=much more strict to 5=much less strict

Documents

In Focus 2018: Campaign Evaluations in West Virginia ... Focus/In Focus 2018 Executive Summary.… · 14/05/2019 · Illinois, and Nevada had an energizing, neutral, or de-energizing