Surfacing norms to increase vaccine acceptance

Alex Moehring12 Avinash Collis23dagger Kiran Garimella24dagger M Amin Rahimian25dagger Sinan Aral124and Dean Eckles124Dagger

1MIT Sloan School of Management 2MIT Initiative on the Digital Economy 3McCombs School of Business TheUniversity of Texas at Austin 4Institute for Data Systems and Society Massachusetts Institute of Technology

5Department of Industrial Engineering University of Pittsburgh

Working paperFirst version 8 February 2021

This version 8 April 2021

Despite the availability of multiple safe vaccines vaccine hesitancy may present a challengeto successful control of the COVID-19 pandemic As with many human behaviors peoplersquosvaccine acceptance may be affected by their beliefs about whether others will accept a vaccine(ie descriptive norms) However information about these descriptive norms may have dif-ferent effects depending on peoplersquos baseline beliefs and the relative importance of conformitysocial learning and free-riding Here using a large pre-registered randomized experiment(N=484239) embedded in an international survey we show that accurate information aboutdescriptive norms can substantially increase intentions to accept a vaccine for COVID-19 Thesepositive effects (eg reducing by 49 the fraction of people who are ldquounsurerdquo or more negativeabout accepting a vaccine) are largely consistent across the 23 included countries but are con-centrated among people who were otherwise uncertain about accepting a vaccine Providingthis normative information in vaccine communications partially corrects individualsrsquo apparentunderestimation of how many other people will accept a vaccine These results suggest thatpublic health communications should present information about the widespread and growingintentions to accept COVID-19 vaccines

Keywords COVID-19 descriptive norms social influence vaccine hesitancy public health

dagger AC KG and MAR contributed equally to this work and are listed alphabeticallyDagger To whom correspondence should be addressed E-mail ecklesmitedu

This preprint research paper has not been peer reviewed Electronic copy available at httpsssrncomabstract=3782082

rint n

Introduction

Nonpharmaceutical interventions in response to epidemics such as the COVID-19 pandemicoften depend on the behavioral responses of the public for their effectiveness Even with theavailability of vaccines success depends on peoplersquos choices to accept or even seek out thevaccine1 since even low vaccine refusal rates can prevent achieving herd immunity23 Given thevalue of individual autonomy and the significant challenges of imposing vaccine mandates4ndash6it is important to understand how public health messaging can increase acceptance of safeand effective COVID-19 vaccines Many messaging strategies address individual barriers tovaccination such as complacency and inconvenience7 as well as perceived risk of both vaccinesand the disease1 However these strategies may have important limitations for examplefield studies show that corrective information about vaccine safety can effectively reducemisconceptions and false beliefs though they are not as effective in changing vaccine-relatedintentions89 Messaging strategies that share recommendations from experts and emphasizereasons for accepting a vaccine have shown promising effects on increasing acceptance in theUnited States10

It may be important to look beyond individuals to consider how public health messagingcan also leverage the significant roles of social networks (broadly defined) in shaping individualvaccination decisions11ndash15 Rather than being a small factor there is growing evidence thatpeoplersquos preventative health behaviors are dramatically influenced by many social and culturalfactors with implications for COVID-1916 In the United States for example analyses ofmobility data during the COVID-19 pandemic revealed that peoplersquos mobility behaviors varywith their partisan affiliation17 and media consumption1819 and are affected by the behaviorsof their social connections20

Acceptance of COVID-19 vaccines likely involves substantial social influence but theoryis not entirely clear on whether learning that othersrsquo are accepting a vaccine will increaseor decrease acceptance Positive peer effects can arise due to information diffusion2122conformity and injunctive norms14 inferring vaccine safety and effectiveness from othersrsquochoices2324 or pro-social motivations such as altruism2526 and reciprocity27 On the otherhand negative effects of othersrsquo acceptance can arise as a result of free-riding on vaccine-generated herd immunity even if only partial or local2829 The empirical evidence on whenpositive peer effects243031 or free-riding28 may dominate is inconclusive Furthermore theeffects of incorporating truthful information about othersrsquo into messaging strategies will dependon what that information is ie how prevalent is vaccine acceptance in a given referencegroup Thus we need further empirical guidance about scalable and effective messagingstrategies leveraging social influence That is while some interpretations of the theoretical

rint n

stan Ita

Survey wave (July 2020 through March 2021) by country

Beliefs about othersrsquo vaccine acceptance

0 10 20 30 40 50 60 70 80 90 100Vaccine AcceptanceNoDont knowYes

0 10 20 30 40 50 60 70 80 90 100

Fig 1 There is substantial country-to-country variation in levels and trends in COVID-19 vaccine acceptancefrom July 2020 to March 2021 Shown are the 23 countries with repeated data collection over time ldquoYesrdquoalso includes respondents indicating they already received a vaccine (inset) Pooling data from all 23countries people who believe a larger fraction of their community will accept a vaccine are on average morelikely to say they will accept a vaccine this is also true within each included country (Figure S16)

and empirical literature could motivate emphasizing high rates of vaccine acceptance in publichealth communications little is known about how realistic interventions of using messageswith factual information about othersrsquo vaccine acceptance will affect intentions to accept thenew COVID-19 vaccines

Here we provide evidence from a large-scale randomized experiment embedded in aninternational survey that information about descriptive norms mdash what other people dobelieve or say mdash can have substantial positive effects on intentions to accept new vaccinesfor COVID-19 To our knowledge there are no other quantitative causal assessments of howexposure to factual descriptive norms affects intentions to receive the COVID-19 vaccines

Through a collaboration with Facebook and Johns Hopkins University and with inputfrom experts at the World Health Organization and the Global Outbreak Alert and ResponseNetwork we fielded a survey in 67 countries in their local languages yielding over 20 millionresponses to date32 This survey assessed peoplersquos knowledge about COVID-19 beliefs aboutand use of preventative behaviors beliefs about othersrsquo behaviors and beliefs and economicexperiences and expectations Recruitment to this survey was via prominent messages from

rint n

Facebook to its to users that encouraged potential respondents to help with research onCOVID-19 (Figure S4) While it is often impossible to account for all factors that mayjointly determine selection into the sample and survey responses our collaboration withFacebook allows using state-of-the-art privacy-preserving weighting for non-response usingrich behavioral and demographic variables as well as further weighting to target the adultpopulation of each country3233 All analyses presented here use these survey weights to ensureour results are as representative of these countriesrsquo adult populations as possible

FrancePhilippines

EgyptTurkeyPoland

RomaniaJapan

IndonesiaNigeria

United StatesArgentinaColombiaGermanyPakistanMalaysia

IndiaThailand

ItalyMexico

BrazilBangladesh

United KingdomVietnam

0 25 50 75 100

Belief about community descriptive norm

Belief compared with countryminuswide descriptive norm

below narrow between narrow and broad above broad

Fig 2 Within-country distributions of beliefs about descriptive norms (ldquoOut of 100 people in your communityhow many do you think would take a COVID-19 vaccine if it were made availablerdquo) during the experimentalperiod (October 2020 to March 2021) To enable comparison with actual country-wide potential vaccineacceptance these histograms are colored by whether they are below (red) the narrow (ldquoYesrdquo only) definitionof vaccine acceptance between (yellow) the narrow and broad (ldquoYesrdquo and ldquoDonrsquot knowrdquo) definitions or above(teal) the broad definition

This survey has documented substantial variation in stated intentions to take a vaccinefor COVID-19 when one is available to the respondent with for example some countrieshaving much larger fractions of people saying they will take a vaccine than others (Figure1) however a plurality consistently say they will accept a vaccine and only a (often small)minority say they will refuse one This is consistent with other smaller-scale national1034 and

rint n

international35 surveys There is also substantial variation in what fraction of other peoplerespondents think will accept the vaccine and these beliefs often substantially differ fromcountry-wide levels of vaccine acceptance (Figure 2) This deviation can have multiple causesincluding responding with round numbers but we posit this is at least partially because somepeople have incorrect beliefs about descriptive norms Underestimation of vaccine acceptanceby others could be partially caused by processes mdash such as news coverage of the challengesposed by vaccine hesitancy or diffusion of anti-vaccine messages on social media mdash that makehesitancy more salient Beliefs about descriptive norms are in turn positively correlated withvaccine acceptance (Figure 1 inset Figure S16) likely reflecting many processes like geographicand social clustering of vaccine hesitancy but also causal effects of beliefs about others onintentions to accept a vaccine Public health communications could present information aboutnorms perhaps correcting some peoplersquos overestimation of the prevalence of vaccine hesitancyUnlike other ongoing frequently observable preventative behaviors like mask wearing peoplemay have little information about whether others intend to or have accepted a vaccine mdashwhich suggests messages with this information could have particularly large effects

Randomized Experiment

To learn about the effects of providing normative information about new vaccines beginningin October 2020 for the 23 countries with ongoing data collection in this study we providedrespondents with accurate information about how previous respondents in their country hadanswered a survey question about vaccine acceptance mask wearing or physical distancingWe randomized at what point in the survey this information was provided which behavior theinformation was about and how we summarized previous respondentsrsquo answers mdash enabling usto estimate the effects of providing information about descriptive norms on peoplersquos statedintentions to accept a vaccine

In the case of vaccine acceptance we told some respondents ldquoYour responses to this surveyare helping researchers in your region and around the world understand how people areresponding to COVID-19 For example we estimate from survey responses in the previousmonth that X of people in your country say they will take a vaccine if one is made availablerdquowhere X is the (weighted) percent of respondents saying ldquoYesrdquo to a vaccine acceptance questionOther respondents received information on how many ldquosay they may take a vaccinerdquo which isthe (weighted) percent who chose ldquoYesrdquo or ldquoDonrsquot knowrdquo for that same question Whether thisinformation occurs before or after a more detailed vaccine acceptance questionlowast and whether

lowastWhen the detailed vaccine acceptance question occurs after the normative information it is always separated by at leastone intervening screen with two questions and it is often separated by several screens of questions (Figure S15a) Mostquestions were unrelated to social norms or vaccines32

rint n

Nigeria

United Kingdom

Germany

Brazil

Thailand

Romania

Colombia

Bangladesh

France

United States

Turkey

Poland

Malaysia

Argentina

Mexico

Philippines

Indonesia

Vietnam

Pakistan

Average

minus005 000 005 010 015Effect on vaccine acceptance scale

BroadNarrow

Vaccine acceptance scale

Narrow

Norms treatment

(b) (c)minus002 000 002 004 006 008Effect on vaccine acceptance scale

Between

Dont know

Average

Nodefinitely not

Probably not Unsure Probably Yesdefinitely

50 ControlOther BehaviorBroadNarrow

Fig 3 (a) The normative information treatments shift people to higher levels of vaccine acceptance whethercompared with receiving no information (control) or information about other non-vaccine-acceptance norms(other behavior ) (b) These estimated effects are largest for respondents who are uncertain about acceptinga vaccine at baseline and respondents with baseline beliefs about descriptive norms that are under (ratherthan above or between) both of the levels of normative information provided in the treatments (c) While thereis some country-level heterogeneity in these effects point estimates of the effect of the broad normativeinformation treatment are positive in all countries Error bars are 95 confidence intervals

it uses the broad (combining ldquoYesrdquo and ldquoDonrsquot knowrdquo) or narrow (ldquoYesrdquo only) definition ofpotential vaccine accepters is randomized mdash allowing us to estimate the causal effects ofthis normative information Here we focus on comparisons between providing the normativeinformation about vaccines before or after measuring outcomes (eg vaccine acceptance) inthe Supplementary Information (SI) we also report similar results when the control groupconsists of those who received information about other behaviors (ie about mask wearingand distancing) which can avoid concerns about differential attrition and researcher demand

On average presenting people with this normative information increases stated intentionsto take a vaccine with the broad and narrow treatments causing 0039 and 0033 increases ona five-point scale (95 confidence intervals [0028 0051] and [0021 0044] respectively)The distribution of responses across treatments (Figure 3a) reveals that the effects of the

rint n

broad (narrow) treatment are concentrated in inducing an additional 16 (11) of people tosay they will at least ldquoprobablyrdquo accept the vaccinedagger and moving 19 (17) to ldquodefinitelyrdquo(Table S8) This is a 49 relative reduction in the fraction of people choosing a response thatis ldquounsurerdquo or more negative A post hoc analysis also concluded that these effects are largestamong people who answer ldquoDonrsquot knowrdquo to the baseline vaccine acceptance question (Figure3b Table S12) consistent with the idea of targeting vaccine ldquofence-sittersrdquo36 These effectsare relatively large and are of similar overall magnitude as global trends in vaccine acceptancefrom November 2020 to January 2021 (011 increase on the five-point scale) mdash a period thatfeatured frequent and widely-distributed vaccine-related newsDagger

These effects on vaccine acceptance can be at least partially explained by changes inrespondentsrsquo beliefs about these descriptive norms We can examine this because the surveyalso measured respondentsrsquo beliefs about vaccine acceptance in their communities (as displayedin Figure 2) and we randomized whether this was measured before or after providing thenormative information As expected the normative information treatment increased thefraction of people that the respondents estimate will accept a vaccine (Figure S8) Amongthose respondents for whom we measured these normative beliefs prior to treatment wecan examine how treatment effects varied by this baseline belief In particular we classifyrespondents according to whether their baseline belief was above the broad (ldquomay takerdquo)number under the narrow (ldquowill takerdquo) number or between these two numberssect Consistentwith the hypothesis that this treatment works through revising beliefs about descriptive normsupwards we find significant effects of the normative information treatment in the groups thatmay be underestimating vaccine acceptance mdash the under and between groups (Figure 3b)though the smaller sample sizes here (since these analyses are only possible for a randomsubset of respondents) do not provide direct evidence that the effect in the under group islarger than that in the above group (p = 038 and p = 031 for broad and narrow treatmentsrespectively)para A post hoc analysis to address possible mismeasurement due to a preferenceto report round numbers (by removing those who reported they believe 0 50 or 100 ofpeople in their community would accept a vaccine) provided was likewise consistent with this

daggerNote that this statement is about effects on the cumulative distribution of the vaccine acceptance scale (the proportionanswering at least ldquoProbablyrdquo) The proportion answering exactly ldquoProbablyrdquo is similar across conditions (Figure 3a)consistent with the treatment shifting some respondents from ldquoUnsurerdquo to ldquoProbablyrdquo but also some from ldquoProbablyrdquoto ldquoDefinitelyrdquo

DaggerThis detailed vaccine acceptance question was only shown to people who had reported not having already received avaccine Therefore for this comparison we restrict to the time period before vaccines were available to the public incountries in our sample

sectThe question measuring beliefs about descriptive norms asks about ldquoyour communityrdquo while the information providedis for the country Thus for an individual respondent these need not exactly match to be consistent

paraWe had also hypothesized that the broad and narrow treatments would differ from each other in their effects onrespondents in the between group but we found no such evidence p = 087

rint n

hypothesis (Figure S11)Having fielded this experiment in 23 countries we can estimate and compare treatment effects

internationally which may be useful for both national and international communication effortsUsing a linear mixed-effects model we estimate positive effects in the majority of countries(Figure 3c) While estimates for some countries are larger (eg Pakistan Vietnam) and someare smaller (eg Nigeria United Kingdom) most countries are statistically indistinguishableFurthermore point estimates of the effect of the broad treatment are nearly uniformly positiveThus we summarise the results as providing evidence that accurate normative informationconsistently increases intentions to accept COVID-19 vaccines

Limitations and Robustness of the Conclusions

The primary drawback of the approach taken in this paper is that we are only able to measurechanges in intentions to accept a vaccine against COVID-19 which could differ from vaccineuptake While it is not possible at this stage to study interventions that measure take up ofthe COVID-19 vaccine on a representative global population we believe that the interventionstudied here is less subject to various threats to validity mdash such as experimenter demandeffects mdash that are typically a concern in survey experiments measuring intentions

This randomized experiment was embedded in a survey with a more general advertisedpurpose that covers several topics so normative information is not particularly prominent(Figure S4) In this broader survey only 15 of questions were specific to vaccinations orsocial norms32 Furthermore unlike other sampling frames with many sophisticated studyparticipants (eg country-specific survey panels Amazon Mechanical Turk) respondents arerecruited from a broader population (Facebook users) In addition we observe null resultsfor observable behaviors such as distancing and mask wearing which would be surprising ifresearcher demand effects were driving the result

A number of robustness checks increase our confidence that experimenter demand is notdriving the result As a first robustness check we compare the outcome of subjects whoreceive the vaccine norm treatment to those receiving the treatment providing informationabout masks and distancing and find the treatment effect persists (Table S9) Moreover wemay expect researcher demand effects to be smaller when the information treatment and theoutcome are not immediately adjacent In all cases for the vaccine acceptance outcome thereis always at least one intervening screen of questions (the future mask-wearing and distancingintentions questions) Furthermore they are often separated by more than this We considera subset of respondents where the treatment and the outcome are separated by at least oneldquoblockrdquo of questions between them Results of this analysis are presented in Figure S14 andTable S13 The estimated effects of the vaccine treatments in this smaller sample are less

rint n

precise but both significantly positive Moreover Table S14 shows even with the larger gapbetween treatment and outcome the information is still moving a relatively large share ofpeople who are unsure or more negative to at least probably accepting the vaccine

Discussion

Framing vaccination as a social norm has been suggested as an effective approach to buildingCOVID-19 vaccine confidence1537ndash39 but this recommendation has lacked direct evidence on ascalable messaging strategy which this international randomized experiment now contributesBrewer et al15 document the case of a vaccine campaign by a major pharmacy retail chain inthe United States that employed negative norms messaging to emphasize risks to individualsldquoGet your flu shot today because 63 of your friends didnrsquotrdquo Although such a strategy canreduce incentives to free-ride on vaccine herd immunity its broader impact on social normperceptions may render it ineffective In general the multimodal effects of descriptive norms onrisk perceptions pro-social motivations and social conformity highlight the value of evidencefrom a large-scale study as we provide here

For social norms to be effective it is critical that they are salient in the target population(eg wearing badges40) While in our randomized experiments norms are made salient throughdirect information treatments the results have implications for communication to the publicthrough health messaging campaigns and the news media For example because very highlevels of vaccine uptake are needed to reach herd immunity3 it is reasonable for news media tocover the challenges presented by vaccine hesitancy but our results suggest that it is valuableto contextualize such reporting by highlighting the widespread norm of accepting COVID-19vaccines Public health campaigns to increase acceptance of safe and effective vaccines caninclude information about descriptive norms In an effort to influence the public some publicfigures have already documented receiving a COVID-19 vaccine in videos on television andsocial media The substantial positive effects of numeric summaries of everyday peoplersquosintentions documented here suggest that simple factual information about descriptive normscan similarly leverage social influence to increased vaccine acceptance Some negative attitudestoward vaccination put disadvantaged communities at more risk so emphasizing country-widevaccination norms may prove critical for removing susceptible pools and reducing the risk ofendemic disease341

How important are the effects of the factual descriptive normative messages studied hereSmaller-scale interventions that treated individuals with misinformation42 pro-social mes-sages43 demographically tailored videos44 text message reminders45 or other informational

For vaccines the treatment effects muted somewhat as the p-values that the treatment effect is equal across this smallersample and the broader sample are 002 and 003 for the broad and narrow treatments respectively

rint n

content46 have yielded similar or smaller effect sizes while lacking the scalability and practicalappeal of accurate descriptive norms The substantial effects of normative information aboutvaccine acceptance may reflect that people have little passive exposure to information abouthow many people in their communities and countries would accept a vaccine or even havedone so already This result contrasts with other preventative behaviors (mask wearing anddistancing) for which we observe smaller or no effects (see Supplementary Information SectionS6) that are both ongoing (ie respondents have repeatedly chosen whether to performthem before) and readily observable in public However it is possible that as people havemore familiarity with social contacts choosing to accept a vaccine this type of normativeinformation will become less impactful making the use of this communication strategy evenmore important in the early stages of a vaccine roll out to the most vulnerable people ineach country More generally changes in stated intentions to accept a vaccine may not fullytranslate into actual take-up though prior studies exhibit important concordance betweenvaccination intentions and subsequent take-up47 mdash and effects of treatments on each4849Thus we encourage the use of these factual normative messages as examined here but wealso emphasize the need for a range of interventions that lower real and perceived barriers tovaccination as well as leveraging descriptive norms and social contagion more generally suchas in spreading information about how to obtain a vaccine21

Materials and Methods

Experiment analysisThe results presented in the main text and elaborated on in the supple-mentary materials each use a similar pre-registered methodology that we briefly describe hereFor the results in Figure 3a we estimate the following linear regression for each behavior k

Yik = δ0k +sumjisinJ

δjkDjik + γkXi +

sumjisinJ

ηjkXiDjik + εik [1]

where Yik is the outcome for individual i and behavior k isin K = vaccine distancing masksDj

ik is an indicator if individual i received treatment j isin J = Broad Narrow for behavior kand Xi is a vector of centered covariates5051 All statistical inference uses heteroskedasticity-consistent HuberndashWhite ldquosandwichrdquo estimates of the variancendashcovariance matrix

For heterogeneous treatment effects (Figure 3b) we estimate a similar regression focusingon the vaccine behavior

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

Mixed-effects model In the main text and Figure 3c we report results from a linear mixed-effects model with coefficients that vary by country This model is also described in ourpreregistered analysis plan Note that the coefficients for the overall (across-country) treatmentseffects in this model differ slightly from the estimates from the model in equation 3 that isthe ldquoAveragerdquo points in Figure 3b and 3c do not match exactly As noted in our analysis planldquosandwichrdquo standard errors are not readily available here so reported 95 confidence intervalsare obtained by estimating the standard errors via a bootstrap

Data and materials and availabilityDocumentation of the survey instrument and aggregateddata from the survey are publicly available at httpscovidsurveymitedu Researchers canrequest access to the microdata from Facebook and MIT at httpsdataforgoodfbcomdocspreventive-health-survey-request-for-data-access Researchers can request access to themicrodata at [redacted] Preregistration details are available at httpsosfio9sk4hview_only=aa44d784c2c34d02beeaa9e45be8317f and httpsosfiovg786view_only=62557532b8564a0caae601dfedde1b55 Analysis code for reproducing the results will be made publicThe Committee on the Use of Humans as Experimental Subjects at MIT approved both thesurvey and embedded randomized experiment as exempt protocols

References

1 Malik A A McFadden S M Elharake J amp Omer S B Determinants of COVID-19 vaccineacceptance in the US EClinicalMedicine 26 100495 (2020)

2 Sanche S et al High contagiousness and rapid spread of severe acute respiratory syndromecoronavirus 2 Emerging Infectious Diseases 26 1470ndash1477 (2020)

3 Anderson R M Vegvari C Truscott J amp Collyer B S Challenges in creating herd immunityto SARS-CoV-2 infection by mass vaccination The Lancet 396 1614ndash1616 (2020)

4 Signorelli C Iannazzo S amp Odone A The imperative of vaccination put into practice TheLancet Infectious Diseases 18 26ndash27 (2018)

5 Omer S B Betsch C amp Leask J Mandate vaccination with care Nature 469ndash472 (2019)

6 Betsch C amp Boumlhm R Detrimental effects of introducing partial compulsory vaccinationExperimental evidence The European Journal of Public Health 26 378ndash381 (2016)

7 Betsch C Boumlhm R amp Chapman G B Using behavioral insights to increase vaccination policyeffectiveness Policy Insights from the Behavioral and Brain Sciences 2 61ndash73 (2015)

8 Nyhan B Reifler J Richey S amp Freed G L Effective messages in vaccine promotion arandomized trial Pediatrics 133 e835ndashe842 (2014)

9 Nyhan B amp Reifler J Does correcting myths about the flu vaccine work An experimentalevaluation of the effects of corrective information Vaccine 33 459ndash464 (2015)

rint n

10 Green J et al Report 36 Evaluation of COVID-19 vaccine communication strategies TheCOVID States Project A 50-State COVID-19 Survey Reports (2021)

11 Nyhan B Reifler J amp Richey S The role of social networks in influenza vaccine attitudes andintentions among college students in the southeastern United States Journal of Adolescent Health51 302ndash304 (2012)

12 Brunson E K The impact of social networks on parentsrsquo vaccination decisions Pediatrics 131e1397ndashe1404 (2013)

13 Bauch C T amp Earn D J Vaccination and the theory of games Proceedings of the NationalAcademy of Sciences 101 13391ndash13394 (2004)

14 Oraby T Thampi V amp Bauch C T The influence of social norms on the dynamics of vaccinatingbehaviour for paediatric infectious diseases Proceedings of the Royal Society B Biological Sciences281 20133172 (2014)

15 Brewer N T Chapman G B Rothman A J Leask J amp Kempe A Increasing vaccinationPutting psychological science into action Psychological Science in the Public Interest 18 149ndash207(2017)

16 DeRoo S S Pudalov N J amp Fu L Y Planning for a COVID-19 vaccination program JAMA323 2458ndash2459 (2020)

17 Allcott H et al Polarization and public health Partisan differences in social distancing duringthe coronavirus pandemic Journal of Public Economics 191 104254 (2020)

18 Bridgman A et al The causes and consequences of COVID-19 misperceptions Understandingthe role of news and social media Harvard Kennedy School Misinformation Review 1 (2020)

19 Simonov A Sacher S K Dubeacute J-P H amp Biswas S The persuasive effect of Fox NewsNon-compliance with social distancing during the COVID-19 pandemic Tech Rep NationalBureau of Economic Research (2020)

20 Holtz D et al Interdependence and the cost of uncoordinated responses to COVID-19 Proceedingsof the National Academy of Sciences 117 19837ndash19843 (2020)

21 Banerjee A Chandrasekhar A G Duflo E amp Jackson M O Using gossips to spreadinformation Theory and evidence from two randomized controlled trials The Review of EconomicStudies 86 2453ndash2490 (2019)

22 Alatas V Chandrasekhar A G Mobius M Olken B A amp Paladines C When celebritiesspeak A nationwide Twitter experiment promoting vaccination in Indonesia Tech Rep 25589National Bureau of Economic Research (2019)

23 Bauch C T amp Bhattacharyya S Evolutionary game theory and social learning can determinehow vaccine scares unfold PLoS Comput Biol 8 e1002452 (2012)

24 Rao N Mobius M M amp Rosenblat T Social networks and vaccination decisions Tech RepFederal Reserve Board of Boston (2007) No 07-12

25 Vietri J T Li M Galvani A P amp Chapman G B Vaccinating to help ourselves and othersMedical Decision Making 32 447ndash458 (2012)

rint n

26 Shim E Chapman G B Townsend J P amp Galvani A P The influence of altruism oninfluenza vaccination decisions Journal of The Royal Society Interface 9 2234ndash2243 (2012)

27 Korn L Boumlhm R Meier N W amp Betsch C Vaccination as a social contract Proceedings ofthe National Academy of Sciences 117 14890ndash14899 (2020)

28 Ibuka Y Li M Vietri J Chapman G B amp Galvani A P Free-riding behavior in vaccinationdecisions An experimental study PloS one 9 e87164 (2014)

29 Boumlhm R Betsch C amp Korn L Selfish-rational non-vaccination Experimental evidence from aninteractive vaccination game Journal of Economic Behavior amp Organization 131 183ndash195 (2016)

30 Hershey J C Asch D A Thumasathit T Meszaros J amp Waters V V The roles of altruismfree riding and bandwagoning in vaccination decisions Organizational Behavior and HumanDecision Processes 59 177ndash187 (1994)

31 Sato R amp Takasaki Y Peer effects on vaccination behavior Experimental evidence from ruralNigeria Economic Development and Cultural Change 68 93ndash129 (2019)

32 Collis A et al Global Survey on COVID-19 Beliefs Behaviors and Norms Tech Rep MITSloan School of Management (2020)

33 Barkay N et al Weights and methodology brief for the COVID-19 symptom survey by Universityof Maryland and Carnegie Mellon University in partnership with Facebook arXiv preprintarXiv200914675 (2020)

34 Fisher K A et al Attitudes toward a potential SARS-CoV-2 vaccine A survey of us adultsAnnals of internal medicine 173 964ndash973 (2020)

35 Lazarus J V et al A global survey of potential acceptance of a COVID-19 vaccine NatureMedicine 1ndash4 (2020)

36 Betsch C Korn L amp Holtmann C Donrsquot try to convert the antivaccinators instead target thefence-sitters Proceedings of the National Academy of Sciences 112 E6725ndashE6726 (2015)

37 Chou W-Y S Burgdorf C E Gaysynsky A amp Hunter C M COVID-19 vaccinationcommunication Applying behavioral and social science to address vaccine hesitancy and fostervaccine confidence Tech Rep National Institutes of Health (2020)

38 Palm R Bolsen T amp Kingsland J T The effect of frames on COVID-19 vaccine hesitancymedRxiv (2021)

39 Chevallier C Hacquin A-S amp Mercier H COVID-19 vaccine hesitancy Shortening the lastmile Trends in Cognitive Sciences (2021)

40 Riphagen-Dalhuisen J et al Hospital-based cluster randomised controlled trial to assess effectsof a multi-faceted programme on influenza vaccine coverage among hospital healthcare workersand nosocomial influenza in the netherlands 2009 to 2011 Eurosurveillance 18 20512 (2013)

41 Paul E Steptoe A amp Fancourt D Attitudes towards vaccines and intention to vaccinate againstCOVID-19 Implications for public health communications The Lancet Regional Health-Europe100012 (2020)

rint n

42 Loomba S de Figueiredo A Piatek S J de Graaf K amp Larson H J Measuring the impactof COVID-19 vaccine misinformation on vaccination intent in the uk and usa Nature HumanBehaviour 1ndash12 (2021)

43 Banker S amp Park J Evaluating prosocial COVID-19 messaging frames Evidence from a fieldstudy on Facebook Judgment and Decision Making 15 1037ndash1043 (2020)

44 Alsan M et al Comparison of knowledge and information-seeking behavior after general COVID-19 public health messages and messages tailored for black and latinx communities A randomizedcontrolled trial Annals of Internal Medicine (2020)

45 Milkman K L et al A mega-study of text-based nudges encouraging patients to get vaccinatedat an upcoming doctorrsquos appointment (2021)

46 Chanel O Luchini S Massoni S amp Vergnaud J-C Impact of information on intentions tovaccinate in a potential epidemic Swine-origin Influenza A (H1N1) Social Science amp Medicine72 142ndash148 (2011)

47 Lehmann B A Ruiter R A Chapman G amp Kok G The intention to get vaccinated againstinfluenza and actual vaccination uptake of dutch healthcare personnel Vaccine 32 6986ndash6991(2014)

48 Bronchetti E T Huffman D B amp Magenheim E Attention intentions and follow-through inpreventive health behavior Field experimental evidence on flu vaccination Journal of EconomicBehavior amp Organization 116 270ndash291 (2015)

49 Alsan M amp Eichmeyer S Persuasion in medicine Messaging to increase vaccine demandWorking Paper 28593 National Bureau of Economic Research (2021) URL httpwwwnberorgpapersw28593

50 Lin W Agnostic notes on regression adjustments to experimental data Reexamining freedmanrsquoscritique Annals of Applied Statistics 7 295ndash318 (2013)

51 Miratrix L W Sekhon J S amp Yu B Adjusting treatment effect estimates by post-stratification inrandomized experiments Journal of the Royal Statistical Society Series B (Statistical Methodology)75 369ndash396 (2013)

52 De Quidt J Haushofer J amp Roth C Measuring and bounding experimenter demand AmericanEconomic Review 108 3266ndash3302 (2018)

53 Mummolo J amp Peterson E Demand effects in survey experiments An empirical assessmentAmerican Political Science Review 113 517ndash529 (2019)

rint n

Supplementary Information for

ldquoSurfacing norms to increase vaccine acceptancerdquo

Contents

S1 Experiment overview 16

S2 Data construction 16

S3 Randomization checks 19

S4 Analysis methods 26S41 Mixed-effects model 26

S5 Effects on beliefs about descriptive norms 26

S6 Effects on intentions 28S61 Heterogeneous treatment effects 28S62 Robustness checks 35

S7 Normndashintention correlations 38

S8 Validating survey data 39

rint n

S1 Experiment overview

During an update to the survey on October 28th 2020 we introduced a prompt to allrespondents that provided information about preventative behaviors in their country basedon information from the survey Although this information was provided to all respondentswho completed the survey from an eligible country the information was provided in a randomorder creating an experiment within the survey For each eligible respondent we showed thefollowing message at a random position in the latter part of the survey

Your responses to this survey are helping researchers in your region and aroundthe world understand how people are responding to COVID-19 For example weestimate from survey responses in the previous month that [[country share]] ofpeople in your country say they [[broad or narrow]] [[preventative behavior]]

We filled in the blanks with one randomly chosen preventative behavior a broad or narrowdefinition of the activity and the true share of responses for the respondentrsquos country Thethree behaviors were vaccine acceptance mask wearing and social distancing In the broadcondition we used a more inclusive definition of the preventative behavior and the narrowcondition used a more restrictive definition For example for vaccine acceptance we eitherreported the share of people responding ldquoYesrdquo or the share of people responding ldquoYesrdquo orldquoDonrsquot knowrdquo to the baseline vaccine acceptance question The numbers shown which areupdated with each wave are displayed in Figure S6

We preregistered our analysis plan which we also updated to reflect continued datacollection and our choice to eliminate the distancing information treatment in later wavesWhile we describe some of the main choices here our pregistered analysis plans can beviewed at httpsosfio9sk4hview_only=aa44d784c2c34d02beeaa9e45be8317f andhttpsosfiovg786view_only=62557532b8564a0caae601dfedde1b55 The analysis of theexperiment in the main text that is not described in the analysis plan is labeled post hoc (inparticular heterogeneity by baseline vaccine acceptance) One set of more complex analysesspeculatively described in the analysis plan (hypothesis 3 ldquomay suggest using instrumentalvariables analysesrdquo) has not yet been pursued

S2 Data construction

Our dataset is constructed from the microdata described in (author) S32 We first code eachoutcome to a 5-point numerical scale We then condition on being eligible for treatment andhaving a waves survey type (ie being in a country with continual data collection) to arrive

rint n

(a) Facebook Recruitment Message (b) Facebook Interstitial

Fig S4 Facebook Promotion and Interstitial

at the full dataset of those eligible for treatmentlowastlowast All randomization and balance checkslowastlowastRespondents in the snapshot survey may have received treatment if they self-reported being in a wave country Theirweights however will be wrong as their country will disagree with the inferred country so they are excluded

rint n

described as ldquointent-to-treatrdquo use this dataset In our preregistered analysis plan we describedhow the sample would be restricted to those who completed the survey and for whom wereceived a full survey completion weight from Facebook This removes approximately 40 ofrespondents resulting in 484239 respondents For the main analysis comparing users whoreceived the vaccine information treatment to control users (eg in Figure 3b) there are365593 respondents

As in our pre-analysis plan the following variables are used in our analysis

1 Outcomes

(a) Over the next two weeks how likely are you to wear a mask when in public[Always Almost always When convenient Rarely Never]

(b) Over the next two weeks how likely are you to maintain a distance of at least 1meter from others when in public [Always Almost always When convenientRarely Never]

(c) If a vaccine against COVID-19 infection is available in the market would you takeit [Yes definitely Probably Unsure Probably not No definitely not]

2 Mediators amp Covariates

(a) Baseline outcomes These questions are similar to the outcome questions Onlythe vaccine question always appears before the treatment in all cases the othersare in a randomized order Thus for use of the other covariates for increasingprecision mean imputation is required

bull Masks How often are you able to wear a mask or face covering when you arein public How effective is wearing a face mask for preventing the spread ofCOVID-19

bull Distancing How often are you able to stay at least 1 meter away from peoplenot in your household How important do you think physical distancing isfor slowing the spread of COVID-19

bull Vaccine If a vaccine for COVID-19 becomes available would you chooseto get vaccinated This will be coded as binary indicators for the possibleoutcomes grouping missing outcomes with ldquoDonrsquot knowrdquo

(b) Beliefs about norms These questions will be randomized to be shown beforethe treatment for some respondents and after treatment for other respondents

rint n

This will allow us to study heterogeneity in baseline beliefs as well as ensure ourrandomization does impact beliefs

bull Masks Out of 100 people in your community how many do you think do thefollowing when they go out in public Wear a mask or face covering

bull Distancing Out of 100 people in your community how many do you thinkdo the following when they go out in public Maintain a distance of at least1 meter from others

bull Vaccine Out of 100 people in your community how many do you think wouldtake a COVID-19 vaccine if it were made available

When used in analysis we require all covariates to be before both treatment and outcomeAs the survey contains randomized order for these questions this ensures that the distributionof question order is the same across treated and control groups and removes any imbalancecreated by differential attrition Missing values are imputed at their (weighted) mean

S3 Randomization checks

Table S1 presents results of a test that the treatment and control shares were equal to 50 asexpected While the final dataset does have some evidence of imbalance that could be causedby differential attrition the ldquorobustrdquo dataset (described in S62) is well balanced and thetreatment is balanced across the three behaviors information could be provided about (TableS2) According to our pre-registered analysis plan in the presence of evidence of differentialattrition we make use of additional analyses that use the information about other behaviorsas an alternative control group throughout this supplement

Table S1 Randomization Tests

p-val Treated Share Control Share

Full 0011 0501 0499Final 0081 0499 0501Robust 0176 0499 0501

The results of a test that the treated share and control shares equal 50 The first row usesintent-to-treat on the full set of eligible respondents the second row uses the final data set afterconditioning on eligibility and completing the survey and the third row uses the subset of responsesin the final dataset that have at least one block between treatment and outcome

In addition baseline covariates measured before both treatment and the outcome arebalanced across treatment and control groups (Table S3) The covariates are also balancedin the final analysis dataset (Table S4) and within treated users across the three possibletreatment behaviors (Table S5)

rint n

Vaccine Masks Distancing

Final 0215 0218 0441Robust 0210 0113 0519

The p-values of a test that each behavior was shown the expected number of times This reportsthe results of a joint test that each period share was equal to the expected For waves 9-12 eachbehavior was shown 13 of the time and for waves 12 on the vaccine treatments were shown to23 of respondents and the mask treatments were shown to 13 of respondents This table cannotinclude the full dataset intent-to-treat analysis because the behavior randomization occurred whenthe treatment was shown

0 20 40 60 80 100FrancePoland

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

NigeriaArgentinaMalaysia

ColombiaItaly

PakistanUnited Kingdom

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

NarrowCountry Belief

(a) Vaccine Treatments

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

BrazilThailandRomania

United KingdomBangladesh

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

Philippines BroadNarrowCountry Belief

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

United KingdomBrazil

IndonesiaItaly

NigeriaColombiaRomania

IndiaTurkey

MalaysiaArgentina

Philippines

BroadNarrowCountry Belief

(c) Distancing Treatments

Fig S5 Treatment Variation

For each behavior (Vaccine Masks Distancing) we plot the information provided to subjects basedon the broad and narrow definitions of compliance The treatments were updated every two weeksas new waves of data were included The points labeled ldquocountry beliefrdquo display the weightedaverage belief in a country of how many people out of 100 practice (or will accept for vaccines)each behavior

rint n

Baseline informationbull Demographicsbull Baseline vaccine acceptancebull Additional tracking questions

Treatmentbull Randomize behaviorbull Randomize whether broad or narrow

definition is used

Outcome

Additional survey blocks

Randomized order

Fig S6 Experiment Flow

Illustration of the flow of a respondent through the survey First they are presented with trackingand demographic questions They then enter a randomized portion where blocks are in randomorder This includes the treatment outcome and many of the baseline covariates included inregressions for precision Recall all covariates used in analysis are only used if they are pre-treatmentand outcome

rint n

0 10 20 30 40 50 60 70x

(a) Balance Tests Intent-to-treat

0 10 20 30 40 50 60 70x

(b) Balance Tests Final Sample

0 10 20 30 40 50 60 70x

(c) Balance Tests Vaccine vs Mask Treatments

0 10 20 30 40 50 60 70x

(d) Balance Tests Vaccine vs Dist Treatments

Fig S7 Balance Test p-Values

Ordered p-values for the balance tests described in Tables S3 S4 and S5 sorted in ascendingorder All available pre-treatment covariates are included which results in 76 tests This includesroughly 40 covariates that are not presented in the tables for brevity These are questions thatpermit multiple responses including news media sources and trust and a more detailed list ofpreventative measures taken

rint n

Table S3 Balance Tests Intent-to-treat

p-val Control Treated

age 0223 2587 (0002) 2583 (0002)gender 0401 1441 (0001) 1440 (0001)education 0468 2781 (0001) 2779 (0001)own health 0068 2410 (0002) 2414 (0002)vaccine accept 0848 1491 (0001) 1491 (0001)knowledge existing treatments 0210 0218 (0001) 0219 (0001)info exposure past week 0439 2300 (0001) 2301 (0001)info exposure more less wanted 0614 2387 (0002) 2386 (0002)know positive case 0405 1281 (0002) 1279 (0002)prevention mask 0999 3607 (0002) 3607 (0002)prevention distancing 0769 2669 (0003) 2670 (0003)prevention hand washing 0526 3299 (0002) 3297 (0002)effect mask 0641 2983 (0003) 2981 (0003)effect hand washing 0195 2996 (0003) 2991 (0003)country management 0082 1831 (0004) 1822 (0004)community management 0878 1929 (0003) 1928 (0003)community action importance 0498 3354 (0002) 3352 (0003)community action norms 0458 2737 (0003) 2734 (0003)distancing importance 0803 3112 (0003) 3111 (0003)norms dist 0221 49008 (0091) 49162 (0090)norms masks 0648 71800 (0086) 71852 (0086)norms vaccine 0837 61939 (0087) 61917 (0086)risk community 0164 2541 (0005) 2531 (0005)risk infection 0638 2165 (0005) 2168 (0005)control infection 0515 1879 (0006) 1873 (0006)infection severity 0083 1272 (0003) 1264 (0003)employed 2020 0542 0725 (0002) 0727 (0002)

Pre-treatment covariate means for all respondents who were eligible for treatment in both thetreatment and control groups along with the p-value for the test of the null that the means areequal For each covariate only responses where the covariate is not missing and occurs before bothtreatment and control are included To account for changes to the sampling frequencies thesep-values are from the coefficient on the intent-to-treat term in a regression of the covariate ontreatment period and centered interactions between treatment and period As we do not haveweights for all respondents this is an unweighted regression

rint n

Table S4 Balance Tests Final Dataset

Pre-treatment covariate means for all respondents who were eligible for treatment completed theentire survey and received a full survey completion weight in both the treatment and controlgroups along with the p-value for the test of the null that the means are equal For each covariateonly responses where the covariate is not missing and occurs before both treatment and controlare included To account for changes to the sampling frequencies these p-values are from thecoefficient on the treatment term in a regression of the covariate on treatment period and centeredinteractions between treatment and period This is a weighted regression using full completionsurvey weights

rint n

Table S5 Balance Tests Between Treatments Final Dataset

VD p-val VM p-val Vaccine Masks Dist

age 0543 0011 2716 2684 2611gender 0229 0630 1440 1442 1445education 0861 0313 2825 2826 2839own health 0278 0629 2399 2401 2386vaccine accept 0419 0000 1524 1505 1442knowledge existing treatments 0308 0926 0159 0209 0555info exposure past week 0294 0633 2375 2369 2354info exposure more less wanted 0339 0361 2420 2404 2355know positive case 0825 0894 1339 1326 1233prevention mask 0920 0391 3649 3640 3609prevention distancing 0479 0551 2723 2711 2687prevention hand washing 0500 0134 3339 3331 3326effect mask 0366 0744 2998 2991 2933effect hand washing 0897 0232 3016 3003 3013country management 0375 0143 1788 1775 1765community management 0544 0013 1912 1885 1880community action importance 0710 0751 3370 3369 3367community action norms 0503 0695 2711 2704 2679distancing importance 0882 0841 3150 3148 3149norms dist 0523 0917 49914 49721 49237norms masks 0693 0447 73143 72896 71308norms vaccine 0521 0829 62837 62512 60816risk community 0813 0331 2547 2545 2515risk infection 0460 0771 2218 2208 2179control infection 0242 0498 1880 1872 1849infection severity 0323 0554 1255 1258 1264employed 2020 0707 0152 0731 0722 0733

Pre-treatment covariate means for all respondents who were treated completed the entire surveyand received a full survey completion weight along with the p-value for the test of the null that themeans between treatment groups are equal For each covariate only responses where the covariateis not missing and occurs before both treatment and control are included To account for changesto the sampling frequencies these p-values are from the coefficient on the treatment behavior termsin a regression of the covariate on treatment behavior period and centered interactions betweentreatment behavior and period This is a weighted regression using full completion survey weights

rint n

S4 Analysis methods

The results presented in the main text and elaborated on in sections S5 S6 and S61 each usea similar pre-registered methodology that we briefly describe here For the results in sectionsS5 and S6 we estimate the following linear regression for each behavior k

δjkDjik + γkXi +

sumjisinJ

ηjkXiDjik + εik (S3)

ik is an indicator if individual i received treatment j isin J = Broad Narrow for behaviork and Xi is a vector of centered covariatesS50 S51 In the figures and tables we report theδjkrsquos and suppress coefficients on covariates and interactions All statistical inference usesheteroskedasticity-consistent HuberndashWhite ldquosandwichrdquo estimates of the variancendashcovariancematrix

In section S61 we estimate a similar regression As our analysis of heterogeneity focuseson the vaccine treatment we will suppress the behavior index k

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

S41 Mixed-effects model In the main text and Figure 3c we report results from a linearmixed-effects model with coefficients that vary by country This model is also described inour preregistered analysis plan Note that the coefficients for the overall (across-country)treatments effects in this model differ slightly from the estimates from the model that is theldquoAveragerdquo points in Figure 3b and 3c do not match exactly As noted in our analysis planldquosandwichrdquo standard errors are not readily available here so reported 95 confidence intervalsare obtained by estimating the standard errors via a bootstrap

S5 Effects on beliefs about descriptive norms

Figure S8 present evidence that the treatments do update beliefs about the descriptive normsof survey respondents The figures plot coefficients on treatment from a regression of surveynorms on treatment status including centered covariates and interactions as described in thepre-analysis plan In this analysis treated respondents are those who receive the treatmentbefore the question eliciting beliefs about norms This will not agree in general with thetreatment status for the main analysis given the randomized question order in the survey Thecovariates included in this analysis are pre-treatment and outcome relative to this treatment

rint n

definition

Figure S8a compares treatment and control respondents and figure S8b conditions on treatedindividuals and then uses individuals who received an information treatment for a differentbehavior as control The coefficients plotted in figure S8b are smaller than in figure S8a whichindicates that normative information on other behaviors may induce an update in beliefs onthe focal behavior

0 1 2 3 4Treatment effect on beliefs about norms

vaccine BroadNarrow

(a) Treatment vs Control

minus1 0 1 2 3Treatment effect on beliefs about norms

vaccine BroadNarrow

(b) Treatment vs Other Behavior Treatment

Fig S8 Effects on beliefs about descriptive norms

(1) (2) (3) (4) (5) (6)

Broad Treatment 1250lowastlowastlowast 0778 2719lowastlowastlowast 2357lowastlowastlowast 3777lowastlowastlowast 3685lowastlowastlowast

(0223) (0479) (0198) (0217) (0464) (0168)Narrow Treatment 0682lowastlowastlowast -0167 0838lowastlowastlowast 1801lowastlowastlowast 2955lowastlowastlowast 1788lowastlowastlowast

(0217) (0477) (0198) (0209) (0463) (0169)Control Other Treatment X X XBehavior masks dist vaccine masks dist vaccineNumber Controls 149458 34002 91217 229715 52724 225615Number Treated 75125 17354 130389 75125 17354 130389

Observations 224583 51356 221606 304840 70078 356004R2 0170 0145 0206 0182 0157 0210Adjusted R2 0170 0143 0206 0182 0156 0210Residual Std Error 25040 27046 24162 25445 27085 24566F Statistic 130581lowastlowastlowast 46184lowastlowastlowast 191547lowastlowastlowast 201909lowastlowastlowast 64198lowastlowastlowast 329570lowastlowastlowast

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001Table S6 Effects on beliefs about descriptive norms for primary and alternative definitions of thecontrol group

rint n

minus004 minus002 000 002 004Treatment effect on five point scale

vaccine BroadNarrow

Fig S9 Treatment effects with primary and alternative definition of the control group

S6 Effects on intentions

Figure S9 displays regression coefficients for the primary analysis where the intention topartake in the outcome behavior is regressed on treatment centered covariates and theirinteractions As discussed in the pre-analysis plan we use both the randomized timing ofinformation treatments and the randomized focal behavior of the intervention Figure S9a usesrespondents who receive the information after the outcome is measured as the control groupand Figure S9b uses individuals who receive the information treatment for a different behavioras the control group The results are largely consistent and suggest that the informationtreatment significantly increases reported vaccine acceptance while effects for distancing andmasks are smaller and not statistically distinguishable from zero

Table S8 presents results from the same analysis after transforming the outcome variableinto binary indicators This allows us to understand across which thresholds the treatmenthas induced people to cross The coefficients indicate that the treatment is inducing people toreport they will at least probably take the vaccine and definitely take the vaccine Similarregressions restricted to those who report they donrsquot know if they will take the vaccine atbaseline are presented in Table S9 Among this group there is a larger effect and it isconcentrated in moving people to say they will ldquoprobablyrdquo take the vaccine

S61 Heterogeneous treatment effectsFigure S10 plots regression coefficients for estimatesof heterogeneous treatment effects in equation 4 across different dimensions We see the positiveeffects of our treatment concentrated in those with lower baseline beliefs about norms (FigureS10a) and in those who are unsure if they will accept a vaccine (Figure S10b) Estimates arealso reported in Tables S11 and S12

rint n

(1) (2) (3) (4) (5) (6)

Broad Treatment 0002 0001 0027lowastlowastlowast 0011lowast 0004 0039lowastlowastlowast

(0006) (0014) (0007) (0006) (0014) (0006)Narrow Treatment 0011lowastlowast -0021 0021lowastlowastlowast 0021lowastlowastlowast -0017 0033lowastlowastlowast

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001

Table S7 Treatment effects with primary and alternative definition of the control group

000 002 004 006 008Treatment effect on five point scale

Between

Average BroadNarrow

(a) Baseline Belief Partition

minus002 000 002 004 006 008Treatment effect on five point scale

Dont know

Average BroadNarrow

(b) Vaccine Acceptance

Fig S10

rint n

gt No definitely not gt Probably not gt Unsure gt Probably

Intercept 0916lowastlowastlowast 0846lowastlowastlowast 0673lowastlowastlowast 0487lowastlowastlowast

(0001) (0001) (0001) (0001)Narrow Treatment 0000 0005lowastlowastlowast 0011lowastlowastlowast 0017lowastlowastlowast

(0002) (0002) (0002) (0002)Broad Treatment 0000 0005lowastlowastlowast 0016lowastlowastlowast 0019lowastlowastlowast

(0002) (0002) (0002) (0002)

Observations 365593 365593 365593 365593R2 0296 0493 0560 0459Adjusted R2 0296 0493 0560 0459Residual Std Error 0232 0257 0310 0368F Statistic 152334lowastlowastlowast 577685lowastlowastlowast 1647634lowastlowastlowast 1508106lowastlowastlowast

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001Estimates of equation 3 with binary outcomes The outcome variable for each column is an indicatorequal to one if the respondent reported a value higher than the column name For example inthe column ldquogt Probably notrdquo the outcome Yi equals one if the respondent answered ldquoUnsurerdquoldquoProbablyrdquo or ldquoYes definitelyrdquo

Table S8 Distributional treatment effects

(0001) (0002) (0003) (0002)Narrow Treatment 0003 0005 0025lowastlowastlowast 0018lowastlowastlowast

(0003) (0004) (0007) (0004)Broad Treatment 0001 0006 0049lowastlowastlowast 0015lowastlowastlowast

(0003) (0004) (0007) (0004)

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001Estimates of equation 3 with binary outcomes on sample of respondents who say they donrsquot know ifthey will take a vaccine at baseline The outcome variable for each column is an indicator equal toone if the respondent reported a value higher than the column name For example in the columnldquogt Probably notrdquo the outcome Yi equals one if the respondent answered ldquoUnsurerdquo ldquoProbablyrdquo orldquoYes definitelyrdquo

Table S9 Distributional treatment effects for ldquoDonrsquot knowrdquo respondents

rint n

(0002) (0003) (0003) (0003)Narrow Treatment 0001 0012lowastlowast 0019lowastlowastlowast 0017lowastlowastlowast

(0005) (0005) (0006) (0007)Broad Treatment -0001 0005 0019lowastlowastlowast 0020lowastlowastlowast

(0005) (0005) (0006) (0006)

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001Estimates of equation 3 with binary outcomes on sample of respondents who have a baseline beliefsabout how many people in their community will take a vaccine under the narrow treatment numberThe outcome variable for each column is an indicator equal to one if the respondent reported avalue higher than the column name For example in the column ldquogt Probably notrdquo the outcome Yi

equals one if the respondent answered ldquoUnsurerdquo ldquoProbablyrdquo or ldquoYes definitelyrdquoTable S10 Distributional treatment effects for ldquoUnderrdquo respondents

minus004 minus002 000 002 004 006 008 010Treatment effect on five point scale

Between

Average BroadNarrow

Heterogeneous treatment effects based on baseline beliefs about how many people in theircommunity will accept a vaccine We remove subjects who respond they believe 0 50 or100 percent of people in their community will accept a vaccine to mitigate measurementerror due to a bias towards round numbers

Fig S11 Mitigating Round Number Bias Baseline Belief Partition

rint n

Average Above Between Under

Broad Treatment 0039lowastlowastlowast 0023 0041lowastlowastlowast 0043lowastlowastlowast

(0006) (0017) (0015) (0016)Narrow Treatment 0033lowastlowastlowast 0026 0037lowastlowast 0049lowastlowastlowast

(0006) (0017) (0015) (0015)

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001The joint test that the broad and narrow coefficients are equal across groups has a p-value of 0807and the two-sided test that the broad (narrow) treatment effects in the Under and Above groupsare equal has a p-value of 038 (031)

Table S11 Heterogeneous Treatment Effects Baseline Beliefs

Average No Donrsquot Know Yes

Broad Treatment 0039lowastlowastlowast 0041lowastlowast 0071lowastlowastlowast 0029lowastlowastlowast

(0006) (0018) (0011) (0006)Narrow Treatment 0033lowastlowastlowast 0017 0052lowastlowastlowast 0027lowastlowastlowast

(0006) (0017) (0011) (0006)

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001The joint test that the broad and narrow coefficients are equal across groups has a p-value of 0012and the two-sided test that the broad (narrow) treatment effects in the Yes and Donrsquot know groupsare equal has a p-value of lt001 (005) The two sided test that the broad (narrow) treatmenteffects in the Donrsquot know and No groups are equal has a p-value of 015 (008)

Table S12 Heterogeneous Treatment Effects Baseline Vaccine Acceptance

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

Fig S12 Correlation of Baseline Vaccine Acceptance and Outcome (Detailed) Vaccine Acceptance

Heatmap showing relationship between baseline vaccine acceptance question (x-axis) and theoutcome vaccine acceptance question (y-axis) for the control users Each cell shows the probabilityof an outcome response conditional on the baseline response and each column sums to one

rint n

Average

(a) Full Sample High Treatment

minus0075minus0050minus0025 0000 0025 0050 0075 0100Treatment effect on five point scale

Average

(b) Full Sample Low Treatment

minus0025 0000 0025 0050 0075 0100 0125 0150Treatment effect on five point scale

Average

(c) Baseline Vaccine Unsure High Treatment

minus010 minus005 000 005 010 015 020Treatment effect on five point scale

Average

(d) Baseline Vaccine Unsure Low Treatment

Fig S13 Treatment effect by treatment number

The coefficients reported in these figures are from a regression of vaccine acceptance measuredon a five point scale on indicators of the treatment number the individual was shown (if treated)grouped into bins of width 10 percentage points We also include covariates and their interactionswith treatment as in the main analysis Figures S13a and S13b report the coefficients from theregression on the full sample of individuals of separate regressions for the high and low treatmentsFigures S13c and S13d report the coefficients from the same regression on the subset of individualswho reported Donrsquot know to the baseline vaccine acceptance question

rint n

S62 Robustness checksOne concern with survey experiments such as ours is the resultscould reflect researcher demand effects where subjects respond how they think the researcherswould want them to respond While we cannot rule this out completely we do not believethis is driving our results cfS52 S53 We may be less worried about researcher demand effectsin this survey as it has a more general advertised purpose and it covers several topics sonormative information is not particularly prominent (Figure S4) Furthermore unlike othersampling frames with many sophisticated study participants (eg Amazon Mechanical Turk)respondents are recruited from a broader population (Facebook users) In addition we observenull results for observable behaviors such as distancing and mask wearing which we wouldexpect to suffer from the same researcher demand effects as vaccine acceptance We alsocompare the outcome of subjects who receive the vaccine norm treatment to those receivingthe treatment providing information about masks and distancing and find the treatmenteffect persists Moreover we may expect researcher demand effects to be smaller when theinformation treatment and the outcome are not immediately adjacent In all cases for thevaccine acceptance outcome there is always at least one intervening screen of questions(the future mask-wearing and distancing intentions questions) Furthermore they are oftenseparated by more than this We consider a subset of respondents where the treatment andthe outcome are separated by at least one ldquoblockrdquo of questions between them Results of thisanalysis are presented in Figure S14 and Table S13 The treatment effect estimates on thissmaller sample are less precise but both positive For vaccines the treatment effects mutedsomewhat as the p-values that the treatment effect is equal across this smaller sample and thebroader sample are 002 and 003 for the broad and narrow treatments respectively MoreoverTable S14 shows even with the larger gap between treatment and outcome the informationis still moving a relatively large share of people who are unsure or more negative to at leastprobably accepting the vaccine

Figure S15 plots the distribution of the number of screens between treated and control InFigure S15a we plot the distribution for the entire sample and in Figure S15b we plot thedistribution for the subset of those with at least one block between treatment and controlFor this group there are at least three pages between the treatment and outcome

rint n

vaccine BroadNarrow

minus002 000 002 004 006Treatment effect on five point scale

vaccine BroadNarrow

Fig S14 Robustness to Researcher Demand Effects

(1) (2) (3) (4) (5) (6)

Broad Treatment -0001 0025 0022lowastlowastlowast 0010 0016 0030lowastlowastlowast

(0008) (0018) (0008) (0008) (0017) (0007)Narrow Treatment 0006 0005 0017lowastlowast 0017lowastlowast -0004 0024lowastlowastlowast

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001Estimates of equation 3 on the restricted sample when outcome and treatment are separated by atleast one additional block of questions

Table S13 Robustness to Greater Separation of Treatment and Outcome

rint n

(0001) (0001) (0001) (0002)Narrow Treatment -0000 0002 0007lowastlowast 0015lowastlowastlowast

(0002) (0002) (0003) (0003)

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001Estimates of equation 3 on the restricted sample when outcome and treatment are separated by atleast one additional block of questions The outcome variable in this analysis are binary indicatorsif the outcome was at least a certain response as in table S8

Table S14 Robustness to Greater Separation of Treatment and Outcome Distributional TreatmentEffects

minus15 minus10 minus5 0 5 10 15Treatment Screen Number - Outcome Screen Number

(a) Number of Screens Between Treatment and Out-come

(b) Number of Screens Between Treatment and Out-come in Robustness Check Sample

Fig S15 Separation of Treatment and Outcome

(a) Histogram of the number of screens between treatment and outcome Negative numbersrepresent treated respondents and positive numbers are control respondents The distribution is notsmooth as the randomized order is at the block level and blocks have varying number of screens(pages) within them (b) The same histogram but for the set of respondents with at least oneblock between treatment and outcome

rint n

S7 Normndashintention correlations

In the main text Figure 1 (inset) shows the association between beliefs about descriptivenorms and intentions to accept a COVID-19 vaccine Figure S16 disaggregates this informationby country As in the main text this is a purely observational association but is computed onthe main experimental sample (ie starting in late October)

United Kingdom United States Vietnam

Philippines Poland Romania Thailand Turkey

Japan Malaysia Mexico Nigeria Pakistan

France Germany India Indonesia Italy

Argentina Bangladesh Brazil Colombia Egypt

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

Beliefs about descrptive norms

Fig S16 People who believe a larger fraction of their community will accept a vaccine are on averagemore likely to say they will accept a vaccine and this is true within the 23 included countries The verticalaxis shows the percentage of respondents who replied Yes (green) Donrsquot know (gray) and No (brown) towhether they will accept a COVID-19 vaccine

rint n

S8 Validating survey data

At the time of this writing vaccinations are not available to the vast majority of people in thesample and it is difficult to compare intentions with actual take-up due to supply constraintsIn an attempt to quantity the link between survey responses and behavior we run an auxiliaryanalysis comparing self-reported COVID-19 from the survey with country level uptakedaggerdagger

The estimated vaccinated share of adults from the survey measure is highly predictive of thecross-country variation in actual vaccination shares explaining over 80 of the cross-sectionalvariation (Table S15)

Intercept -1565(1379)

Survey Share Vacc 0794lowastlowastlowast

(0158)

Observations 22R2 0829Adjusted R2 0820Residual Std Error 4488F Statistic 25183lowastlowastlowast

lowastplt01 lowastlowastplt005 lowastlowastlowastplt001Regression of true vaccination share on estimated share vaccinated based on the survey measureThere are only 22 observations rather than 23 because data on the true share vaccinated areunavailable in Egypt Adjustment for attenuation bias due to measurement error in the surveyresults in nearly identical results

Table S15 Survey Data Predictive of Cross-Sectional Vaccine Take-up

daggerdaggerCOVID-19 vaccine data retrieved from httpsgithubcomowidcovid-19-datatreemasterpublicdatavaccinations

rint n

Experiment overview

Data construction

Randomization checks

Analysis methods

Mixed-effects model

Effects on beliefs about descriptive norms

Effects on intentions

Heterogeneous treatment effects

Robustness checks

Normndashintention correlations

Validating survey data

Introduction

Nonpharmaceutical interventions in response to epidemics such as the COVID-19 pandemicoften depend on the behavioral responses of the public for their effectiveness Even with theavailability of vaccines success depends on peoplersquos choices to accept or even seek out thevaccine1 since even low vaccine refusal rates can prevent achieving herd immunity23 Given thevalue of individual autonomy and the significant challenges of imposing vaccine mandates4ndash6it is important to understand how public health messaging can increase acceptance of safeand effective COVID-19 vaccines Many messaging strategies address individual barriers tovaccination such as complacency and inconvenience7 as well as perceived risk of both vaccinesand the disease1 However these strategies may have important limitations for examplefield studies show that corrective information about vaccine safety can effectively reducemisconceptions and false beliefs though they are not as effective in changing vaccine-relatedintentions89 Messaging strategies that share recommendations from experts and emphasizereasons for accepting a vaccine have shown promising effects on increasing acceptance in theUnited States10

It may be important to look beyond individuals to consider how public health messagingcan also leverage the significant roles of social networks (broadly defined) in shaping individualvaccination decisions11ndash15 Rather than being a small factor there is growing evidence thatpeoplersquos preventative health behaviors are dramatically influenced by many social and culturalfactors with implications for COVID-1916 In the United States for example analyses ofmobility data during the COVID-19 pandemic revealed that peoplersquos mobility behaviors varywith their partisan affiliation17 and media consumption1819 and are affected by the behaviorsof their social connections20

Acceptance of COVID-19 vaccines likely involves substantial social influence but theoryis not entirely clear on whether learning that othersrsquo are accepting a vaccine will increaseor decrease acceptance Positive peer effects can arise due to information diffusion2122conformity and injunctive norms14 inferring vaccine safety and effectiveness from othersrsquochoices2324 or pro-social motivations such as altruism2526 and reciprocity27 On the otherhand negative effects of othersrsquo acceptance can arise as a result of free-riding on vaccine-generated herd immunity even if only partial or local2829 The empirical evidence on whenpositive peer effects243031 or free-riding28 may dominate is inconclusive Furthermore theeffects of incorporating truthful information about othersrsquo into messaging strategies will dependon what that information is ie how prevalent is vaccine acceptance in a given referencegroup Thus we need further empirical guidance about scalable and effective messagingstrategies leveraging social influence That is while some interpretations of the theoretical

rint n

stan Ita

0 10 20 30 40 50 60 70 80 90 100

rint n

FrancePhilippines

EgyptTurkeyPoland

RomaniaJapan

IndonesiaNigeria

IndiaThailand

ItalyMexico

BrazilBangladesh

0 25 50 75 100

rint n

Nigeria

United Kingdom

Germany

Brazil

Thailand

Romania

Colombia

Bangladesh

France

United States

Turkey

Poland

Malaysia

Argentina

Mexico

Philippines

Indonesia

Vietnam

Pakistan

Average

BroadNarrow

Narrow

Norms treatment

Between

Dont know

Average

Nodefinitely not

rint n

Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



stan Ita

0 10 20 30 40 50 60 70 80 90 100

rint n

FrancePhilippines

EgyptTurkeyPoland

RomaniaJapan

IndonesiaNigeria

IndiaThailand

ItalyMexico

BrazilBangladesh

0 25 50 75 100

rint n

Nigeria

United Kingdom

Germany

Brazil

Thailand

Romania

Colombia

Bangladesh

France

United States

Turkey

Poland

Malaysia

Argentina

Mexico

Philippines

Indonesia

Vietnam

Pakistan

Average

BroadNarrow

Narrow

Norms treatment

Between

Dont know

Average

Nodefinitely not

rint n

Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



FrancePhilippines

EgyptTurkeyPoland

RomaniaJapan

IndonesiaNigeria

IndiaThailand

ItalyMexico

BrazilBangladesh

0 25 50 75 100

rint n

Nigeria

United Kingdom

Germany

Brazil

Thailand

Romania

Colombia

Bangladesh

France

United States

Turkey

Poland

Malaysia

Argentina

Mexico

Philippines

Indonesia

Vietnam

Pakistan

Average

BroadNarrow

Narrow

Norms treatment

Between

Dont know

Average

Nodefinitely not

rint n

Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

Nigeria

United Kingdom

Germany

Brazil

Thailand

Romania

Colombia

Bangladesh

France

United States

Turkey

Poland

Malaysia

Argentina

Mexico

Philippines

Indonesia

Vietnam

Pakistan

Average

BroadNarrow

Narrow

Norms treatment

Between

Dont know

Average

Nodefinitely not

rint n

Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



Nigeria

United Kingdom

Germany

Brazil

Thailand

Romania

Colombia

Bangladesh

France

United States

Turkey

Poland

Malaysia

Argentina

Mexico

Philippines

Indonesia

Vietnam

Pakistan

Average

BroadNarrow

Narrow

Norms treatment

Between

Dont know

Average

Nodefinitely not

rint n

Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



Discussion

rint n

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

rint n

References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



References

rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



Contents

rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



1 Outcomes

rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



Final 0215 0218 0441Robust 0210 0113 0519

RomaniaPhilippines

EgyptUnited States

JapanTurkey

IndonesiaGermany

ColombiaItaly

BrazilBangladesh

MexicoThailand

IndiaVietnam Broad

0 20 40 60 80 100Egypt

VietnamGermany

NigeriaPakistan

PolandIndonesia

IndiaFrance

United StatesItaly

TurkeyMalaysia

JapanMexico

ArgentinaColombia

(b) Mask Treatments

0 20 40 60 80 100Poland

VietnamPakistan

JapanEgypt

GermanyBangladesh

ThailandFrance

United StatesMexico

IndonesiaItaly

IndiaTurkey

MalaysiaArgentina

Philippines

rint n

definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



definition is used

Outcome

Randomized order

rint n

0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



0 10 20 30 40 50 60 70x

rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



S4 Analysis methods

δjkDjik + γkXi +

sumjisinJ

Yi =sumbisinB

1[bi = b]

δb0 +

sumjisinJ

bij + γkXi +

sumjisinJ

ηbjXiD

+ εik (S4)

rint n

definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



definition

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



vaccine BroadNarrow

rint n

(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



(1) (2) (3) (4) (5) (6)

Between

Average BroadNarrow

Dont know

Average BroadNarrow

Fig S10

rint n

(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



(0002) (0002) (0002) (0002)

(0003) (0004) (0007) (0004)

rint n

(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



(0005) (0005) (0006) (0006)

Between

Average BroadNarrow

rint n

(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



(0006) (0017) (0015) (0015)

(0006) (0017) (0011) (0006)

rint n

No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



No Dont know Yes

No definitely not

Probably not

Unsure

Probably

Yes definitely

038 0021 0013

034 0062 00087

019 062 0032

0071 026 021

0026 0042 074 01

rint n

Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



Average

rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



rint n

vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



vaccine BroadNarrow

(1) (2) (3) (4) (5) (6)

rint n

(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



(0002) (0002) (0003) (0003)

rint n

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

20 40 60 80 100 20 40 60 80 100

rint n

(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



(0158)

rint n

Experiment overview

Data construction

Analysis methods

Mixed-effects model




Robustness checks



Documents

Surfacing norms to increase vaccine acceptance