Open Government Communities: Does Design Affect Participation?

OPEN GOVERNMENT COMMUNITIES

DOES DESIGN AFFECT PARTICIPATION?

!1

ABOUT ME

M.Sc. Political Science, University of Copenhagen (2009)

!

Partner & Chief Strategy Officer in Better Collective

- Start-up (est. 2004) focusing on online communities in

the igaming industry.

- Industrial PhD focuses on increasing participation to

online communities in open government.

!2

PRESENTATION AGENDA

- PART I: WHY STUDY OPEN GOVERNMENT?

- PART II: RESEARCH QUESTION

- PART III: K10: AN OPEN GOVERNMENT COMMUNITY

- PART IV: EXPERIMENTS, DESIGN & FINDINGS

- PART V: LEAN EXPERIMENTATION

!3

PART I WHY STUDY OPEN

GOVERNMENT?

!4

WHAT IS IT? OPEN GOVERNMENT DEFINED

Open government is a global movement to make government more open and increase participation & collaboration through the means of technology.

!

(Obama 2009; Lathrop & Ruma 2010b; Open Government Partnership 2013)

!5

OPEN GOVERNMENT CAN BE MANY THINGS

Open data

Public service delivery

Transparency

Accountability

Innovation

eRulemaking

Citizen participation

Anti-corruption

Access

!6

EXPECTED BENEFITS OF CITIZEN PARTICIPATION

- Improve democracy (Harrison et al. 2011)

- Better decisions (Janssen 2011)

- Increased efficiency (Lakhani et al. 2010)

- Foster Innovation (Lakhani & Panetta 2007)

!7

Growing, global phenomenon: Over 60 countries have signed the Open Government Partnership since 2011

!8

WE ARE INVESTING A LOT IN OPEN GOVERNMENT

The Danish government is currently implementing 33 different open government initiatives.

!9

THERE IS A PROBLEM: PARTICIPATION

- Citizen participation initiatives will only succeed if

the citizens actively participate.

- But only 25 % of online communities gather more

than 1,000 users in their lifetime (Farzan et al.

2011).

- 1,000 users is hardly enough to fulfill the aims and

ambitions of open government.!10

kbh.dk: Social network for Copenhagen Citizens (2008).

Price: DKK 7 million.

Aim: 125,000 users. Only got 1,800 users (1,44% of the goal).!11

Worldclimatecommunity.org (2011).

Price: DKK 5 million.

Was quickly closed due to lack of participation.!12

Borger.dk e-democracy forum.

Closed due to lack of participation.!13

- Open government offers great potential and many promises.

!

- But efforts often fail to meet expectations due to lack of participation.

THE STATE OF OPEN GOVERNMENT

!14

EXISTING RESEARCH CAN HELP WITH THE PROBLEM

-Community design affects participation. (Wasko, Faraj and Teigland 2004; Ma and Agarwal 2007; Venters and Wood 2007; Zhang and Watts 2008; Moon and Sproull 2008; Shen and Khalifa 2009).

-Social psychology theory can be used to generate ideas and hypotheses about community design. (Beenen et al. 2005; Cosley et al. 2006; Chen et al. 2010; Burke, Marlow, and Lento 2009; Kraut and Resnick 2012).

!15

PART II RESEARCH QUESTION

!16

RESEARCH QUESTION

“Can social psychology theory help increase

participation in open government communities?”

!17

RESEARCH COMMUNITIES

!18

MAIN CONTRIBUTIONS

!19

-Substantial knowledge about increasing participation to open government communities through social psychology inspired design. !

-Lean Experimentation Framework: Make it cheaper and faster to test design ideas in the future.

PART III K10: AN OPEN GOVERNMENT

COMMUNITY

!20

K10 is an open government community for people involved with “early retirement pension” and “flexjob”.

!21

ABOUT K10

!22

- Purpose: “K10 is created with the aim of sick people, who are trapped in the system during their case handling with the municipalities, have a place where they can get advice and support from other people who are in the same situation. This support can be of statutory or moral nature…”

!

- Registered users: 9,500

-Contributing users: 4,000

PART IV EXPERIMENTS, RESEARCH

DESIGN & FINDINGS

!23

RESEARCH APPROACH

!24

ExperimentsExperiments are used to test different design patterns

Experiment Experiment 1 Experiment 2 Experiment 3 Experiment 4

TheorySocial compa-rison theory

Goal setting theory

Self-efficacy theory

Social identity theory

Proxy Email Survey Email & MessageGoogle

advertisements

EffectMeasure effects on participation

Measure effects on participation

Measure effects on participation

Measure effects on participation

!25

EXPERIMENT 1Research Question: How does exposure to social comparison information affect users’

participation in K10?

GROUP 1 (n = 107)

GROUP 2 (n = 111)

GROUP 3 (n = 1,265)

T0 Measure participation

Inter-vention

Receives an email with

comparative

participation

information

Receives an email with

non-comparative

participation

information

Receives no email

T1 Measure participation

!26

EXPERIMENT 1 FINDINGS

MAIN FINDINGS

- Below-means users participate significantly more.

- No significant difference between comparative and non-

comparative information is detected.

P-VALUES - Ranging from p < 0.026 to p < 0.269

EFFECT SIZE - Medium-large

IMPLICATIONS FOR DESIGN

- Participatory information can alter participation patterns.

- Participatory information might upset users.

- Can be implemented through stats, leader boards or emails.

VALIDITY CONCERNS

- Sending emails per se might have an effect.

- Extrapolation unknown

- Dependent variable might not be most salient comparison

metric.

!27

EXPERIMENT 2Research Question: How does a self-assigned goal to a number of contributions to K10

affect subsequent participation?

GROUP 1 (n = 51)

GROUP 2 (n = 33)

GROUP 3 (n = 3,998)

T0 Measure participation

Inter-vention

Subjects are asked to

set a participation goal

in a survey.

Subjects are answering

a survey without

participation goals.

Subjects are not

answering a survey.

T1 Measure participation

!28

EXPERIMENT 2 FINDINGS

MAIN FINDINGS- Goal assignment to high goals might increase participation.

- Only significant effect for subjects committing to a higher goal.

P-VALUES - Ranging from p < 0.003* to p < 0.057*

EFFECT SIZE - Large

IMPLICATIONS FOR DESIGN

- Simply asking users how much they will contribute might

increase participation.

- Goal-setting can be implemented in several ways: in sign-up

process, in surveys and through direct outreach.

VALIDITY CONCERNS

- * Significant change in control group, indicating potential

spurious variable - no extrapolation should be made.

- Extrapolation unknown

!29

EXPERIMENT 3Research Question: How does knowledge of other users’ gratitude for previous

contributions affect future contributions?

GROUP 1 (n = 23)

GROUP 2 (n = 23)

T0

Intervention

Receives an email and message at

K10 containing a thank you note.Receives no email or message.

Measure participation

T1

InterventionReceives no email or message.

Receives an email and message at

K10 containing a thank you note.

Measure participation!30

EXPERIMENT 3 FINDINGS

MAIN FINDINGS

- Thanking subjects increase subsequent participation

- As a medium, internal message system is more effective than

email

P-VALUE - P < 0.040

EFFECT SIZE - Medium

IMPLICATIONS FOR DESIGN

- Expressing gratitude can be used to attain desired behavior.

- Recognition can be implemented in different ways including

automated recognition, peer-to-peer recognition and expert

recognition.

VALIDITY CONCERNS

- Potential carryover effect. Compromises effect size, not logic

- Non-random sampling makes it challenging to extrapolate

- Possibly unclear who is thanking the subjects

-Extrapolation unknown.

!31

EXPERIMENT 4Research Question: How does benefit of contribution affect subsequent behavior on

K10?

GROUP 1 (n = 185)

GROUP 2 (n = 86)

GROUP 3 (n = 70)

GROUP 4 (n = 112)

Inter-vention

Subjects see an

advertisement

highlighting

benefit to self by

joining K10.

Subjects see an

advertisement

highlighting

benefit to similar

others (small

group) by joining

K10.

Subjects see an

advertisement

highlighting

benefit to others

(large group) by

joining K10.

Subjects see an

advertisement

highlighting no

benefit by joining

K10.

T1 Measure participation!32

EXPERIMENT 4 FINDINGSMAIN FINDINGS

Identity constructs do not affect subjects’ participation. But it

does affect engagement.

P-VALUES - Ranging from p < 0.000 to p < 0.906

EFFECT SIZE - Small

IMPLICATIONS FOR DESIGN

- First impressions matter - it affects the subjects’ subsequent

behavior or attracts certain types of subjects.

- Advertisement platforms are useful for recruiting and

assigning subjects

VALIDITY CONCERNS

-The two-week time span might have been too short.

-The different advertisements might attract different people to

begin with.

- Poor group constructs.

!33

OVERALL FINDINGS

INSIGHTS ACROSS EXPERIMENTS

- Design can influence participation behavior.

- Effects not unambiguous as expected.

-Participation problem can be addressed.

-SHARED

METHODOLOGICAL CHARACTERISTICS

- True experiments

- Theory-based

- Conducted in the field

- Use Proxies to test design ideas

KEY CHALLENGES ACROSS

EXPERIMENTS

- External validity: Unknown to which degree findings apply to

other open government communities.

- Between different demographic groups in Denmark

- Even more so between different cultures such as East / West

-Temporal validity: Unknown how findings apply in 1, 5 or 10 years.

!34

PART V LEAN EXPERIMENTATION: AN

ANSWER TO THE VALIDITY CONCERNS

!35

LEAN EXPERIMENTATION

!36

LEAN EXPERIMENTATION & BETTER COLLECTIVE

- Conduct experiments early on before investing in products / features / changes.

-Using proxies and website to conduct tests on.

- Many tests fail to improve status quo.

!37

SUMMARY

-Open government has a potential to improve democracy, decisions, efficiency and innovation.

-Success requires sustained participation.

-Social psychology can help inspire design.

-Experimental methodology can help identify successful design patterns.

-Much more experimentation is necessary.

-Lean Experimentation might help on scalability.!38