WWW.OPENABM.ORG 1

Changing the rules of the game: experiments with humans and virtual agents

Marco JanssenSchool of Human Evolution and Social Change,

School of Computing and Informatics,Center for the Study of Institutional Diversity

In cooperation with:ASU: Allen Lee, Deepali Bhagvat, Marty Anderies, Sanket Joshi, Daniel Merritt, Clint Bushman, Marcel Hurtado, Takao Sasaki, Priyanka Vanjari, Christine HendricksIndiana University: Elinor Ostrom, Robert Goldstone, Fil Menczer, Yajing Wang, Muzaffer Ozakca, Michael Schoon, Tun Myint, David Schwab, Pamela Jagger, Frank van Laerhoven, Rachel Vilensky Thailand: Francois Bousquet, Kobchai Worrapimphong, Chutapa Khunsuk, Sonthaya Jumparnin, Pongchai Dumrongrojwatthana Colombia: Juan-Camilo Cardenas, Daniel Castillo, Jorge Maldonado, Rocio Moreno, Silene Gómez, Maria Quintero, Rocio Polania, Sandra Polania, Adriana Vasquez, Carmen Candelo, Olga Nieto, Ana Roldan, Diana Maya

WWW.OPENABM.ORG 2

The commons dilemma

• Dilemma between individual and group interests– Group interest: cooperation

– Individual interest: free riding on efforts of others

• Public goods and common pool resources• Expectation with rational selfish agents

– No public goods

– Overharvesting of common pool resources

• But, many empirical examples of self-governance

WWW.OPENABM.ORG 3

• Repeated interactions• Face-to-Face communication• Information on past actions on participant• Monitoring and sanctioning by subjects themselves• Diversity in motivation: Not all humans are selfish

and rational

What contributes to cooperation in commons dilemmas?

(based on research with artificial agents and humans)

But problem is not binary: cooperate or defect. Important is defining the rules of the games and enforcing them.

WWW.OPENABM.ORG 4

Grammar of Rules

• Rules are defined as shared understanding about enforced prescriptions, concerning what actions (or outcomes) are required, prohibited, or permitted (Ostrom, 2005).

• Rules in use vs rules on paper• Formal rules vs informal rules (formal rules have

explicit consequences defined for when the rules are broken (sanctions) and can be enforced by a third party)

WWW.OPENABM.ORG 5

Puzzles

• In what way do users of a common resource change the rules?

• What makes communication effective?

• How do this relate to experience?

• And to ecological dynamics?

WWW.OPENABM.ORG 6

Combining experiments and agent-based models

• Traditionally agent-based models on cooperation very abstract

• Experiments in lab and field challenge simplistic models of behavior

• Micro-level data to test models

• Going back and forth between experiments and modeling may stimulate theory development

WWW.OPENABM.ORG 7

Common research questions

Laboratory experiments

models

Field experiments

models “role games”

Statistical analysisSurveysInterviews

Artificial worlds

Statistical analysis, SurveysText analysis, ..

WWW.OPENABM.ORG 8

Field experiments

• 3 types of games in 3 types of villages in Thailand and Colombia

• Pencil and paper experiments• First 10 rounds: open access• Voting round: 3 types of rules: lottery, rotation,

private property• Survey on rule options• Second set of 10 rounds with chosen rule• Survey• In depth interviews with a few villagers

WWW.OPENABM.ORG 9

Field experiments (2)

• Fishery game: – where to fish (A,B) – how much effort

• Irrigation game (different position; upstream):

– How much investment in public good (water)– What amount to take from (remaining) water

• Forestry game:– How much harvest

WWW.OPENABM.ORG 10

Fishery village (Baru)

Water irrigation village (Lenguazaque)

Logging village (Salahonda)

WWW.OPENABM.ORG 11 Phetchaburi river Forest village Irrigation village Fishery village

WWW.OPENABM.ORG 12

Rule choice

0

5

10

15

20

25

lotery (C) rotation(C)

propertyrights (C)

lotery (T) rotation(T)

propertyrights (T)

nu

mb

er

of g

rou

ps

irrigation

forestry

fishery

WWW.OPENABM.ORG 13

Forestry game

WWW.OPENABM.ORG 14

Fishery game

WWW.OPENABM.ORG 15

Irrigation game

WWW.OPENABM.ORG 16

WWW.OPENABM.ORG 17

Laboratory experiments

• Various spatially explicit real-time virtual environments for small groups.

• Various rounds

• Treatments include different options of rule choice and/or participants chat on informal rules

WWW.OPENABM.ORG 18

Experiments from Spring 2007• Renewable resource, density dependent regrowth• Resource is 28x28 cells• 4 participants• Duration round 4 minutes• First round is individual round (14x14 cells)• Text chat between the rounds• Option to reduce tokens of others at the end of each round (at

a cost)• Explicit and implicit mode• Different resource growth experiments:

• Low growth (6 groups)• High growth (4 groups)• High / Low growth (6 groups)• Mixed growth (6 groups)

WWW.OPENABM.ORG 19

Tokens in the resource during the rounds

0

50

100

150

200

250

300

350

400

450

500

0 30 60 90 120 150 180 210 240

Round 1

Round 2

Round 3

Round 4

Round 5

0

50

100

150

200

250

300

350

400

450

500

0 30 60 90 120 150 180 210

Round 1

Round 2

Round 3

Round 4

Round 5

`

0

50

100

150

200

250

300

350

400

450

500

0 30 60 90 120 150 180 210 240

Round 1

Round 2

Round 3

Round 4

Round 5

0

50

100

150

200

250

300

350

400

450

500

0 30 60 90 120 150 180 210 240

Round 1Round 2Round 3Round 4Round 5

Low

Mixed

High

High-Low

WWW.OPENABM.ORG 20

Average number of tokens collected (blue) and left over (red) for the 5 rounds

0

50

100

150

200

250

300

1 2 3 4 5

0

50

100

150

200

250

300

1 2 3 4 5

0

50

100

150

200

250

300

1 2 3 4 5

0

50

100

150

200

250

300

1 2 3 4 5

H L

HL Mix

WWW.OPENABM.ORG 21

Text analysis

• Coding the text: kind of rules, making sure people understand agreement, off-topic chat, meaning of experiment, etc.

• Is there a relation between the type of conversation and the performance of the group?

• We would expect that groups who are more explicit on the rules and make clear people understand it do better.

• In some groups there is a clear dominance of one person, how does this affect the outcome?

WWW.OPENABM.ORG 22

Initial results

0

0.05

0.1

0.15

0.2

0.25

past

roun

ds

sanc

tionin

g

gene

ral s

trate

gy

time

strat

egy

spac

e str

ateg

y

affir

mat

ion

expe

rimen

tch

at

off t

opic

3

4

5

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

past

roun

ds

sanc

tionin

g

gene

ral s

trate

gy

time

strat

egy

spac

e str

ateg

y

affir

mat

ion

expe

rimen

tch

at

off t

opic

H

M

L

WWW.OPENABM.ORG 23

Models of Rule changes

• Laboratory experiments will give us basic empirical information to develop agent-based model.

• ABM will be used to explore rule evolution is agents adjust rules

WWW.OPENABM.ORG 24

Reasons for making a model of the experimental data

• Testing alternative assumptions of behavior ( compare model with naïve models)

• Methodological challenge: What do we mean with calibrating an agent-based model?

• Future option: experiments with artificial agents and humans

• Using the “informed” agent-based model for exploring theoretical questions in an artificial world.

WWW.OPENABM.ORG 25

Model outline

• Timestep: 1 second.• Actions: move and harvest (explicit mode)• Each agent has a basic default speed (moves per second), and

number of moves can vary a little bit between seconds.• Define direction (target):

– the more nearby a token is to the agent, the more valuable– the more nearby a token is to the current target, the more valuable– the more other agents nearby a token, the less valuable– tokens who are straight ahead in the current path of direction of the

agent are more valuable.

• Harvest (expl mode); probabilistic choice depending on number of tokens nearby

0

5

10

15

20

25

30

0 5 10 15 20 25 30

1

2

3

4

WWW.OPENABM.ORG 26

Testing the model

• Calibration on multiple metrics using genetic algorithms

• Comparing calibrated model with naïve models (random movement; greedy agents, no heterogeneity)

• Turing tests

WWW.OPENABM.ORG 27

Towards a theoretical model of the evolution of rules

• Artificial world where agents play many rounds and adjust the rules of the game.

• What kind of rule sets will evolve? Are there attractors of rule sets?

• How is this dependent on the ecological dynamics?

• How is this dependent on the rule to change the rules (constitution)?

WWW.OPENABM.ORG 28

Coding rules

• Grammar of Institutions (Crawford and Ostrom, 1995)

• Rules is build up from 5 components:– Attributes (characteristics of the agents)– Deontic: may/must/must not– Aim: action of the agent– Conditions: when, where and how– Or else: sanctions when not following a rule

WWW.OPENABM.ORG 29

Process of constructing a rule from the libraries

IF “other agent” in “my area” it MUST NOT “collect tokens” ELSE “penalty”

WWW.OPENABM.ORG 30

Rule space based on experiments(Not yet in building blocks)

• Explicit mode required of not

• Start time harvesting

• Time left before “going crazy”

• Spatial allocation (none, corners, horizontal, vertical)

• Speed limit

WWW.OPENABM.ORG 31

Including monitoring and sanctioning

• Monitoring:– None– One monitor who cannot harvest are receives a

quarter of the income– Everybody monitors, and sanctioning is costly– Monitoring rotates every x seconds (when

monitoring one cannot harvest)

WWW.OPENABM.ORG 32

Tinkering the rules

• After every round agents update their preferences for rules (reinforcement learning), propose which rule set for next round, after which one of the proposed rule sets is chosen and implemented.

WWW.OPENABM.ORG 33

Agents breaking rules

• Agents can break rules. If an action is not allowed, it might break a rule with a probability related to the opportunities available (amount of tokens available nearby)

WWW.OPENABM.ORG 34

WWW.OPENABM.ORG 35

Distribution of total earnings(100 evolutions of 100 rounds)

0

2

4

6

8

10

12

14

16

18

20

0 100 200 300 400

bin

fre

qu

en

cy

one

everybody

WWW.OPENABM.ORG 36

Initial experiments• Multiple (100) runs with 100 rounds with agents who conditionally

cheat. Best solution:

Low growth (one) Low growth (everybody)

Speed limit 7.5 5

Mode Expl Not expl

Boundaries Vertical Vertical

Start-time 90 110

Time to go crazy 210 140

Earnings (tokens) 337 409

WWW.OPENABM.ORG 37

From ABM back to experiments

• Further analysis may provide us expectations of outcomes for experiments with human participants. Additional experiments can be done to test those.

WWW.OPENABM.ORG 38

Areas to explore in model analysis

• Do clusters of rules evolve? And do these clusters change with different tendencies of agents breaking the rules.

• Co-evolution of cheating behavior and rules (incl. monitoring/sanctioning)

• What are path-dependent trajectories?• What if growth rates change between rounds? How

will this affect the evolved rule sets?• How will differences in constitutional rules will

affect the ability to derive high performance.

WWW.OPENABM.ORG 39

Concluding remarks

• Combining agent-based models with experiments in the field and the lab. The aim is not to make predictive models, but theoretical models grounded in empirical observations.

• Challenges:– Calibration of agent-based models (multiple

metrics)– Modeling communication– Large scale controlled experiments with humans

WWW.OPENABM.ORG 40

Questions?