Icec11slides uno

A Multi-AgentPrediction Marketbased on Partially

ObservableStochastic Game

Janyl Jumadinova,Raj Dasgupta

Outline

Introduction

Research Problem

POSGI

Trading Agents’Strategy

ExperimentalResults

Future Work

A Multi-Agent Prediction Market based onPartially Observable Stochastic Game

Janyl Jumadinova, Raj Dasgupta

C-MANTIC Research GroupComputer Science Department

University of Nebraska at Omaha, USA

ICEC 2011

1 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Outline

Problem: Traders’ behavior in a prediction market andits impact on the prediction market’s behavior

Solution: A multi-agent system that formalizes thestrategic behavior and decision making by market’sparticipants based on a partially observable stochasticgame

Experimental validation: Comparison with othertrading approaches

1 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

A Prediction market is

a market-based mechanism used to

- combine the opinions on a future event from differentpeople and

- forecast the possible outcome of the event based on theaggregated opinion

2 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

3 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

4 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

5 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

6 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

7 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

8 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

9 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

10 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

11 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

12 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction Market

13 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Prediction MarketMain Features

A prediction market is run for a real-life unknown event

Each event has a finite duration

Each event’s outcome has a security associated with it

Traders buy and sell the securities based on their beliefsabout the outcome of the event

Traders’ beliefs are expressed as probabilities

Market maker aggregates the probabilities from all thetraders into a single probability, market price

Market price of a security represents the probability ofthe outcome of an event associated with that securityhappening

Traders get paid according to their reported beliefs

14 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

A Multi-Agent Prediction Market

Software trading agents perform calculations and tradeon behalf of human tradersProvides testbed for modeling different strategicbehaviors of traders through simulations with tradingagents

15 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Do Prediction Markets Work?Yes, evidence from real markets, laboratory experiments, and theory

I.E.M. beat political polls 451/596 [Forsythe 1999, Berg2001, Pennock 2002]

HP market beat sales forecast 6/8 [Plott 2000]

Sports betting markets provide accurate forecasts ofgame outcomes [Debnath 2003, Schmidt 2002]

Market games work [Pennock 2001]

Laboratory experiments confirm informationaggregation [Forsythe 1990, Plott 1997, Chen 2001]

Theory of Rational Expectations [Lucas 1972, Grossman1981]

and more...

16 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Research Problem Addressed

Develop a formal, game-theoretic model of the tradingagent behavior in prediction markets including

- impact of information from external sources on tradingagent decisions/behavior,

- a solution concept for calculating the equilibriumstrategies of the trading agents

Research Questions:

How does different traders’ behaviors affect marketprices?

What trading strategies give the highest utilities to thetraders?

How can prediction markets incentivize traders toparticipate and report their beliefs truthfully in order toachieve a higher prediction accuracy?

17 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Our Solution

A partially observable stochastic game with information(POSGI)-based model of the trading agent behavior

A correlated equilibrium (CE)-based solution todetermine equilibrium strategy in the POSGIrepresentation

18 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Partially Observable Stochastic Game withInformation (POSGI)

- Logarithmic Market Scoring Rule (LMSR) [Hanson2007] gives formula to calculate aggregate market pricefrom the outstanding quantity of a security

19 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


- Logarithmic Market Scoring Rule (LMSR) [Hanson2007] gives formula to calculate aggregate market pricefrom the outstanding quantity of a security

19 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


- Belief state is updated using a Bayesian model of pastbeliefs, past actions and current observation

20 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


- Information signal can be {−1, 0, 1} representingpositive, neutral, or negative information

21 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


22 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Two Agent Trading Scenario with 1 securityExample

Actions = to buy, sell, or hold one security

Market state is denoted by q

C(q) is the cost function calculated by LMSR, thatreflects the total money collected by the market maker

Can construct a normal form game to capture thedecision problem for each agent

23 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Correlated Equilibrium (CE)Set up

Let N be the set of trading agents

Let Ai be the set of actions for agent i ∈ NLet Φi be the set of mixed strategy profiles defined overAi

Joint strategy space is then Φ = ×|N |i=1Φi

Let φ ∈ Φ be a strategy profile and φi denote agent i’scomponent in φ

Let ui(φ) be the utility of agent i from joint strategy φ

24 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Correlated Equilibrium (CE)

Correlated Equilibrium is a probability distribution p onΦ such that

for all agents i and all strategies φi, φ′i

if all agents follow a strategy profile φ,agent i has no incentive to play another strategy φ′iinstead

The Chicken-Dare GameAssume there is a trusted third party that draws astrategy for each player and announces it to each playerseparatelyPlayers agree to follow the strategy suggested by thethird partyPlayers get higher payoffs when use CE than NE

25 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Correlated Equilibrium (CE) Algorithm

CE algorithm applied to our POSGI model is based onthe method proposed by Papadimitriou [2008]

CE algorithm of the POSGI gives the action (buy orsell; quantity) for each agent at each time step

Time complexity of our CE algorithm is O(|N | × |Φi|2)

26 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Correlated Equilibrium (CE)with Trading Agents’ Risk Preferences

The beliefs and risk preferences of traders are directlycorrelated [Kadane 1988, Dimitrov 2008]

Use constant relative risk averse (CRRA) utility function

To find CE in the market with risk-averse tradingagents:

- Find the set of all Pareto optimal strategy profiles- Check whether a Pareto optimal strategy profile satisfies

CE constraints

27 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Correlated Equilibrium (CE)with Trading Agents’ Risk Preferences

A strategy profile φP is Pareto optimal if there does notexist another strategy profile φ′ such thatui(φ

′) ≥ ui(φP )∀i ∈ N with at least one inequalitystrict

Pareto optimal strategy is found by maximizingweighted utilities and solving the maximization problemusing Lagrangian method

Proposition

If p is a correlated equilibrium and φP is a Pareto optimalstrategy profile calculated by p in a prediction market withrisk averse agents, then the strategy profile φP is incentivecompatible, that is each agent is best off reporting truthfully

28 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Experimental ResultsSet up

Default values for market related parameters came fromIowa Electronic Marketplace (IEM) movie market

Assume there is one event with two outcomes

Consider only disjoint events

Report utilities and market price for the securitycorresponding to the outcome of the event being 1

29 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Experimental ResultsSet up

We use the following strategies for comparison of the tradingagents’ and market’s behavior

1 ZI (Zero Intelligence) - each agent submits randomlycalculated quantity to buy or sell

2 ZIP (Zero Intelligence Plus) - each agent selects a quantityto buy or sell that satisfies a particular level of profit

3 CP (by Preist and Tol) - each agent adjusts its quantity tobuy or sell based on past prices and tries to choose thatquantity so that it is competitive among other agents

4 GD (by Gjerstad and Dickhaut) - each agent maintains ahistory of past transactions and chooses the quantity to buyor sell that maximizes its expected utility

5 DP (Dynamic Programming solution for POSG game) - each

agent uses dynamic programming solution to find the best

quantity to buy or sell [Hansen 2004]30 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Experimental ResultsRisk-neutral Agents

(a) DP strategy has the highest percentage of adoption ofsome CE

(b) Prediction market with trading agents using CE strategyresult in more accurate market prices

(c) Trading agents using CE strategy obtain 38% moreutility than the agents following the next bestperforming strategy DP

31 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work





31 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work





31 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


(a) Trading agents using CE strategy obtain more utilitythan trading agents using ZIP strategy

(b) Trading agents using CE strategy obtain more utilitythan trading agents using CP strategy

32 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


(a) Trading agents using CE strategy obtain more utilitythan trading agents using ZIP strategy

(b) Trading agents using CE strategy obtain more utilitythan trading agents using CP strategy

32 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


(c) Trading agents using CE strategy obtain more utilitythan trading agents using GD strategy

(d) Trading agents using CE strategy obtain more utilitythan trading agents using DP strategy

33 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work


(c) Trading agents using CE strategy obtain more utilitythan trading agents using GD strategy

(d) Trading agents using CE strategy obtain more utilitythan trading agents using DP strategy

33 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Experimental ResultsRisk-averse Agents


(b) Prediction market with risk-averse trading agents usingCE strategy result in more accurate market prices

(c) Risk-averse trading agents using CE strategy obtain41% more utility than the agents following the nextbest performing strategy DP

34 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work





34 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work





34 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Conclusions and Future Work

In this work we have:

- described a novel multi-agent based representation ofthe prediction market using POSG

- developed a CE solution for solving POSG

- empirically compared different agent behavior strategiesin the prediction market

- showed how CE can be obtained in the predictionmarket with risk averse agents

In the future we plan to:

Conduct experiments in an n-agent scenario using richercommercial data sets

Investigate the dynamics evolving from multipleprediction markets that interact with each other

35 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Conclusions and Future Work

In this work we have:

- described a novel multi-agent based representation ofthe prediction market using POSG

- developed a CE solution for solving POSG

- empirically compared different agent behavior strategiesin the prediction market

- showed how CE can be obtained in the predictionmarket with risk averse agents

In the future we plan to:

Conduct experiments in an n-agent scenario using richercommercial data sets

Investigate the dynamics evolving from multipleprediction markets that interact with each other

35 / 37




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

References

1 Y. Chen, D. Pennock. Utility Framework forBounded-Loss Market Maker. Proc. of the 23rdConference on Uncertainty in Artificial Intelligence (UAI2007), pages 49-56, 2007.

2 E. Hansen, D. Bernstein, S. Zilberstein. Dynamicprogramming for partially observable stochastic games.In Proceedings of the 19th National Conference onArtificial Intelligence, pages 709-715, 2004.

3 R. Hanson. Logarithmic Market scoring rules forModular Combinatorial Information Aggregation.Journal of Prediction Markets, 1(1):3-15, 2007.

4 Iowa Electronic Marketplace. URL:www.biz.uiowa.edu/iem/

5 C. Papadimitriou, T. Roughgarden. Computingcorrelated equilibria in multi-player games. Journal ofACM, 55(3):1-29, 2008.36 / 37

www.biz.uiowa.edu/iem/




Outline

Introduction

Research Problem

POSGI


ExperimentalResults

Future Work

Thank You!

Questions?

[email protected]

C-MANTIC Research Group

http://cmantic.unomaha.edu/

37 / 37

mailto:[email protected]



Education

Icec11slides uno