Modeling Agents’ Reasoning in Strategic Situations

Avi Pfeffer

Sevan Ficici

Kobi Gal

The Big Question

• Why do agents (people or computers) do things in strategic situations?

• Possible directions– social norms– cognitive or computational limitations– beliefs– preferences– reasoning patterns

Outline

• Characterizing the reasoning patterns agents use (Avi)

• The Colored Trails framework (Kobi)• Modeling people’s preferences and

reasoning in dynamic situations (Kobi)• Modeling how people reason about other

people (Sevan)• Modeling the subjective beliefs people

have about other people (Sevan)

Why Understand Reasoning Patterns?

• Understand people’s strategies: people are likely to prefer some reasoning patterns to others

• Explanation: if we want to explain people’s strategies, or we want to explain computational agents’ strategies to people, it is useful to know what reasoning patterns are being employed

• Computational benefits• Analogy to logic

Multi-Agent Influence Diagrams (MAIDs)

• Graphical representation of strategic situations, based on Bayesian networks

• Compact representation– exploits independences in decision making– naturally captures limited observations– capture structure in multi-attribute utility

functions

• Tell the “story” of a game• Ideal representation for thinking about

reasoning patterns

Bayesian networks

• Represent probability distribution• Nodes are random variables• Edges represent direct influence• Each node has conditional probability of the node given

its parents

Seismic Structure

Oil

Test

Drill

Profit

Test ResultP(Oil | Seismic Structure)

Influence Diagrams

• Represent single-agent decision problems– Chance nodes (ellipses)– Decision nodes (rectangles)– Utility nodes (diamonds)

• Parents of decision node represent information available to decision maker at time of making decision

Seismic Structure

OilTest Result

Drill

Test

Profit

Multi-Agent Influence Diagrams

• Represent multi-agent strategic situations• Like influence diagrams, except that decision and utility

nodes are associated with specific agents

Seismic Structure

OilTest Result

Drill

Test

Tester’s Profit Driller’s Profit

Reasoning Pattern #1: Direct Effect

• An agent takes a decision because of its direct effect on its utility– without being mediated by other agents’ actions

DA

UA

Influencing

• All other reasoning patterns fall under the category of influencing: trying to get another agent to do something that is beneficial to you

• The possible reasoning patterns depend on what strategies are considered for other agents

• Let’s restrict attention to those strategies where the other agent has “good reason” to play them

Reasoning Pattern #2: Manipulation

• B knows about A’s action• A cares about B’s action• A’s action influences B’s outcome, so B has to

react to what A does• A can manipulate B to respond to her in a

favorable way

DA

UA

DB

UB

Reasoning Pattern #3: Signaling

• A communicates something that she knows to B, thus changing B’s behavior

• Interesting point: A must care about the thing she is communicating, otherwise B won’t believe her

DA

UA

DB

UB

C

Reasoning Pattern #4: Revealing/Denying

• A causes B to find out about information A herself does not know

• Tiger example• Also works the other way round

DA

UA

DB

UB

C

E

Key Question

• Are these all the reasoning patterns?• Answer: it depends what strategies you

allow for other agents• If you allow general strategies, any pattern

in which there is a directed path from a decision node to a utility node is a pattern

• But if we restrict attention to a more restricted class of strategies, we get a more nuanced answer

Well-Distinguishing Strategies

• Intuition: if a strategy makes a distinction, the distinction should make a difference

• A well-distinguishing (WD) strategy is one in which– if the strategy distinguishes between two

values of the parents of a node, the expected utility is different for the two values

– if the strategy assigns different probability to two actions, the expected utility of the two actions is different

Reassuring Fact

• Theorem: The set of WD strategies always includes a Nash equilibrium

• But not all Nash equilibria are WD

• And not all WD strategies are Nash equilibria

Completeness Result

• Theorem: If other agents are playing WD strategies, then the four patterns of reasoning described earlier are the only patterns in which an agent cares about its decision

Converse

• When one of the reasoning patterns holds, does an agent necessarily care about its decision?

• Answer: not in general• But we can prove a weak converse:• Theorem: If one of the reasoning patterns holds

in a MAID, there exist parameter values for the MAID such that the agent cares about its decision

• Similar situation to Bayesian networks

Opportunities for Synergistic Research

• What reasoning patterns do people use? • Do people prefer some reasoning patterns

to others?• How can we study this in the lab or the

field?• What about non-WD strategies (which I

think people use) – can we find a good solution concept that adequately capture’s people’s behavior?