© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN 1 ART Testbed Join the discussion group at:

1© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN

ART Testbed

Join the discussion group at:

http://www.art-testbed.net


ART Testbed Questions Can agents request reputations about themselves? Can an agent produce an appraisal without purchasing opinions? Does the Testbed assume a common representation for

reputations? Does the Testbed prevent agents from winning via action-planning

skills, as opposed to trust-modeling skills? What if an agent can’t or won’t give a reputation value? Why does it cost more to generate an accurate opinion than an

inaccurate one? Why not have a centralized reputation broker? Isn’t it unrealistic to assume a true value of a painting can be

known? Is art appraisal a realistic domain? Why not design an incentive-compatible mechanism to enforce

truth-telling?


“Really Good” ART Testbed Questions Is there a consensus on the definitions of “trustworthiness” and

“reputation”? How can collusion be avoided? Is truth-telling a dominant strategy? Will the system reach equilibrium, at which point reputations are no

longer useful? What happens if client fee (100), opinion cost (10), and reputation

cost (1) are changed? Do any equilibria exist? What happens when agents enter or leave the system? When will agents seek out reputations? Space of experiments is underexplored—that’s a good thing!


Questions about the Paper What is a “trust model”? How does q-learning work? How related to reinforcement learning?

How do rewards tie in? What is lambda? How can experience- and reputation-based learning be combined to

overcome the weaknesses of each (intermediate lambda values)? What about different combinations of (more sophisticated) agents in

a game? Why the assumptions chosen? They seem too extreme. Reputation decisions weren’t examined very well.

The Agent Reputation and Trust Testbed: The Agent Reputation and Trust Testbed: Experimentation and Competition Experimentation and Competition

for Trust in Agent Societiesfor Trust in Agent Societies

Karen K. Fullam1, Tomas B. Klos2, Guillaume Muller3, Jordi Sabater4, Andreas Schlosser5, Zvi Topol6, K. Suzanne Barber1, Jeffrey S. Rosenschein6, Laurent Vercouter3, and Marco Voss5

1Laboratory for Intelligent Processes and Systems, University of Texas at Austin, USA2Center for Mathematics and Computer Science (CWI), Amsterdam, The Netherlands

3 Ecole Nationale Superieure des Mines, Saint-Etienne, France4Institute of Cognitive Science and Technology (ISTC), National Research Council (CNR), Rome, Italy

5IT Transfer Office, Darmstadt University of Technology, Darmstadt, Germany6Multiagent Systems Research Group—Critical MAS, Hebrew University, Jerusalem, Israel

The Agent Reputation and Trust Testbed, 2006

Appraiser Agent

Appraiser Agent

Client

Client

Client

Client Share

Opinions and Reputations

Appraiser Agent

Appraiser Agent

Appraiser Agent

Testbed Game RulesTestbed Game Rules

Agents function as art appraisers with varying expertise in

different artistic eras.

For a fixed price, clients ask appraisers to provide

appraisals of paintings from various eras.

If an appraiser is not very knowledgeable

about a painting, it can purchase "opinions"

from other appraisers.

Appraisers can also buy and sell reputation information about other

appraisers.

Appraisers whose appraisals are more

accurate receive larger shares of the client base

in the future. Appraisers compete to achieve the highest earnings by the end of the game.


Step 1: Client and Expertise AssignmentsStep 1: Client and Expertise Assignments

Appraisers receive clients who pay a fixed price to request appraisals

Client paintings are randomly distributed across eras

As game progresses, more accurate appraisers receive more clients (thus more profit)


Step 2: Reputation TransactionsStep 2: Reputation Transactions

Appraisers know their own level of expertise for each era

Appraisers are not informed (by the simulation) of the expertise levels of other appraisers

Appraisers may purchase reputations, for a fixed fee, from other appraisers

Reputations are values between zero and one • Might not correspond to

appraiser’s internal trust model

• Serves as standardized format for inter-agent communication


Step 2: Reputation TransactionsStep 2: Reputation Transactions

ProviderRequester

Request

Accept

Payment

Reputation

Requester sends request message to a potential reputation provider, identifying

appraiser whose reputation is

requested

Potential reputation provider sends

“accept” message

Requester sends fixed payment to the

provider

Provider sends reputation

information, which may not be truthful


Step 3: Opinion TransactionsStep 3: Opinion Transactions

For a single painting, an appraiser may request opinions (each at a fixed price) from as many other appraisers as desired

The simulation “generates” opinions about paintings for opinion-providing appraisers

Accuracy of opinion is proportional to opinion provider’s expertise for the era and cost it is willing to pay to generate opinion

Appraisers are not required to truthfully reveal opinions to requesting appraisers


Step 3: Opinion TransactionsStep 3: Opinion Transactions

ProviderRequester

Request

Certainty

Payment

Opinion

Requester sends request message to a

potential opinion provider, identifying

painting

Potential provider sends a certainty

assessment about the opinion it can provide- Real number (0 – 1)

- Not required to truthfully report certainty

assessment

Requester sends fixed payment to the

providerProvider sends

opinion, which may not be truthful


Step 4: Appraisal CalculationStep 4: Appraisal Calculation

Upon paying providers and before receiving opinions, requesting appraiser submits to simulation a weight (self-assessed reputation) for each other appraiser

Simulation collects opinions sent to appraiser (appraisers may not alter weights or received opinions)

Simulation calculates “final appraisal” as weighted average of received opinions

True value of painting and calculated final appraisal are revealed to appraiser

Appraiser may use revealed information to revise trust models of other appraisers

The Laboratory for Intelligent Processes and SystemsElectrical and Computer Engineering

The University of Texas at Austinhttp://www.lips.utexas.edu

Karen K. Fullam

2006 ART TestbedCompetition Results


Competition Organization

“Practice” Competition• Spanish Agent School, Madrid, April 2006 • 12 participants

International Competition • AAMAS, Hakodate, May 2006 • Preliminary Round

13 Participants 5 games each

• Final Round 5 Finalists 10 games with all finalists participating


Bank BalancesIam achieves highest bank

balances


Opinion Purchases

Joey and Neil do not purchases opinions

Sabatini purchases the most opinions


Opinion Earnings

Sabatini and Iam provide the most opinions

Neil and Frost do not

provide many

opinions


Opinion Sensing Costs

Iam invests the

most in opinions it generates


Expertise vs. Bank Balance

Iam’s average expertise was

not significantly higher than

others’

Greater Expertise

The Laboratory for Intelligent Processes and SystemsElectrical and Computer Engineering

The University of Texas at Austinhttp://www.lips.utexas.edu

Karen K. Fullam

K. Suzanne Barber

Learning Trust Strategies in Reputation Exchange Networks


Trust Decisions in Reputation Exchange Networks Agents perform transactions to obtain needed resources

• Transactions have risk because partners may be untrustworthy• Agents must learn whom to trust and how trustworthy to be

When agents can exchange reputations• Agents must also learn when to request reputations and what

reputations to tell• Agents’ trust decisions affect each other

Difficult to learn each decision independently

Resources (goods, services,

information)

How trustworthy should I be?

Reputations

Which reputations should I listen to?

What reputations should I tell?

Should I trust?

If I lie to others that C is bad, can I monopolize C’s

interactions?

If I cheat A, and A tells B, will it

hurt my interactions with B?


Enumerating Decisions in a Trust StrategyT

rust

erT

rust

ee

Ag

ent

Ro

le

Transaction

Fundamental Reputation

How trustworthy should I be?

Should I trust?

Should I tell an accurate

reputation?

Should I believe this reputation?

Truster Trustee

combinations aen2

combinations a en

combinations aen2

combinations a en

Num agents = aNum transaction types = eNum choices/decision = n

How to learn the best

strategy with so many choices?

If these decisions affect

each other, there are possible

strategies!

2 1ae an


Reinforcement Learning

Select a strategy

Strategy feedback influences expected reward

Strategies with higher expected

rewards are more likely to be selected

Strategy Expected RewardABCD


Learning In Reputation Exchange Networks

Strategy Expected RewardTr(A),Tr(B),Tr(C)…

. . .

⌐Tr(A),Tr(B),Tr(C)…

Tr(A),⌐Tr(B),Tr(C)…

⌐Tr(A),⌐Tr(B),Tr(C)…

Tr(A),Tr(B),⌐Tr(C)…

⌐Tr(A),Tr(B),⌐Tr(C)…

⌐Tr(A),⌐Tr(B),⌐Tr(C)…

Tr(A),⌐Tr(B),⌐Tr(C)…

Decision Expected RewardTr(A)

⌐Tr(A)

Decision Expected RewardTr(B)

⌐Tr(B)

Decision Expected RewardTr(C)

⌐Tr(C)Removing

interdepend-encies makes each decision in the strategy

learnable

Use the ART Testbed as a case study

Because decisions are interdependent,

there are . possible

strategies!

2 1ae an


Many Interdependent Decisions

Accuracy of Opinion

Requester’s appraisals

Reputation Requester’s

reputation costs

Opinion Requester’s

client revenue

Other Appraisers’

client revenue

Number of requests received

by Opinion Provider

Opinion Provider’s

opinion revenue


opinion order costs


opinion costs

Accuracy of Reputation

Requester’s trust models

Number of requests received

by Reputation Provider

Reputation Provider’s reputation revenue

Opinion Provider

Opinion Requester

Reputation Provider

Reputation Requester

When Reputation Requester is

Opinion Requester


Opinion Requester Feedback


client revenue


opinion costs

Opinion Requester

Assume: Client revenue feedback is wholly attributed to Opinion Requester

decision

Divide revenue (client revenue) among opinions

based on opinion accuracy


client revenue


opinion costs

Reward = –

Client Revenue

Opinion Purchase

Costs


Opinion Provider Feedback

Other Appraisers’

client revenue


opinion revenue


opinion order costs

Opinion Provider

Assume: Client revenue is not related to Opinion Provider

decision

Reward = –

Opinion Selling

Revenue

Opinion Generating

Costs


opinion revenue


opinion order costs


Reputation Provider Feedback

Other Appraisers’

client revenue


Assume: Client revenue is not

related to Reputation Provider decision

Reward =

Reputation Selling

Revenue


Reputation Provider


Reputation Requester

Reputation Requester Feedback

Reputation Requester’s

reputation costs


client revenue


opinion costs

determines influence of: past experience vs. reputations in

deciding to purchase opinions

= 0: Past experience only Opinion-requesting decision No reward for requesting reputations

= 1: Reputations only Reputation-requesting decision Full reward for requesting reputations

Opinion Requester

Reward = –

Opinion Requester

Reward

Reputation Purchase

Costs( )

Documents

© 2006 THE UNIVERSITY OF TEXAS AT AUSTIN 1 ART Testbed Join the discussion group at: