Katia SycaraAAAI'971 James Bond and Michael Ovitz The Secret Life of Agents Katia Sycara The Robotics Institute Carnegie Mellon University Pittsburgh,

Katia Sycara AAAI'97 1

James Bond and Michael Ovitz The Secret Life of Agents

Katia Sycara

The Robotics Institute

Carnegie Mellon University

Pittsburgh, PA 15213

(412) 268-8825

[email protected]

http://www.cs.cmu.edu/~sycara

Project Homepage: http://www.cs.cmu.edu/~softagents


Liren Chen

Somesh Jha

Rande Shern

Dajun Zeng

Keith Decker

Anadeep Pannu

Vandana Verma

Team Members CMU

Prasad Chalasani

Kostya Domashnev

Onn Shehory


Talk Outline

• Introduction to Agents

• Motivations and advantages of distributed agent technology

• The Retsina Approachª

• Retsina Agent Architecture

• Middle Agents

• Multi-agent interaction protocols (negotiation, contingent contracting)

• Retsina Applications

• Concluding Remarks

________________________________

ª Retsina stands for “Reusable Task Structured Intelligent Networked Agents.”


The (Re)-Emergence of Agents: The Marriage of Two Holy Grails

•Goal-directed•Adaptive•Knowledge-based

•Reusability•Robustness•Flexibility

AI SE

Ubiquitous Networked Information Access

Intelligent Software Agents!


What is an Agent?

• A computational system that

– has goals, sensors and effectors

– is autonomous

– is adaptive

– is long lived

– lives in a networked infrastructure

– interacts with other agents


Agent vs. Agent

James Bond Michael Ovitz


Next Generation of Agent Technology

• Currently, agent technology is mostly single agent focusing on information retrieval and filtering according to user profile

• Multi-Agent Systems that interact with humans and each other

• Integrate information management and decision support

• Enable real-time synchronization of the tasks and actions of humans in teams and organizations

• Acquire and disseminate timely and relevant information

• Anticipate and satisfy human information and problem solving needs

• Notify about changes in the environment

• Adapt to user, task and situation


Motivation for Multi Agent Systems (MAS) Technology

• Global Information & Markets

• Increasingly networked world

• Vast quantities of unorganized information

• Diverse information sources

• Inability for human to manage information access process---information overloading

• Moving from locating documents to making decisions


Features of MAS

• Multiple agents connected through communication networks

• Coordination - no agent has sufficient information or capabilities to solve problem aloneª

• Decentralized control - no master agent

• Decentralized data - no global data storage

• Agent Coupling - balancing computation and communication

• Asynchronous - multiple activities operating in parallel

_______________________

ª Agents could be cooperative or self-interested.


MAS Basic Questions (Bond and Gasser, 88)

• Coherence in coordinated decision making

• Recognition and reconciliation of disparate viewpoints or conflicting intentions

• Graceful performance degradation in face of missing information and resources

• Satisfaction of system-wide criteria (e.g., optimality of solution, hard real-time deadlines, etc.)

• How to recognize workload imbalances and appropriately redistribute activities and responsibilities among agents

• Modeling other agents

• Synthesizing different views and results


MAS Concepts and Tools

• Distributed Constraint Satisfaction and Optimization (Yokoo, et al,~1991; Sycara, et al. 1991)

• Distributed Truth Maintenance and Multi-Agent Search (Huhns et al. 1991, Ishida 1997)

• Organizational Structuring (Lesser and Corkill 1991, Gasser 1993, Decker 1995)

• Multiagent Planning (Georgeff 1983, 1984, 1995; Durfee and Lesser 1991, Jennings 1995, Grosz et al. 1996)

• Contracting (Smith 1980, Mueller 1993, Sandholm 1997)

• Negotiation (Sycara 1990; Kraus 1991, Zlotkin 1996)


MAS Concepts and Tools (Contd.)

• Economic and Game Theoretic Techniques (Rosenschein 1995; Gmytrasiewicz et al. 1991, Wellman 1993)

• Open systems (Hewitt 1991, Gasser 91)

• Multiagent Logics and Ecological Approaches (Cohen and Levesque 1987; Ferber 1990, Shoham 1993; Huberman et al. 1996)

• Social Laws and Norms (Tenenholtz 1991, Castelfranchi 1993)

• Multi-Agent Learning (Sen 1993, Durfee 1994, Sycara 1996)


Retsina Approach

• Architecture that includes data- and knowledge-bases and a distributed collection of intelligent agentsª

• Reusable and composable agent components (agent editor, agent operating system)

• Operates in an open world

– agents, network links, and information sources appear/disappear

– uncertainty

• Dynamic agent team formation on-demand

____________________________

ª K.Sycara, K.Decker, A.Pannu, M.Williamson, and D.Zeng. Distributed Intelligent Agents. IEEE Expert, Dec-96


Retsina Functional Organization


Main System Issues in MAS

• Single agent architecture

– Retsina agent architecture

– Agent Self-cloning

• Finding other agents

– Middle agents

– Matchmaking and brokering

• Agent interaction protocols

– Negotiation

– Contingent contracting


Retsina Agent Architecture

• Planning

– hierarchical task network-based formalism

– library of task reduction schemas

alternative task reductions

contingent plans, loops

– incremental task reduction, interleaved with execution

information gathered during execution directs future planning

• Scheduling

– fully expanded leaf nodes = executable basic actions

– enabled actions (all parameters and provisions in place)

– adjust periodic tasks with missed deadlines


•Communication and Coordination

–processes incoming and outgoing messages

–creates new goals/objectives

–determines coordination interactions

–addresses security issues

•Execution Monitoring

•setup execution context (parameters and provisions)

•action monitoring

deadlines/timeouts

data collection for decisions (e.g. cloning)

•complete execution (provide results to appropriate downstream actions)


Retsina Agent Architecture


A task Structure (Advertisement Task Structure)


Reusable Behaviors

• Advertising

– send agent capability model to middle-agent(s)

– shared query behavior for other agents

• Polling for messages

• Answering queries: one-shot and periodic

• Monitoring for changes and notification

• Self-cloning


Self-Cloning Process

• Agents that perceive an overload look for other agents to pass tasks to (simple model to predict idle time using learned estimation of task durations)

• When other agents not found:

– locate a host with resources for cloning

– create a clone on the host

– partition tasks and transfer to clone

– the ``old'' agent updates its advertisement; the ``clone'' agent advertises

• When clone is idle -- consider self-extinction, and shut down if necessary.


Cloning Experimental Setting

• Number of agents: 10 to 20.

• Number of clones allowed: up to 10.

• Number of tasks arriving at the system: up to 1000.

• Task distribution with respect to the required capabilities for execution: normal distribution, where 10% of the tasks are beyond the capabilities of the agents.

• Agent capabilities: an agent can perform up to 20 average tasks

simultaneously.


Task Completion w/wo Cloning


Middle Agents

• An agent needs to have some task/service performed. How can it find agents able to perform that task?

• In an open system:

– agents generally don't have knowledge of all other agents

– service providers are liable to come and go over time

• A solution: middle agents that specialize in making connections between agentsª

_________________________

ª K.Decker, K.Sycara, M. Williamson. Middle-Agents for the Internet. IJCAI-97


Middle Agent Types

PreferencesInitially Known By

Provider Only Provider +Middle Agent

Provider + Middle +Requester

Requester Only (Broadcaster) “Front-Agent” Matchmaker/Yellowpages

Requester +Middle Agent

Anonymizer Broker Recommender

Requestor +Middle + Provider

Blackboard Introducer/Bodyguard

Arbitrator

Capabilities Initially Known By


Matchmaking: Agent Yellow Page Services


Matchmaking in Agent Coordination

• When an agent A advertises its capability, it Intends toª perform any task that fits the specification of that capability.

– In the Retsina system an agent A advertises a relational schema SA, i.e., agent A intends to answer any query on its schema.

• If an agent B finds another agent A with a certain capability through matchmaking, B believes that agent A can successfully perform the task.

• Matchmaking gives operational semantics to predicates such as Intend.to, Bel.

_______________________________

ª Grosz and Kraus 1996


Performance of Match-made System


Performance of Brokered System


Agents in Electronic Commerce


Adaptive Negotiation (the Bazaar Model)

• Aims at modeling multi-issue negotiation processesª

• Combines the strategic modeling aspects of game-theoretic models and single agent sequential decision making models

• Supports an open world model

• Addresses heterogeneous multi-agent learning utilizing the iterative nature of sequential decision making and the explicit

representation of beliefs about other agents ______________________________

ª D.Zeng and K.Sycara. “Benefits of Learning in Negotiation.” Proceedings of AAAI-97.


Utility of Learning: Experimental Design

• The set of players N is comprised of one buyer and one supplier who make alternative proposals.

• For simplicity, the range of possible prices is from 0 to 100 units and this is public information

• The set of possible actions (proposed prices by either the buyer or the supplier) A equals to {0, 1, 2,…, 100}

• Reservation prices are private information.

• Each player's utility is linear to the final price ( a number between 0 and 100) accepted by both players

• Normalized Nash product as joint utility (the optimal joint utility

when full information is available is 0.25)


Average Performance of Three Experimental Configurations in Bazaar

• A non-learning agent makes decisions based solely on his own reservation price

• A learning agents makes decisions based on both the agent's own and the opponent's reservation price

Configuration JointUtility

Buyer’sUtility

Supplier’s Utility

# of ProposalsExchanged

Both Learn 0.22 0.49 .051 24

Neither Learn 0.18 0.49 0.51 34

Only Buyer Learns 0.15 0.59 0.41 28


Contingent Contracts and Options

• Most multi-agent systems don't handle uncertainty effectively

– rigid task delegation mechanism (contracts are binding rather than contingent

– no explicit modeling of stochastic events

– no explicit mechanism for controlling agent performance variability

• We are exploring the use of option pricing to address the above issues


Evaluation of Contingent vs Binding Contracts

• In the experiments, we only had two kinds of agents:

– Interface Agents: Accept queries from the user.

– Information Agents: Answer queries given by the Interface agents.

• In each cycle a new information agent with a load randomly distributed

between L and 0.9 appears with probability .

• When a new information agent comes up, interface agents have the option to abort the query on the old information agent and restart it on the new one.

• Interface agents can only switch a bounded number of queries to the new agent. This is indicated as Bound in the graphs.

• In the experiments the average delay in answering the queries was measured. This is indicated as Delay in the graphs.


Contingent Contracts

L


Contingent Contracts

Bound


Contingent Contracting to Handle Unreliability of Information Sources

• Uncertain waiting time in response to queries

– random network congestion

– uncertain serve congestion/breakdown


The Query Restart Problem

• Agent A sends query to Agent B.

• Agent B can complete the query in time X, where

X = 1 with probability p.

X = c (c > 1) with probability 1 - p.

Expectation: EX = p + (1 - p) c

• If not done by time 1, should agent A abort and restart, or wait?

• Can restarting reduce expectation? The variance? Both?

• Does it help to repeatedly restart k times?


Strategy: restart just after time 1, if not done by then.

Let Xi = completion time of i'th query, i = 1,2.

X1, X2 are independent, identically distributed.

New completion time is Y:

Y =

New expectation

EY = p + (1 - p)(1 + E X2) (X1, X2 indep.)

= 1 + p (1 - p) + (1 - p) c

If (and only if) c > 1 + 1 / p, EY < X1 !

A Simple Scenario: Single restart

{ 1 if X1 = 1,

1 + X2 if X1 = c.


A Simple Scenario: k Restarts

Number of Restarts k


Applications

• Visitor Hoster (PLEIADES)

• Satellite Visibility (THALES)

• Portfolio Management (WARREN)


Characteristics of Retsina

• Open System

• Adaptivity at the agent and organization level provides robustness

• Service-based, economic coordination of agents

• Reusable and extensible domain-independent computational infrastructure

• Integrates information gathering and execution monitoring with decision making

• Framework for addressing uncertainty and strategic

interactions


Future of Software Agents• Agent-based software development is an emerging paradigm

• Agent society that parallels human society

• Implication of the emergence of agent society for human workplaces, institutions, and social relations

• Agent society as a unit of intelligence

• Opportunities and Challenges

– The WEB is a vast knowledge base presenting novel opportunities for AI

– Overall system (human and software agent) predictability

– Security, privacy, trust issues

– Integration of legacy systems


Overall Issues in Open MAS

• Overall agent organization



Agent Self-cloning


– Middle agents

Matchmaking and brokering


– Negotiation







Agent Self-cloning


– Middle agents



– Negotiation







Agent Self-cloning


– Middle agents



– Negotiation


Documents

Katia SycaraAAAI'971 James Bond and Michael Ovitz The Secret Life of Agents Katia Sycara The Robotics Institute Carnegie Mellon University Pittsburgh,