Katia SycaraAAAI'971 James Bond and Michael Ovitz The Secret Life of Agents Katia Sycara The...

Katia Sycara AAAI'97 1

James Bond and Michael Ovitz The Secret Life of Agents

Katia Sycara

The Robotics Institute

Carnegie Mellon University

Pittsburgh, PA 15213

(412) 268-8825

katia@cmu.edu

http://www.cs.cmu.edu/~sycara

Project Homepage: http://www.cs.cmu.edu/~softagents

Liren Chen

Somesh Jha

Rande Shern

Dajun Zeng

Keith Decker

Anadeep Pannu

Vandana Verma

Team Members CMU

Prasad Chalasani

Kostya Domashnev

Onn Shehory

Talk Outline

• Introduction to Agents

• Motivations and advantages of distributed agent technology

• The Retsina Approachª

• Retsina Agent Architecture

• Middle Agents

• Multi-agent interaction protocols (negotiation, contingent contracting)

• Retsina Applications

• Concluding Remarks

________________________________

ª Retsina stands for “Reusable Task Structured Intelligent Networked Agents.”

The (Re)-Emergence of Agents: The Marriage of Two Holy Grails

•Goal-directed•Adaptive•Knowledge-based

•Reusability•Robustness•Flexibility

Ubiquitous Networked Information Access

Intelligent Software Agents!

What is an Agent?

• A computational system that

– has goals, sensors and effectors

– is autonomous

– is adaptive

– is long lived

– lives in a networked infrastructure

– interacts with other agents

Agent vs. Agent

James Bond Michael Ovitz

Next Generation of Agent Technology

• Currently, agent technology is mostly single agent focusing on information retrieval and filtering according to user profile

• Multi-Agent Systems that interact with humans and each other

• Integrate information management and decision support

• Enable real-time synchronization of the tasks and actions of humans in teams and organizations

• Acquire and disseminate timely and relevant information

• Anticipate and satisfy human information and problem solving needs

• Notify about changes in the environment

• Adapt to user, task and situation

Motivation for Multi Agent Systems (MAS) Technology

• Global Information & Markets

• Increasingly networked world

• Vast quantities of unorganized information

• Diverse information sources

• Inability for human to manage information access process---information overloading

• Moving from locating documents to making decisions

Features of MAS

• Multiple agents connected through communication networks

• Coordination - no agent has sufficient information or capabilities to solve problem aloneª

• Decentralized control - no master agent

• Decentralized data - no global data storage

• Agent Coupling - balancing computation and communication

• Asynchronous - multiple activities operating in parallel

_______________________

ª Agents could be cooperative or self-interested.

MAS Basic Questions (Bond and Gasser, 88)

• Coherence in coordinated decision making

• Recognition and reconciliation of disparate viewpoints or conflicting intentions

• Graceful performance degradation in face of missing information and resources

• Satisfaction of system-wide criteria (e.g., optimality of solution, hard real-time deadlines, etc.)

• How to recognize workload imbalances and appropriately redistribute activities and responsibilities among agents

• Modeling other agents

• Synthesizing different views and results

MAS Concepts and Tools

• Distributed Constraint Satisfaction and Optimization (Yokoo, et al,~1991; Sycara, et al. 1991)

• Distributed Truth Maintenance and Multi-Agent Search (Huhns et al. 1991, Ishida 1997)

• Organizational Structuring (Lesser and Corkill 1991, Gasser 1993, Decker 1995)

• Multiagent Planning (Georgeff 1983, 1984, 1995; Durfee and Lesser 1991, Jennings 1995, Grosz et al. 1996)

• Contracting (Smith 1980, Mueller 1993, Sandholm 1997)

• Negotiation (Sycara 1990; Kraus 1991, Zlotkin 1996)

MAS Concepts and Tools (Contd.)

• Economic and Game Theoretic Techniques (Rosenschein 1995; Gmytrasiewicz et al. 1991, Wellman 1993)

• Open systems (Hewitt 1991, Gasser 91)

• Multiagent Logics and Ecological Approaches (Cohen and Levesque 1987; Ferber 1990, Shoham 1993; Huberman et al. 1996)

• Social Laws and Norms (Tenenholtz 1991, Castelfranchi 1993)

• Multi-Agent Learning (Sen 1993, Durfee 1994, Sycara 1996)

Retsina Approach

• Architecture that includes data- and knowledge-bases and a distributed collection of intelligent agentsª

• Reusable and composable agent components (agent editor, agent operating system)

• Operates in an open world

– agents, network links, and information sources appear/disappear

– uncertainty

• Dynamic agent team formation on-demand

____________________________

ª K.Sycara, K.Decker, A.Pannu, M.Williamson, and D.Zeng. Distributed Intelligent Agents. IEEE Expert, Dec-96

Retsina Functional Organization

Main System Issues in MAS

• Single agent architecture

– Retsina agent architecture

– Agent Self-cloning

• Finding other agents

– Middle agents

– Matchmaking and brokering

• Agent interaction protocols

– Negotiation

– Contingent contracting

Retsina Agent Architecture

• Planning

– hierarchical task network-based formalism

– library of task reduction schemas

alternative task reductions

contingent plans, loops

– incremental task reduction, interleaved with execution

information gathered during execution directs future planning

• Scheduling

– fully expanded leaf nodes = executable basic actions

– enabled actions (all parameters and provisions in place)

– adjust periodic tasks with missed deadlines

•Communication and Coordination

–processes incoming and outgoing messages

–creates new goals/objectives

–determines coordination interactions

–addresses security issues

•Execution Monitoring

•setup execution context (parameters and provisions)

•action monitoring

deadlines/timeouts

data collection for decisions (e.g. cloning)

•complete execution (provide results to appropriate downstream actions)

Retsina Agent Architecture

A task Structure (Advertisement Task Structure)

Reusable Behaviors

• Advertising

– send agent capability model to middle-agent(s)

– shared query behavior for other agents

• Polling for messages

• Answering queries: one-shot and periodic

• Monitoring for changes and notification

• Self-cloning

Self-Cloning Process

• Agents that perceive an overload look for other agents to pass tasks to (simple model to predict idle time using learned estimation of task durations)

• When other agents not found:

– locate a host with resources for cloning

– create a clone on the host

– partition tasks and transfer to clone

– the ``old'' agent updates its advertisement; the ``clone'' agent advertises

• When clone is idle -- consider self-extinction, and shut down if necessary.

Cloning Experimental Setting

• Number of agents: 10 to 20.

• Number of clones allowed: up to 10.

• Number of tasks arriving at the system: up to 1000.

• Task distribution with respect to the required capabilities for execution: normal distribution, where 10% of the tasks are beyond the capabilities of the agents.

• Agent capabilities: an agent can perform up to 20 average tasks

simultaneously.

Task Completion w/wo Cloning

Middle Agents

• An agent needs to have some task/service performed. How can it find agents able to perform that task?

• In an open system:

– agents generally don't have knowledge of all other agents

– service providers are liable to come and go over time

• A solution: middle agents that specialize in making connections between agentsª

_________________________

ª K.Decker, K.Sycara, M. Williamson. Middle-Agents for the Internet. IJCAI-97

Middle Agent Types

PreferencesInitially Known By

Provider Only Provider +Middle Agent

Provider + Middle +Requester

Requester Only (Broadcaster) “Front-Agent” Matchmaker/Yellowpages

Requester +Middle Agent

Anonymizer Broker Recommender

Requestor +Middle + Provider

Blackboard Introducer/Bodyguard

Arbitrator

Capabilities Initially Known By

Matchmaking: Agent Yellow Page Services

Matchmaking in Agent Coordination

• When an agent A advertises its capability, it Intends toª perform any task that fits the specification of that capability.

– In the Retsina system an agent A advertises a relational schema SA, i.e., agent A intends to answer any query on its schema.

• If an agent B finds another agent A with a certain capability through matchmaking, B believes that agent A can successfully perform the task.

• Matchmaking gives operational semantics to predicates such as Intend.to, Bel.

_______________________________

ª Grosz and Kraus 1996

Performance of Match-made System

Performance of Brokered System

Agents in Electronic Commerce

Adaptive Negotiation (the Bazaar Model)

• Aims at modeling multi-issue negotiation processesª

• Combines the strategic modeling aspects of game-theoretic models and single agent sequential decision making models

• Supports an open world model

• Addresses heterogeneous multi-agent learning utilizing the iterative nature of sequential decision making and the explicit

representation of beliefs about other agents ______________________________

ª D.Zeng and K.Sycara. “Benefits of Learning in Negotiation.” Proceedings of AAAI-97.

Utility of Learning: Experimental Design

• The set of players N is comprised of one buyer and one supplier who make alternative proposals.

• For simplicity, the range of possible prices is from 0 to 100 units and this is public information

• The set of possible actions (proposed prices by either the buyer or the supplier) A equals to {0, 1, 2,…, 100}

• Reservation prices are private information.

• Each player's utility is linear to the final price ( a number between 0 and 100) accepted by both players

• Normalized Nash product as joint utility (the optimal joint utility

when full information is available is 0.25)

Average Performance of Three Experimental Configurations in Bazaar

• A non-learning agent makes decisions based solely on his own reservation price

• A learning agents makes decisions based on both the agent's own and the opponent's reservation price

Configuration JointUtility

Buyer’sUtility

Supplier’s Utility

# of ProposalsExchanged

Both Learn 0.22 0.49 .051 24

Neither Learn 0.18 0.49 0.51 34

Only Buyer Learns 0.15 0.59 0.41 28

Contingent Contracts and Options

• Most multi-agent systems don't handle uncertainty effectively

– rigid task delegation mechanism (contracts are binding rather than contingent

– no explicit modeling of stochastic events

– no explicit mechanism for controlling agent performance variability

• We are exploring the use of option pricing to address the above issues

Evaluation of Contingent vs Binding Contracts

• In the experiments, we only had two kinds of agents:

– Interface Agents: Accept queries from the user.

– Information Agents: Answer queries given by the Interface agents.

• In each cycle a new information agent with a load randomly distributed

between L and 0.9 appears with probability .

• When a new information agent comes up, interface agents have the option to abort the query on the old information agent and restart it on the new one.

• Interface agents can only switch a bounded number of queries to the new agent. This is indicated as Bound in the graphs.

• In the experiments the average delay in answering the queries was measured. This is indicated as Delay in the graphs.

Contingent Contracts

Contingent Contracting to Handle Unreliability of Information Sources

• Uncertain waiting time in response to queries

– random network congestion

– uncertain serve congestion/breakdown

The Query Restart Problem

• Agent A sends query to Agent B.

• Agent B can complete the query in time X, where

X = 1 with probability p.

X = c (c > 1) with probability 1 - p.

Expectation: EX = p + (1 - p) c

• If not done by time 1, should agent A abort and restart, or wait?

• Can restarting reduce expectation? The variance? Both?

• Does it help to repeatedly restart k times?

Strategy: restart just after time 1, if not done by then.

Let Xi = completion time of i'th query, i = 1,2.

X1, X2 are independent, identically distributed.

New completion time is Y:

New expectation

EY = p + (1 - p)(1 + E X2) (X1, X2 indep.)

= 1 + p (1 - p) + (1 - p) c

If (and only if) c > 1 + 1 / p, EY < X1 !

A Simple Scenario: Single restart

{ 1 if X1 = 1,

1 + X2 if X1 = c.

A Simple Scenario: k Restarts

Number of Restarts k

Applications

• Visitor Hoster (PLEIADES)

• Satellite Visibility (THALES)

• Portfolio Management (WARREN)

Characteristics of Retsina

• Open System

• Adaptivity at the agent and organization level provides robustness

• Service-based, economic coordination of agents

• Reusable and extensible domain-independent computational infrastructure

• Integrates information gathering and execution monitoring with decision making

• Framework for addressing uncertainty and strategic

interactions

Future of Software Agents• Agent-based software development is an emerging paradigm

• Agent society that parallels human society

• Implication of the emergence of agent society for human workplaces, institutions, and social relations

• Agent society as a unit of intelligence

• Opportunities and Challenges

– The WEB is a vast knowledge base presenting novel opportunities for AI

– Overall system (human and software agent) predictability

– Security, privacy, trust issues

– Integration of legacy systems

Overall Issues in Open MAS

• Overall agent organization

Agent Self-cloning

– Middle agents

Matchmaking and brokering

– Negotiation

Agent Self-cloning

– Middle agents

– Negotiation

Agent Self-cloning

– Middle agents

– Negotiation

Katia SycaraAAAI'971 James Bond and Michael Ovitz The Secret Life of Agents Katia Sycara The...

Documents

Intelligent Agents - Lecture 3 Katia Sycara The Robotics Institute katia@cs.cmu.edu softagents

Multirobot Coordination in USAR Katia Sycara The Robotics Institute katia@cs.cmu.edu softagents

1 Intelligent Agents - Security Katia Sycara The Robotics Institute email: katia@cs.cmu.edu softagents

Probabilistic plan recognition for proactive assistant agents · 2014-08-28 · Probabilistic plan recognition for proactive assistant agents Jean Oh Felipe Meneguzzi Katia Sycara

Predictive Assistance for Manufacturing Robotics · Predictive Assistance for Manufacturing Robotics Ruben D’Sa, Nilanjan Chakraborty, Katia Sycara Problem • Certain assemblies

The influence of Cultural Factors on Trust in Automation (FA9550-13-1-0129) PI: Katia Sycara (Carnegie Mellon University, USA) Co-PI: Michael Lewis (University

Towards an Understanding of the Impact of Autonomous Path Planning on Victim Search in USAR Paul Scerri, Prasanna Velagapudi, Katia Sycara, Huadong Wang,

Yuezhang Li, Katia Sycara, and Rahul Iyer · Object-sensitive DRL Yuezhang, Rahul and Katia 2.3 Object recognition Object recognition aims to nd and identify objects in an image or

Integrating Intelligent Assistants into Human Teams Katia Sycara The Robotics Institute Carnegie Mellon University Pittsburgh, PA 15213 (412) 268-8225

Intelligent Agents Katia Sycara katia@cs.cmu.edu The Robotics Institute Joseph Giampapa garof@cs.cmu.edu softagents

Predictive Indoor Navigation using Commercial … Indoor Navigation using Commercial Smart-phones Felipe Meneguzzi, Balajee Kannan , Katia Sycara, Chet Gnegyy, Evan Glasgowz, Piotr

Genetic Programming and the Predictive Power of Internet Message Traffic James D Thomas Katia Sycara

Semantic Web Services Tutorial - WSMO · Semantic Web Services Tutorial ISWC 2004, Hiroshima, Japan Massimo Paolucci Katia Sycara David Martin Sinuhe Arroyo Christoph Bussler

Matchmaking Among Heterogeneous Agents on the …...Matchmaking among Heterogeneous Agents on the Internet* Katia Sycara, Matthias Klusch, Seth Widoff The Robotics Institute, Carnegie

JEAN OH FELIPE MENEGUZZI KATIA SYCARA CARNEGIE MELLON UNIVERSITY TIMOYHY NORMAN UNIV. OF ABERDEEN Anticipatory information & planning agent

Decentralized prioritized planning in large multirobot teams Prasanna Velagapudi Paul Scerri Katia Sycara Carnegie Mellon University, Robotics Institute

Enterprise Management Network Architecture Distrubuted ...Enterprise Management Network Architecture Distributed Knowledge Base Support Michel Roboam, Mark S. Fox and Katia Sycara

Control of UAV Teams Paul Scerri & Katia Sycara Carnegie Mellon University Michael Lewis University of Pittsburgh P-LOCAAS Flight Test AC-130 Flank Support

1 2nd European Semantic Web Conference, Heraklion, Greece, May 2005 Carnegie Mellon University Katia Sycara Massimo Paolucci Christoph Bussler Emilia

AAMAS 2009, Budapest1 Analyzing the Performance of Randomized Information Sharing Prasanna Velagapudi, Katia Sycara and Paul Scerri Robotics Institute,