Upload
justin-mcdowell
View
219
Download
3
Embed Size (px)
Citation preview
peer-to-peer and agent-based computingBasic Theory of Agency
2
Plan of next two lectures• Motivation• States and actions• Runs• State transformer functions• Agents and systems• Purely reactive agents • Perception• Agents with state• Utilities• Achievement and maintenance tasks• Agent synthesis
3
An Abstract Agent Architecture• We need a way to “tie down” the concept of an
agent• We present an abstract architecture to formalise
the concepts of:– environmental state– actions and state transformations– agent decision making
4
Why not look at Java code?• Answer:
– A program is not the best way to communicate with humans about computations
• Code is verbose (i.e., lots of it!) and may contain a lot of unnecessary “housekeeping”:– Open sockets, parse XML, set variables/flags, loops,…
• Abstractions help us understand essential features
Let us assume that• The environment is part of a finite set E of discrete,
instantaneous states:
E = {e1, e2, …}
• Agents have a repertoire of actions available to them, which transform the state of the environment:
Ac = {1, 2, …}
5
States and actions (1)
6
States and actions (2)• Sample environments:
– Readings from a thermostat E = {-10,-9,…,0,1,…,39,40}
• Sample actions:– Turning on/off heating (or leaving it alone)
Ac = {on, off, nil}
• A run r of an agent in an environment is a sequence of interleaved states and actions:
r : e0 e1 e2 e3
… en
• Let:– R be the set of all possible finite runs (over E and Ac)– RAc be the subset of finite runs that end with an action– RE be the subset of finite runs that end with a state
7
Runs (1)
0 n -11 2 3
8
• A sample run, using previous environment & actions:
r : 10 20 30 25
… -1
• Sets:– R = {(10,off),(30,off,20),(-1,nil,10,on,12),…}– RAc ={(10,off),(10,off,5,on),…}– RE = {(30,off,20),(35,off,10,nil,-2),…}
Runs (2)
on nil off nil nil
• A state transformer function represents the behaviour of the environment:
: RAc (E )– Environments: history-dependent & non-deterministic– If (r )=, then there are no possible successor states to r
; i.e. the system has ended its run.• Formally, an environment consists of
– A set of environment states E– The initial state e0– A transformer function
Env = E,e0,
9
State transformer functions (1)
10
• Given RAc ={(10,off),(10,off,5,on),…}
and E = {-10,-9,…,0,1,…,39,40}
• We can define the following state transformer function
((10,off)) = {-10,…,10} ((10,off,5,on)) = {6,…,40}…
• A sample environmentEnv = {-10,…,0,…,40},0,
State transformer functions (2)
• An agent is a function mapping runs to actions:
Ag : RE Ac
– An agent decides which action to perform based on the history it has witnessed so far…
• Let AG = {Ag1, Ag2,… , Agn}
be the set of all agents in a multi-agent system.
11
Agents (1)
12
• Given RE = {(30,off,20),(35,off,10,off,-2),…}
we can define the following agent function:– Ag ((30,off,20)) = off– Ag ((35,off,10,nil,-2)) = on– …
• N.B.: there are compact ways to describe functions:– Ag ((…,on,x)) = off, if x 20– Ag ((…,on,x)) = nil, if x < 20– Ag ((…,off,x)) = on, if x < 20– Ag ((…,nil,x)) = on, if x < 20– Ag ((…,nil,x)) = nil, if x 20– …
Agents (2)
• A system comprises a pair agent/environment:
Ag, Env
– Any system has a set of possible runs associated with it• The set of runs of an agent in an environment is:
R (Ag, Env)
– Although the set of runs can be infinite, each run is finite– I.e., we do not consider (for the time being) infinite
runs…
13
Systems (1)
14
• A sample system:
Ag, {-10,…,0,…,40},0,
where– Ag (r ) = Ac (agent def’d as a function) (r ) = ({-10,…,0,…,40}) (state transformer function)
• A sample set of runs of an agent in an environment:
R (Ag, {-10,…,40},0, ) = {(0,on,20,nil,15),…}
Systems (2)
• A sequence(e0, 0, e1, 1, e2, …)
represents a run of an agent Ag in an environment Env = E, e0, if
15
Systems (3)
1. e0 is the initial state of Env
2. 0=Ag (e0) ; and
3. for i > 0, ei ((e0,0,…,i -1)) where i = Ag ((e0, 0, …, ei))
• Two agents Ag1 and Ag2 are behaviourally equivalent with respect to environment Env if, and only if,
R (Ag1, Env) = R (Ag2, Env) • Two agents Ag1 and Ag2 are behaviourally
equivalent if, and only if, they are behaviourally equivalent with respect to all environments Env.
16
Behavioural equivalence of agents
• Some agents decide what to do without reference to their history: – Their decision-making is based entirely on the present– I.e., there is no reference whatsoever to the past!
• Such agents are called purely reactive:
Ag : E Ac
• A thermostat is a purely reactive agent:
Ag (e ) = off if e 20Ag (e ) = on if e < 20
17
Purely Reactive Agents
18
• We can now introduce a perception system:
– see : agent’s ability to observe the environment– action : agent’s decision making function
Perception (1)
• The output of the see function is a percept:
see : E Per
which maps environmental states to percepts• action is now a function
action : Per* Ac
which maps sequences of percepts to actions• Agents are considered from now on as the pair
Ag = see, action
19
Perception (2)
20
• Sample see functions:– Robot with an infrared sensor– Sofware agent performing commands such as “ls” or
“finger” or retrieving a Web page– The output is stored in some data structure
• Sample action functions:– Move towards direction of source of heat– Delete all files with extension .jpg obtained via “ls” – Submit a Web form using the retrieved page
Perception (3)
21
Suggested Reading• An Introduction to Multi-Agent Systems, M. Wooldridge,
John Wiley & Sons, 2002. Chapter 2.