Intelligent AgentsLecture # 2&3
Objectives
2
Agents and environmentsRationalityPEAS (Performance measure, Environment,
Actuators, Sensors)Environment typesAgent types
Agents
3
An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators
Human agent: eyes, ears, and other organs for sensors; legs, mouth, and other body parts for actuators
Robotic agent: cameras and infrared range finders for sensors;various motors for actuators
Software agents or softbots that have some functions as sensors and some functions as actuators. Askjeeves.com is an example of a softbot.
Glossary
4
Percept– agents perceptual inputs
Percept sequence– History of everything the agent has perceived
Agent function– Describes agent’s behaviour – maps any percept to an action
Agent program– Implements agent function
Agents and environments
5
The agent function maps from percept histories to actions:
[f: P* A]The agent program runs on the physical
architecture to produce f agent = architecture + program
Example: Vacuum-cleaner Agent
6
• Environment: square A and B• Percepts: [location and content] e.g. [A, Dirty]• Actions: left, right, suck, and no-op• A simple agent function may be “if the current square is dirty, then suck or move to other square..”
Example: Vacuum-cleaner Agent
Agent Program !!!function REFLEX-VACUUM-
AGENT( [location,status]) returns actionif status = Dirty then return Suckelse if location = A then return Rightelse if location = B then return Left
7
Intelligent Agents
8
The fundamental faculties of intelligence are Acting Sensing Understanding, reasoning, learning
An Intelligent Agent must sense, must act, must be autonomous (to some extent).
It also must be rational.
AI is about building rational agents.
Rational Agent
9
An agent should strive to "do the right thing", based on what it can perceive and the actions it can perform.
What is the right thing?– Causes the agent to be most successful
– Rationality is not the same as Perfection – Rationality maximizes “Expected Performance”– Perfection maximizes “Actual Performance”
How to evaluate agent’s success? Performance Measure evaluates How.
• Performance measure – is a criteria to measure an agent’s behavior
e.g., performance measure of a vacuum-cleaner agent could be amount of dirt cleaned up, amount of time taken, amount of electricity consumed, amount of noise generated, etc
Performance measure according to what is wanted in the environment instead of how the agents should behave.
Rationality – Cont’d
10
What is rational - depends on four things:– The performance measure– The agent’s prior knowledge of the environment– The actions the agent can perform– The agent’s percept sequence to date
A rational agent is:For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and what ever built in knowledge the agent has.
Rational Agents – Cont’d
11
Rationality is distinct from omniscience (all-knowing with infinite knowledge)
Agents can perform actions in order to modify future percepts so as to obtain useful information (information gathering, exploration – an important part of rationality)
An agent is autonomous if its behavior is determined by its own experience (with ability to learn and adapt) – a rational agent should be autonomous ….!
Rational ⇒ exploration, learning, autonomy
Is our vacuum-cleaner agent rational?
12
Keeping in view four aspects to measure rationality:
The performance measure awards one point for each clean square at each time step over 10000 time steps.
The geography of the environment is known.- Clean squares stay clean and sucking cleansthe current square.
Only actions are left, right, suck, and NoOpThe agent correctly perceives its location and
whether that location contains dirt.
What’s your Answer -- Yes/No
Building Rational AgentsPEAS Description to Specify Task Environments
13
To design a rational agent we need to specify a task environment
a problem specification for which the agent is a solution
PEAS: to specify a task environmentP:Performance MeasureE: EnvironmentA: ActuatorsS: Sensors
PEAS: Specifying an automatedtaxi driverPerformance measure: ?Environment: ?Actuators: ?Sensors:?
14
PEAS: Specifying an automatedtaxi driverPerformance measure: safety, speed, legal, comfortable,
maximize profitsEnvironment: ?Actuators: ?Sensors: ?
15
PEAS: Specifying an automatedtaxi driverPerformance measure:safe, fast, legal, comfortable, maximize
profitsEnvironment: roads, other traffic, pedestrians, customersActuators: ?Sensors: ?
16
PEAS: Specifying an automatedtaxi driverPerformance measure: safe, fast, legal, comfortable, maximize
profitsEnvironment: roads, other traffic, pedestrians, customersActuators:steering, accelerator, brake, signal, hornSensors:?
17
PEAS: Specifying an automatedtaxi driverPerformance measure:safe, fast, legal, comfortable, maximize
profitsEnvironment:roads, other traffic, pedestrians, customersActuators:steering, accelerator, brake, signal, hornSensors:cameras, sonar, speedometer, GPS
18
PEAS: Specifying a part picking robot
19
Performance measure: Percentage of parts in correct bins
Environment: Conveyor belt with parts, binsActuators: Jointed arm and handSensors: Camera, joint angle sensors
PEAS: Specifying an interactive English tutor
20
Performance measure: Maximize student's score on test
Environment: Set of studentsActuators: Screen display (exercises,
suggestions, corrections)Sensors: Keyboard
Environment types/Properties of Task Environments
21
Fully observable (vs. partially observable): An agent's sensors give it access to the complete state of the environment at each point in time.
e.g a taxi agent doesn’t has sensor to see what other drivers are doing/thinking ….
Deterministic (vs. stochastic): The next state of the environment is completely determined by the current state and the action executed by the agent. (If the environment is deterministic except for the actions of other agents, then the environment is strategic)
Vacuum worlds is Deterministic while Taxi Driving is Stochastic – as one can exactly predict the behaviour of traffic
Environment types
22
Episodic (vs. sequential): The agent's experience is divided into atomic "episodes" (each episode consists of the agent perceiving and then performing a single action), and the choice of action in each episode depends only on the episode itself.
E.g. an agent sorting defective parts in an assembly line is episodic while a taxi driving agent or a chess playing agent are sequential ….
Static (vs. dynamic): The environment is unchanged while an agent is deliberating. (The environment is semidynamic if the environment
itself does not change with the passage of time but the agent's performance score does)Taxi Driving is Dynamic, Crossword Puzzle solver is static…
Environment types – cont’d
23
Discrete (vs. continuous): A limited number of distinct, clearly defined percepts and actions.
e.g. chess game has finite number of statesTaxi Driving is continuous-state and continuous-time
problem …
Single agent (vs. multiagent): An agent operating by itself in an environment.
An agent solving a crossword puzzle is in a single agent environment
Agent in chess playing is in two-agent environment
Examples
24
• The environment type largely determines the agent design• The real world is (of course) partially observable, stochastic, sequential, dynamic, continuous, multi-agent
Agent types
25
Four basic types in order of increasing generality:
Simple reflex agentsModel-based reflex agentsGoal-based agentsUtility-based agents
– All of these can be turned into learning agents
Simple reflex agents
26
Information comes from sensors - percepts Changes the agents current state of the
world These agents select actions on the basis of current percept
Triggers actions through the effectors The condition-action rules allow the agent to make
connection from percept to actionsif car-in-front-is-braking then brakeif light-becomes-green then move-forwardif intersection-has-stop-sign then stopIf dirty then suck
Simple reflex agentsCharacteristics
Such agents have limited intelligence…Efficient No internal representation for reasoning,
inference. No strategic planning, learning. Are not good for multiple, opposing, goals. Works only if correct decision can be
made on basis of current percept.
27
Simple reflex agentsFunction SIMPLE-REFLEX-AGENT(percept)
returns an actionstatic: rules, a set of condition-action rulesstate←INTERPRET-INPUT(percept)rule←RULE-MATCH(state, rule)action←RULE-ACTION[rule]return action
Will only work if the environment is fully observable otherwise infinite loops may occur.
28
Simple reflex agents
29
Model based reflex agents
30
These agents keep track of part of world it can’t see …..
To tackle partially observable environments.To update its state the agent needs two kinds of
knowledge: 1. how the world evolves independently from the agent; Ex: an overtaking car gets closer with time. 2. how the world is affected by the agent’s actions. Ex: if I turn left, what was to my right is now behind Me.Thus a model based agent works as follows:
information comes from sensors - percepts based on this, the agent changes the current state of the world based on state of the world and knowledge (memory), it triggers
actions through the effectors E.g.
Know about other car location in overtaking scenario for taxi driving agent When agent turns the steering clockwise, car turns right ….
Model based reflex agentsFunction REFLEX-AGENT-WITH-
STATE(percept) returns an actionstatic: rules, a set of condition-action rulesstate, a description of the current world state action, the most recent action.state←UPDATE-STATE(state, action, percept)rule←RULE-MATCH(state, rule)action←RULE-ACTION[rule]return action
31
Model-based reflex agents
32
Goal based agents
33
Current state of environments is not always enough ….e.g at a road junction, it can turn left, right or go straight ..
Correct decision in such cases depends on where taxi is trying to get to ….
Major difference: future is taken into accountCombining goal information with the knowledge of its
actions, the agent can choose those actions that will achieve the goal.
Goal-based Agents are much more flexible in responding to a changing environment; accepting different goals.Such agents work as follows:
information comes from sensors - percepts changes the agents current state of the world based on state of the world and knowledge (memory) and
goals/intentions, it chooses actions and does them through the effectors.
Goal-based agents
34
Utility based agents
35
Goals alone are not always enough to generate quality behaviourseg different action sequences can take the taxi
agent to destination (and achieving thereby the “goal”) – but some may be quicker, safer, economical etc ..
A general performance measure is required to compare different world states
A utility function maps a state (or sequence of states) to a real number to take rational decisions and to specify tradeoffs when:
goals are conflicting – like speed and safetyThere are several goals and none of which can be
achieved with certainty …
Utility-based agents
36
Learning agents
37
A learning agent can be divided into four conceptual components:Learning Element
Responsible for making improvements in Performance elementuses feedback from Critic to determine how performance
element should be modified to do better….Performance Element
Responsible for taking external actionsselecting actions based on percepts
CriticTells the learning element how well agent is doing w.r.t. to
fixed performance standards…Problem Generator
Responsible for suggesting actions that will lead to improved and informative experiences.
….
38
For a taxi driver agent:Performance element consists of collection of
knowledge and procedures for selecting driving actions
The Critic observes the world and passes the information to performance element – e.g. reaction/response of other drivers when the agent takes quick left turn from top lane …!!
Learning element then can formulate a rule to mark a “bad action”…
Problem generator identifies certain areas of behaviour improvement and suggest experiments – trying brakes on different road conditions etc…
The Learning element can make changes in “knowledge” components – by observing pairs of successive states allow an agent to learn (learn from what happens when strong brake is applied on a wet road …)
Learning agents
39
Summary
40
An agent is something that perceives and acts in an environment. The agent function specifies the action taken by the agent in response to any percept sequence.• The performance measure evaluates the behaviour of the agent in the environment. A rational agent acts to maximise the expected value of the performance measure.• Task environments can be fully or partially observable, deterministic or stochastic, episodic or sequential, static or dynamic, discrete or continuous, and single-agent or multiagent
Summary
41
Simplex reflex agents respond directly to percepts, whereas model-based reflex agents maintain internal state to track aspects of the world that are not evident in the current percept.
Goal-based agents act to achieve their goals, and utility-based agents try to maximize their own expected “happiness”.
All agents can improve their performance through learning