CS 380: ARTIFICIAL INTELLIGENCE RATIONAL AGENTSsanti/teaching/2013/CS380/...Solitaire Chess...

CS 380: ARTIFICIAL INTELLIGENCE RATIONAL AGENTS

9/25/2013 Santiago Ontañón santi@cs.drexel.edu https://www.cs.drexel.edu/~santi/teaching/2013/CS380/intro.html

• Do you think a machine can be made that replicates human intelligence?

Outline • What is an Agent? • Rationality • Agents and Environments • Agent Types

(these slides are adapted from Russel & Norvig’s official slides)

Outline • What is an Agent? • Rationality • Agents and Environments • Agent Types

(these slides are adapted from Russel & Norvig’s official slides)

What is an Agent? • Agent: autonomous entity which observes and acts upon

an environment and directs its activity towards achieving goals.

• Example agents: •  Humans •  Robots •  Software agents •  ATMs •  etc.

What is an Agent?

•  Theoretically:

•  In practice: the agent runs on a physical system (e.g. a computer) to produce

Agents and environments

?

agent

percepts

sensors

actionsenvironment

actuators

Agents include humans, robots, softbots, thermostats, etc.

The agent function maps from percept histories to actions:

f : P∗ → A

The agent program runs on the physical architecture to produce f

Chapter 2 4

f : P ⇤ ! A

f

What is an Agent?

•  Theoretically:

•  In practice: the agent runs on a physical system (e.g. a computer) to produce

Agents and environments

?

agent

percepts

sensors

actionsenvironment

actuators

Agents include humans, robots, softbots, thermostats, etc.

The agent function maps from percept histories to actions:

f : P∗ → A

The agent program runs on the physical architecture to produce f

Chapter 2 4

f : P ⇤ ! A

f

Maps sequences of percepts to actions

Example: Vacuum Cleaner

• Percepts: •  Location sensor and Dirt sensor

• Actions: •  Left, Right, Suck, Wait

Room A Room B

Vacuum-cleaner world

A B

Percepts: location and contents, e.g., [A, Dirty]

Actions: Left, Right, Suck, NoOp

Chapter 2 5

Example: Vacuum Cleaner

• Percepts: •  Location sensor and Dirt sensor

• Actions: •  Left, Right, Suck, Wait

Room A Room B

Vacuum-cleaner world

A B

Percepts: location and contents, e.g., [A, Dirty]

Actions: Left, Right, Suck, NoOp

Chapter 2 5

[A, Clean]

Example: Vacuum Cleaner A vacuum-cleaner agent

Percept sequence Action[A, Clean] Right[A, Dirty] Suck[B, Clean] Left[B, Dirty] Suck[A, Clean], [A, Clean] Right[A, Clean], [A, Dirty] Suck... ...

function Reflex-Vacuum-Agent( [location,status]) returns an action

if status = Dirty then return Suckelse if location = A then return Rightelse if location = B then return Left

What is the right function?Can it be implemented in a small agent program?

Chapter 2 6

Example: Vacuum Cleaner A vacuum-cleaner agent

Percept sequence Action[A, Clean] Right[A, Dirty] Suck[B, Clean] Left[B, Dirty] Suck[A, Clean], [A, Clean] Right[A, Clean], [A, Dirty] Suck... ...

function Reflex-Vacuum-Agent( [location,status]) returns an action

if status = Dirty then return Suckelse if location = A then return Rightelse if location = B then return Left

What is the right function?Can it be implemented in a small agent program?

Chapter 2 6

Two questions:

•  What is the function?

•  Can it be implemented as a computer program? (How?)

Outline • What is an Agent? • Rationality • Agents and Environments • Agent Types

(these slides are adapted from Russel & Norvig’s official slides)

Rationality • As we discussed last class, “intelligence” is an ill-defined

concept.

•  Let us assume a performance measure, for example: •  $1 per room cleaned •  $10 per room cleaned minus $1 per each movement •  etc.

• Rational agent: •  Chooses actions that maximize the expected value of the

performance measure given the current percept sequence.

Rationality • As we discussed last class, “intelligence” is an ill-defined

concept.

•  Let us assume a performance measure, for example: •  $1 per room cleaned •  $10 per room cleaned minus $1 per each movement •  etc.

• Rational agent: •  Chooses actions that maximize the expected value of the

performance measure given the current percept sequence.

Performance measure is given (external) to the agent.

Rationality •  Imagine two agents A and B whose performance measure

is defined by how many $ they make at the casino. •  A and B play one game of Black Jack •  A plays the move that theoretically has higher chances of winning •  B plays a different, more risky move •  B ends up winning

• Which agent is rational?

Rationality •  Imagine two agents A and B whose performance measure

is defined by how many $ they make at the casino. •  A and B play one game of Black Jack •  A plays the move that theoretically has higher chances of winning •  B plays a different, more risky move •  B ends up winning

• Which agent is rational? A is the rational agent •  Rationality does not imply clairvoyance •  Rationality does not imply omniscient •  Thus, rational does not mean successful

Outline • What is an Agent? • Rationality • Agents and Environments • Agent Types

(these slides are adapted from Russel & Norvig’s official slides)

How do we Design a Rational Agent? •  To design a rational agent, we first need to define its

environment, performance measure, etc.

• PEAS: •  Performance •  Environment •  Actions •  Sensors (percepts)

Example (informal): Automated Taxi • Performance:

• Environment:

• Actions:

• Sensors:

Example (informal): Automated Taxi • Performance: profit, safety, legality, …

• Environment: streets, pedestrians, cars, weather, traffic lights/signs, …

• Actions: steering, throttle, break, horn, display, …

• Sensors: GPS, video, sonar, keyboard, accelerometers, microphones, …

Example: Chess Agent • Performance:

• Environment: • Actions:

• Sensors:

Example: Chess Agent • Performance:

• Environment: Chess board, and opponent.

• Actions:

• Sensors:

Example: Chess Agent • Performance: Once reached a terminal state (draw, win):

-1 for losing, 0 for draw, 1 for winning. • Environment: Chess board, and opponent.

• Actions:

• Sensors:

Example: Chess Agent • Performance: Once reached a terminal state (draw, win):

-1 for losing, 0 for draw, 1 for winning. • Environment: Chess board, and opponent.

• Actions: Pairs of coordinates: (x1,y1) à (x2, y2). Specify the source piece and the target position (for castling, the king’s move is specified).

• Sensors:

Example: Chess Agent • Performance: Once reached a terminal state (draw, win):

-1 for losing, 0 for draw, 1 for winning. • Environment: Chess board, and opponent.

• Actions: Pairs of coordinates: (x1,y1) à (x2, y2). Specify the source piece and the target position (for castling, the king’s move is specified).

• Sensors: Board perceived as a 8x8 matrix. Each element in the matrix can take one of the following values: E, BP, WP, BB, WB, BK, WN, BR, WN, BQ, WK, BK, WK

Example: Chess Agent • Performance: Once reached a terminal state (draw, win):

-1 for losing, 0 for draw, 1 for winning. • Environment: Chess board, and opponent.

• Actions: Pairs of coordinates: (x1,y1) à (x2, y2). Specify the source piece and the target position (for castling, the king’s move is specified).

• Sensors: Board perceived as a 8x8 matrix. Each element in the matrix can take one of the following values: E, BP, WP, BB, WB, BK, WN, BR, WN, BQ, WK, BK, WK

The environment might have lots of features (piece size and material,

opponent’s facial expression, etc.). When designing a rational agent, we only need to provide sensors for the

information that is necessary to achieve the agent’s goals (maximize

performance measure)

Example: Chess Agent

Environment

WR WN WB WK WQ WB WN WR

WP WP WP WP WP WP WP WP

BP BP BP BP BP BP BP BP

BR BN BB BK BQ BB BN BR

Percept

Environment Types Observable Deterministic Sequential Static Discrete Single-Agent

Solitaire

Chess

Backgammon

Taxi

StarCraft

Real Life

Environment Types Observable Deterministic Sequential Static Discrete Single-Agent

Solitaire No Yes Yes Yes Yes Yes

Chess

Backgammon

Taxi

StarCraft

Real Life

Environment Types Observable Deterministic Sequential Static Discrete Single-Agent

Solitaire No Yes Yes Yes Yes Yes

Chess Yes Yes Yes Yes Yes No

Backgammon

Taxi

StarCraft

Real Life

Environment Types Observable Deterministic Sequential Static Discrete Single-Agent

Solitaire No Yes Yes Yes Yes Yes

Chess Yes Yes Yes Yes Yes No

Backgammon Yes No Yes Yes Yes No

Taxi

StarCraft

Real Life

Environment Types Observable Deterministic Sequential Static Discrete Single-Agent

Solitaire No Yes Yes Yes Yes Yes

Chess Yes Yes Yes Yes Yes No

Backgammon Yes No Yes Yes Yes No

Taxi No No Yes No No No

StarCraft

Real Life

Environment Types Observable Deterministic Sequential Static Discrete Single-Agent

Solitaire No Yes Yes Yes Yes Yes

Chess Yes Yes Yes Yes Yes No

Backgammon Yes No Yes Yes Yes No

Taxi No No Yes No No No

StarCraft No Almost Yes No Yes No

Real Life No No (?) Yes No No (?) No

Outline • What is an Agent? • Rationality • Agents and Environments • Agent Types

(these slides are adapted from Russel & Norvig’s official slides)

Agent Types • We will cover 4 basic types:

•  Reactive agents (reflex agents) •  State-based agents •  Goal-based agents •  Utility-based agents

•  They can all incorporate learning

Reactive Agents Simple reflex agents

AgentEnvironm

entSensors

What the worldis like now

What action Ishould do nowCondition−action rules

Actuators

Chapter 2 20

State-based Agents Reflex agents with state

Agent

Environment

Sensors

What action Ishould do now

State

How the world evolves

What my actions do

Condition−action rules

Actuators

What the worldis like now

Chapter 2 22

Goal-based Agents Goal-based agents

Agent

Environment

Sensors

What it will be like if I do action A

What action Ishould do now

State

How the world evolves

What my actions do

Goals

Actuators

What the worldis like now

Chapter 2 24

Utility-based Agents Utility-based agents

Agent

Environment

Sensors

What it will be like if I do action A

How happy I will be in such a state

What action Ishould do now

State

How the world evolves

What my actions do

Utility

Actuators

What the worldis like now

Chapter 2 25

Learning Agents Learning agentsPerformance standard

Agent

Environment

Sensors

Performance element

changes

knowledgelearning goals

Problem generator

feedback

Learning element

Critic

Actuators

Chapter 2 26

Summary • Agents:

•  Interact with environments via actuators (actions) and sensors (percepts)

• Rational agent: maximizes expected performance

• PEAS

• Environment types

• Basic agent architectures