Intelligent Agents

What is the basic framework we use to construct intelligent programs?

Agent structure• Intelligent programs do not run in isolation.

Most of them are custom build for a specific hardware set - the architecture.

• Intelligent agents are this combination of software and hardware, program and architecture.

• Ex: Deep Blue, Stanley, IRIS

Percept-think-action• All of our agents will have the same

framework: the percept-think-action cycle.– Agents gather information from the

environment in the form of a percept– Then they think about what they ought to do, by

which we mean they process the percept in some fashion

– They then decide upon an action to engage in.

Reflex agents

• Agents that only respond to the current percept.

• Generally composed of condition-action rules: If a certain condition is satisfied, implement the associated action.

• Can be modeled with an FA.

State-based agents• Respond only to current percept, but also

take into account the current state they are in.

• Ex: Roomba

• Percept-action rules also account for state.

• Also modeled with an FA.

Goal-based agents.• These agents identify some goal they want

to achieve, and make decisions about what to do based on the “best” way to achieve that goal.

• What defines “best” is the subject of a lot of debate and research.

Utility-based agents• Sometimes it isn’t sufficient to merely

achieve a goal (“The ends justify the means”). Sometimes it matters how we achieve that goal.

• Utility is a measurement of more subjective parameters. Roughly, positive utility corresponds to “pleasure” and negative utility to “pain.”

Learning agents

• Some agents need to achieve the same goals more than once.

• Learning agents evaluate the methods by which they achieve (or fail to achieve) their goals and modify their behavior to more efficiently or successfully achieve their goals.

Problem Solvers• Goal-based agents are generally used as

problem-solving agents.

• These are agents that are given a problem definition and a goal, and find some path of actions that leads to the goal from the problem definition.

Problem definitions• Four components:

– Start state: The starting conditions of the problem, including the status of all relevant elements.

– Goal state: What status all the relevant problem elements must be in for the problem to be considered “solved.”

– Operators: How can the relevant problem elements be manipulated to move from one state to another.

– Path cost: What is the cost associated with choosing one operator over another.

Breadth-first search

– First check the root node.

– Then check all the child nodes

– Then check all of their child nodes.

– etc….

Depth-first search– Check the root node.– Check the left child– Etc. all the way down

the left branch.– When a leaf is

reached, backtrack and check the right child.

– repeat

Relative advantages and disadvantages of each

• DFS:– May go on forever, if the depth is infinite

– Solved with depth-limited search, but that might cause us to not search deeply enough for a solution

– Low memory requirements (only need to remember current node)

• BFS:– Guaranteed to find the optimal (shortest) solution path.

– Not guaranteed to find it quickly– Struggles with problems with high branchiness

– High memory requirements.

Uninformed vs. Informed search• The previous searches were all what we call

uninformed searches. We only used the information in the problem definition to determine how to search the problem space.

• Wouldn’t it be nice to be able to use other information about the problem to guide and restrict our search?

• How can we take advantage of a heuristic measuring how “far” a given state is from the goal? h(n)

Greedy search• Always expands the node that is closest to

the goal.

f(n) = h(n)

• When might this cause problems?

A* search• Search in such a fashion that we take into

account not just the cost to reach the goal, but also the cost required to get to a state in the first place (g(n))

f(n) = g(n) + h(n)• Why is this an improvement over greedy

search?

• Still a memory hog!

Heuristics• Of course, the trick with any of these is

the nature of the heuristic function, h(n).• Ideally, we want to pick a heuristic that

a) Minimizes the branchiness of our search. Branchiness is a function of the number of states visited and the depth we had to go.

b) Is assured of finding as close to an optimal solution as possible, i.e., the solution path with the lowest cost

c) Finds such a solution as quickly as possible

The 8-puzzle

Searching efficiency• Is the type of search we’ve been discussing

always the most efficient way to solve a problem?

• What are some examples where this won’t work very well?

Solving problems as optimization

• What about problems with non-discrete states?

• Or problems with a very high branching factor, but a constrained range of values for h(n)?

• Solving these problems involves an optimization of the state space.

Rethinking h(n)

• Until now, we’ve thought of h(n) as a discrete value that can be attached to any particular state.

• What if we think of it as a true function, which we can plot in a 2- (or 3-, or 4-….) D space?

• In this case, the ideal solution becomes one that selects the optimal point on the curve - a global maximum if we’re trying to optimize a positive aspect, or a global minimum for a negative aspect.

• “Searching” becomes a matter of moving along this curve in a fashion designed to find the optimal solution point.

Hill climbing• Hill climbing is the greedy search of

optimization. Basically, we should always take a step in the direction with the steepest gradient.

• Why might this fail?

Simulated annealing• The primary problem with hill climbing is

the mild danger of getting stuck on a plateau, or the serious danger of getting stuck at a local minimum (cost).

• In simulated annealing, every time we stop at a minimum, we take a random step in some direction. We want to design the size of that step to get us out of local minima, but to stay inside of the global minimum.

Local beam search• A sort of “natural selection” based search.

– Start with k states chosen at random.– Generate all the successors of these k states. If

one is the goal, stop.– Otherwise, choose the k best states from the

successors.– Repeat.

• Why is this better?

Genetic Algorithms• Similar to local beam search (more properly the

stochastic beam search variant).– Start with k random states.– Evaluate each state using a fitness function– Using this function, generate a set of probabilities for

each of the k states.– Based on these probabilities, select a random choice of

k/2 pairs.• For each pair, choose a random crossover point and swap

corresponding parts of the pairs.• This leaves us with k new states.

– Finally, each element of the resulting state has the possibility for a mutation.