1

1 Module 4

Rule-based KS

Overview: Rules Inferencing Searching

  Literature:   chapter 2

2 25/11/08 Module 4

Overview

  Representing knowledge as rules   Types of inference

  induction, deduction, abduction,…   Reasoning in Knowledge Systems

  Forward Chaining   Backward Chaining

  Problem solving through search   Unguided Search: depth first, breadth first   Guided Search or Heuristics

2

3 25/11/08 Module 4

Knowledge rules

  (Production) rules   IF <antecedent> THEN <consequent>   IF <a1> AND <a2> AND … <an> THEN <c1>   IF <a1> OR <a2> OR … <an> THEN <c1>   IF <a1> THEN <c1> AND <c2> AND … <cn>

  Antecedents and consequents •  <object> <operator> <value> •  <predicate>

: age > 18 : father(john, mary)

4 25/11/08 Module 4

Types of rules

  Relation   IF drivers-license(mary) THEN age(mary) >= 18

  Recommendation   IF rainy(today) THEN take_umbrella(today)

  Directive   IF empty(fuel-tank,car) THEN refuel(car)

  Heuristic   IF lights-are-dim THEN battery-is-flat

3

5 25/11/08 Module 4

Reasoning

  How to combine rules to derive new knowledge?   Reasoning is how humans work with knowledge, facts and

problem solving strategies to draw conclusions.   deductive reasoning   inductive reasoning   abductive reasoning   analogical reasoning   common-sense reasoning   non-monotonic reasoning

6 25/11/08 Module 4

Deductive reasoning

  Idea: Deduce new information from logically related known information.   obtaining new facts from known facts

  Mathematically correct   If premises are true, the conclusions are guaranteed to be true

4

7 25/11/08 Module 4

Deduction

  uses facts with related rules to deduce new facts   rule: IF Ann is human THEN Ann is mortal   fact: Ann is human   new fact: Ann is mortal

  Modus Ponens   Given A is true and (A implies B) is true   Assert B is true

8 25/11/08 Module 4

Inductive reasoning

  Idea: To form general ideas or concepts from observing a set of facts or examples   generalization of known facts

  Not mathematically exact   conclusions can be false   takes a closed-world assumption

  Matches human reasoning

5

9 25/11/08 Module 4

Induction

  uses facts (premises) to draw conclusions   premise 1: Fred is mortal   premise 2: Joe is mortal   conclusion: all men are mortal   (only guaranteed to hold for set

{Joe, Fred})   inductive learning from data

  if X= { a, b, c,...} and   P is true for a, b and c   then P is true for all X

  type of learning used by neural networks, rule induction and decision trees

10 25/11/08 Module 4

Abductive reasoning

  Idea: explain effects in terms of their causes   allows for plausible inference, i.e. the conclusion logically

follows from the evidence but still may be incorrect.

6

11 25/11/08 Module 4

Abduction

  Uses rules to explain effects in terms of their causes   rule: if it is raining then ground is wet   fact: ground is wet   conclusion: likely it is raining (or: is it raining?)

  Contrary to causal reasoning   It could be the case that its not raining but someone is

watering the grass!   If B is true and if A implies B   then is A true?

12 25/11/08 Module 4

Analogical reasoning

  Idea: uses past cases to provide analogies with future cases

  uses object similarities and differences to draw conclusions   fact: Tiger is a big cat, eats meat, lives in Asia   new fact: Lion is similar to Tiger   conclusion: Lion is a big cat, eats meat, lives in Asia

  CBR uses this technique

7

13 25/11/08 Module 4

Common-sense reasoning

  Idea: “short-cut” to limit the search space to certain areas or to immediately propose a solution.   Relies more on judgement than precise logic   no guarantee that solution is optimum or even correct!

14 25/11/08 Module 4

Common-sense reasoning

  Uses heuristics to draw conclusions/ propose hypothesis   (most likely reason for too hot is faulty thermostat)   Rule: if the motor temperature is very hot then thermostat

maybe faulty   Hypothesis: thermostat faulty

  Humans use past experiences to quickly solve new problems.   e.g. in chess, humans apply heuristics to prune the HUGE

search space presented by the possible moves.

  Shallow vs. deep reasoning

8

15 25/11/08 Module 4

Non-monotonic reasoning

  Idea: allow to revise assumptions in the light of new information   adapt to changing external conditions   adjust the chain of dependent events accordingly

  Monotonic reasoning   knowledge can only be added   most reasoning strategies assume that the axioms and the

conclusions formed from axioms remain fixed

16 25/11/08 Module 4

Non-monotonic logic

  Fact: Tweety is a bird   Conclusion: Tweety flies (commonsense deduction)

  New fact: Tweety is an ostrich   New conclusion: Tweety does not fly

9

17 25/11/08 Module 4

Strategy Overview

18 25/11/08 Module 4

Applying Rules

  How to deal with more than one rule? And many facts?   Reasoning in Knowledge Systems

  Forward Chaining   Backward Chaining

10

19 25/11/08 Module 4

Forward vs. backward?

 Forward  Take each initial

state (fact) and try to reach goal

 Backward  Start from goal and

try to find a possible initial state (fact)

How can the piglets get to mommy?

goal

fact

fact fact

fact

20 25/11/08 Module 4

Forward Chaining

  Data driven reasoning   bottom up   Search from facts to valid conclusions

  Given database of true facts   Apply all rules that match facts in database   Add conclusions to database   Repeat until a goal is reached, OR repeat until no new facts

added

11

21 25/11/08 Module 4

Example

R1: IF hot AND smoky THEN fire R2: IF alarm-beeps THEN smoky R3: IF fire THEN switch-on-sprinkler

•  alarm-beeps •  hot

Facts Rules

•  smoky First cycle: R2 holds

•  fire Second cycle: R1 holds

•  switch-sprinkler Third cycle: R3 holds

Action

22 25/11/08 Module 4

Forward Chaining: Conflict Resolution

  Order in which rules fire depends on facts in working memory, not order of rules.

  When more than one rule applies, possible heuristics are:   apply more recent facts first: recency   prefer rules with more specific conditions   Alternative conflict resolution strategies

•  (e.g., allow user to specify preference on rules)

12

23 25/11/08 Module 4

Backward Chaining

  Goal driven reasoning   top down   Search from hypothesis and finds supporting facts

  To prove goal G:   If G is in the initial facts, it is proven.   Otherwise, find a rule which can be used to conclude G, and

try to prove each of that rule’s conditions.

24 25/11/08 Module 4

Example

R1: IF hot AND smoky THEN fire R2: IF alarm-beeps THEN smoky R3: IF fire THEN switch-sprinkler

Rules

Should I switch the sprinklers on?

Hypothesis

IF fire Use R3

IF hot IF smoky Use R1

IF alarm-beeps Use R2

Evidence

•  alarm-beeps •  hot

Facts Yes!

13

25 25/11/08 Module 4

Backward Chaining: Conflict resolution

  Order in which rules fire depends on order of rules.   When more than one rule applies:

  Try both. •  Either might be used to validly prove the hypothesis.

  This is a search problem. •  How to systematically go through all possibilities.

26 25/11/08 Module 4

Application of Rule Based Systems

  Wide use in expert systems   Backward chaining: Diagnosis systems

•  start with set of hypotheses and try to prove each one, asking additional questions of user when fact is unknown.

  Forward chaining: design/configuration systems •  see what can be done with available components. •  exclude hypothesis, before asserting more facts (too expensive,

painful)

14

27 25/11/08 Module 4

Comparison

  Backward chaining   From hypotheses to

relevant facts   Good when:

•  Limited number of (clear) hypotheses

•  Determining truth of facts is expensive

•  Large number of possible facts, mostly irrelevant

•  when you have specific goals in mind

  Forward chaining   From facts to valid

conclusions   Good when

•  Less clear hypothesis •  Very large number of

possible conclusions •  True facts known at

start, or inexpensive •  maximize the use of new

data

28 25/11/08 Module 4

Search in State Spaces

  Many problems in AI can be mapped onto searches in particular state spaces.   especially useful if the system (world) can be defined as

having a finite number of states, including an initial state and one or more goal states.

  finite number of actions to take,   well-defined state transitions only depending on current state

and current action.

15

29 25/11/08 Module 4

Search Techniques in KS

  Search space   nodes (states) and edges (actions)

  Blind search   not guided by (previous) experience

•  Depth first or Breadth first   combinatorial explosion

  Informed search (heuristic)   guided by knowledge about the problem   evaluation of next choice (meta reasoning)

30 25/11/08 Module 4

Search Problem: Examples

  Chess: search set of possible moves   Looking for one which best improves position

  Route planning: search set of paths   Looking for one which will minimize distance

  Theorem proving: Search sets of reasoning steps   Looking for a reasoning progression which proves theorem

  Machine learning: Search set of concepts   Looking for a concept which achieves target categorisation

16

31 25/11/08 Module 4

Search example

  Search space: Road network   States: being in a city   Actions: moving along a road

from one city to the next   Goal: Target city (G)

  Strategy: is a function for expanding and evaluating states according to the state space graph

A

B

C G

D E

start

goal

F 11

21 14

21

15

27

10

node d

Arc de

COST: cost(de) = 14 PATH: (ab,bd), or (a,b,d)

32 25/11/08 Module 4

General Search Algorithm

initialize the search tree using the initial state of problem loop do

if no candidates for expansion then return fail else choose node for expansion according to strategy if the node contains a goal state

then return the solution else expand the node and add the results to search tree

end

17

33 25/11/08 Module 4

Issues in search strategies

  Completeness   Is the strategy guaranteed to find solutions (if any)

  Time complexity   How long does it take in the worst case relative to the size of

the problem   Space complexity

  How much memory does it require in the worst case relative to the size of the problem

  Optimality   Is the solution found the best (shortest/cheapest)

34 25/11/08 Module 4

Blind Search Strategies

  Breadth-first   nodes are examined level by level   begins at initial state, continues searching all nodes at each

level before moving into deeper level   Depth-first

  nodes are examined depth by depth   begins at initial state, continues searching at next lower level,

if dead end then backtracks

18

35 25/11/08 Module 4

Breath first search

Search space: Road network

Search tree

A

B

C G

D E

start

goal

F

Breadth first:

A

B C

F D

E

G

G

A

B C

F D G

A B C D F G  Complete  Optimal if step cost is 1  High use time and space

36 25/11/08 Module 4

A

B C

F D

E

G

G

Depth first search

Search space: Road network

Search tree

A

B

C G

D E

start

goal

F

A

B

F D

E G

Depth first:

E G

A B D E B F G   Complete for finite search tree   Not optimal   Low use space and time for ‘short’ trees

19

37 25/11/08 Module 4

Comparison of Strategies

  Breadth-first is complete and optimal, but has high space complexity

  Depth-first is space efficient, but neither complete nor optimal

  However, neither uses prior information to guide search

38 25/11/08 Module 4

Heuristic Search

  Heuristic (Greek): ‘to find’ or ‘to discover’   In AI, heuristic is most often used as an adjective,

referring to any technique that improves the average case performance on a problem solving task, but does not necessarily improve the worst case performance.

Heuristic = informed search

20

39 25/11/08 Module 4

Heuristics and algorithms

  Algorithm is a finite set of well-defined instructions for accomplishing some task   Goals of algorithms

•  provably good run times •  provably good or optimal solution quality.

  Heuristic gives up one or both of these goals;   pretty good solutions, but no proof the solutions could not get

arbitrarily bad;   reasonably quickly, but there is no argument that this will

always be the case.

40 25/11/08 Module 4

Heuristic Search

  Heuristics orient the search along promising paths   The time spent computing heuristics must be recovered by

a better search   A reasonable heuristic should be:

  Reasonably accurate   Easy to compute

  Deciding which node to expand is sometimes called meta-reasoning

  Heuristics are not always computational and may involve large amount of knowledge

21

41 25/11/08 Module 4

A* Search

  A* search combines   evaluation function f(n) = g(n) + h(n) where   g(n): actual cost of path from start node n0 to n   h(n): estimated cost of path from node n to goal

  Heuristic h(n) is   admissible = it never overestimates the actual cost of the best

solution. For example, straight-line calculation from A to B.   optimistic = they operate by assuming that the cost of solving

the problem is less than it actually is.   The key to all informed search techniques is to select an

appropriate heuristic function

42 25/11/08 Module 4

A* search example

Search space: Road network

Search tree

A

B

C G

D E

start

goal

F

A

B C

F D

G H(n) (straight line distance to goal)

H(A) = 9 H(B) = 4 H(C) = 7 H(D) = 12 H(F) = 3

5

4

3

2

3

8

F(A) = 0+9

F(C) = 4+7 F(B) = 5+4

F(D) = 5+2+12 F(F) = 5+3+3

A

B

F

G

Actual cost of chosen path is 5+3+3 = 11

22

43 25/11/08 Module 4

Advantages of rule-based systems

  Simplicity   Rules are closely related to human reasoning

  Uniformity   All rules have the same structure

  Modularity   Separation of knowledge (facts) and reasoning   ‘Blocks’ of rules often independent of each other

  Explanation   Transparent facilities -> rules describe ‘why’

  Uncertainty?   RBS can be extended to handle uncertain knowledge ->

lecture 6

44 25/11/08 Module 4

Disadvantages of rule-based systems

  Opaque   Combinations of rules are difficult to see   Link between individual rules and overall strategy   Lack of hierarchy

  Ineffective search   Exhaustive search is slow if large rule set   Apply search strategies

  Inability to learn   Rules cannot modify themselves   Know when to ‘break the rule’

23

45 25/11/08 Module 4

Practicum: 27 November Decision tables: modeling knowledge

Next lecture: 2 December

Classification Ontologies