1

CS 301 - Lecture 27 Complexity

Fall 2008

Review •  Languages and Grammars

–  Alphabets, strings, languages •  Regular Languages

–  Deterministic Finite and Nondeterministic Automata –  Equivalence of NFA and DFA –  Regular Expressions and Regular Grammars –  Properties of Regular Languages –  Languages that are not regular and the pumping lemma

•  Context Free Languages –  Context Free Grammars –  Derivations: leftmost, rightmost and derivation trees –  Parsing, Ambiguity, Simplifications and Normal Forms –  Nondeterministic Pushdown Automata –  Pushdown Automata and Context Free Grammars –  Deterministic Pushdown Automata –  Pumping Lemma for context free grammars –  Properties of Context Free Grammars

•  Turing Machines –  Definition, Accepting Languages, and Computing Functions –  Combining Turing Machines and Turing’s Thesis –  Turing Machine Variations, Universal Turing Machine, and Linear Bounded Automata –  Recursive and Recursively Enumerable Languages, Unrestricted Grammars –  Context Sensitive Grammars and the Chomsky Hierarchy

•  Computational Limits and Complexity –  Computability and Decidability –  Complexity

Computational Complexity

Example: Given a set of n numbers x1, x2, …, xn is number y in the set? Algorithm to answer this:

• If x = x1, stop and answer yes • If x = x2, stop and answer yes • If x = x3, stop and answer yes • ….. • If x = xn, stop and answer yes • Answer no. Time Required depends on n

worst case requires n compares Let’s formalize this idea…

2

Time Complexity:

Space Complexity:

The number of steps during a computation

Space used during a computation

Time Complexity • Model Computation Using Turing machine

• We count the number of steps until a string is accepted

• We use the O(n) notation

Example: }0:{ ≥= nbaL nn

Algorithm to accept a string : w

• Use a two-tape Turing machine

• Copy the on the second tape

• Compare the and

a

a b

|)(|wO• Copy the on the second tape

• Compare the and

a

a b

Time needed:

|)(|wO

Total time: |)(|wO

}0:{ ≥= nbaL nn

3

For string of length

time needed for acceptance:

}0:{ ≥= nbaL nn

n

)(nO

Language class: )(nDTIME

A Deterministic Turing Machine accepts each string of length in time

)(nDTIME

1L2L 3L

)(nOn

)(nDTIME

}0:{ ≥nba nn

In a similar way we define the class

))(( nTDTIME

)(nTfor any time function:

Examples: ),...(),( 32 nDTIMEnDTIME

4

Example: The membership problem for context free languages

}grammar by generated is :{ GwwL =

)( 3nDTIMEL∈ (CYK - algorithm)

Polynomial time

Theorem:

)( knDTIME

)( 1+knDTIME

)()( 1 kk nDTIMEnDTIME ⊂+

Polynomial time algorithms: )( knDTIME

Represent tractable algorithms: For small we can compute the result fast

k

Example: The Satisfiability Problem

Boolean expressions in Conjunctive Normal Form:

ktttt ∧∧∧∧ 321

pi xxxxt ∨∨∨∨= 321

Variables

Question: is expression satisfiable?

5

)()( 3121 xxxx ∨∧∨

Satisfiable? Can you find values for x1, x2, and x3 so that

1)()( 3121 =∨∧∨ xxxx

Example: )()( 3121 xxxx ∨∧∨

Satisfiable: 1,1,0 321 === xxx

1)()( 3121 =∨∧∨ xxxx

Example:

2121 )( xxxx ∧∧∨Example:

Satisfiable? Can you find values for x1, x2, and x3 so that

2121 )( xxxx ∧∧∨

Not satisfiable

Example:

6

e}satisfiabl is expression :{ wwL =

For variables, combinations to try n

Algorithm: search exhaustively all the possible binary values of the variables

Exponential, can we do better?

Exponential time algorithms: )2( nDTIME

Represent intractable algorithms: Some problem instances may take centuries to solve

Example: The satisfiability problem

Non-Deterministic algorithm:

• Guess an assignment of the variables

e}satisfiabl is expression :{ wwL =

• Check if this is a satisfying assignment

Time for variables: n

)(nO

e}satisfiabl is expression :{ wwL =

Total time:

• Guess an assignment of the variables

• Check if this is a satisfying assignment )(nO

)(nO

7

Non-Determinism Language class: )(nNTIME

A Non-Deterministic Turing Machine accepts each string of length in time

)(nNTIME

1L2L 3L

)(nOn

Example: }{wwL =

Non-Deterministic Algorithm to accept a string : ww

• Use a two-tape Turing machine

• Guess the middle of the string and copy on the second tape

• Compare the two tapes

w

|)(|wO

Time needed:

|)(|wO

Total time: |)(|wO

}{wwL =

• Use a two-tape Turing machine

• Guess the middle of the string and copy on the second tape

• Compare the two tapes

w

)(nNTIME

}{wwL =

8

In a similar way we define the class

))(( nTNTIME

)(nTfor any time function:

Examples: ),...(),( 32 nNTIMEnNTIME

Non-Deterministic Polynomial time algorithms:

)( knNTIMEL∈

Selecting the “Right” Machine • Model Computation Using Turing machine

• But the choice of machine seems to matter a great deal! • Two tapes machines might take fewer steps than one tape machines • Non-Deterministic machines might take fewer steps than Deterministic ones

•  Is there a “right” choice to make?

Selecting the “Right” Machine • If a two-tape machine takes steps, a one tape machine can simulate this in

• If a non-deterministic machine takes steps, a deterministic one can simulate this in

So do we need classes for each machine type??

9

What’s Next •  Read

–  Linz Chapter 1,2.1, 2.2, 2.3, (skip 2.4), 3, 4, 5, 6.1, 6.2, (skip 6.3), 7.1, 7.2, 7.3, (skip 7.4), 8, 9, 10, 11, 12.1, 12.2, (skip 12.3, 12.4, 12.5, 13), 14.1, 14.2, and 14.3

–  JFLAP Chapter 1, 2.1, (skip 2.2), 3, 4, 5, 6, 7, (skip 8), 9, (skip 10), 11 •  Next Lecture Topics from 14.4 - 14.7

–  P, NP, Reductions, and NP-complete problems •  Quiz 4 Wednesday 12/3

–  Closed book, but you may bring one sheet of 8.5 x 11 inch paper with any notes you like.

•  Homework –  Homework 13 Due Today –  Homework 14 Due Thursday –  Homework 15 Due Tuesday 12/9 –  Homework 16 = study for the final!!