15-453 - Formal Languages, Automata, and Computability Spring 2015, Tuesday/Thursday 12:00-1:20 PM, Baker Hall 136A

cvFirst class I taught at CMU.

This class will be a small variation of the Of Lenore Blum’s Spring 2014. (quizes)

INSTRUCTORS & TAs

Owen Kahn

Steven Rudich

Andy Smith

Asa Frank

Office Hours

Prof. Rudich: Tuesday, 1:30-2:30 PM GHC 7219

Asa: Monday, 6:30-8:00 PM

Owen: Wednesday, 7:00-9:00 PM

Andrew: TBD

(Locations TBD)

http://www.contrib.andrew.cmu.edu/~okahn/flac-s15/index.html

15-453 - Formal Languages, Automata, and Computability Spring 2015, Tuesday/Thursday 12:00-1:20 PM, Baker Hall 136A

Project 15%

Final 20%

Midterm II 15%

Midterm I 15%

Quizzes 5%

Participation 5%

Homework 25%

HOMEWORK

Homework will be assigned every Thursday and will be due one week later at the beginning of class. Late homework will be accepted only under exceptional circumstances.

All assignments must be typeset (exceptions allowed for diagrams). Each problem should be done on a separate page.

HOMEWORK

Homework will be assigned every Thursday and will be due one week later at the beginning of class. Late homework will be accepted only under exceptional circumstances.

All assignments must be typeset (exceptions allowed for diagrams). Each problem should be done on a separate page.

You must list your collaborators (including yourself) and all references in every homework assignment in a References section at the end.

COURSE PROJECT

Meet with an instructor/TA once a month

Choose a (unique) topic

Learn about your topic

Write progress reports (Feb 5, March 24)

Prepare an 8-minute presentation (April 21-30)

Final Report (April 30)

COURSE PROJECTSuggested places to look for project topics

Any paper that has appeared in the proceedings of FOCS or STOC in the last 5 years. FOCS (Foundations of Computer Science) and STOC (Symposium on the Theory of Computing) are the two major conferences of general computer science theory. The proceedings of both conferences are available at the E&S library or electronically.

· Electronic version of the proceedings of STOC · Electronic version of the proceedings of FOCS

• What's New]

This class is about mathematical models of computation

Course Outline

PART 1Automata and Languages: finite automata, regular languages, pushdown automata, context-free languages, pumping lemmas.

PART 2Computability Theory: Turing Machines, decidability, reducibility, the arithmetic hierarchy, the recursion theorem, the Post correspondence problem.

PART 3Complexity Theory and Applications: time complexity, classes P and NP, NP-completeness, space complexity, PSPACE, PSPACE-completeness, the polynomial hierarchy, randomized complexity, classes RP and BPP.

PART 1Automata and Languages: finite automata, regular languages, pushdown automata, context-free languages, pumping lemmas.

PART 2Computability Theory: Turing Machines, decidability, reducibility, the arithmetic hierarchy, the recursion theorem, the Post correspondence problem.

PART 3Complexity Theory and Applications: time complexity, classes P and NP, NP-completeness, space complexity, PSPACE, PSPACE-completeness, the polynomial hierarchy, randomized complexity, classes RP and BPP.

Mathematical Models of Computation

(predated computers as we know them)1940’s-50’s (neurophysiology, linguistics)

1930’s-40’s (logic, decidability)

1960’s-70’s (computers)

This class will emphasize PROOFS

A good proof should be:

Easy to understand

Correct

Suppose A ⊆ {1, 2, …, 2n}

TRUE or FALSE: There are always two numbers in A such that one divides the other

with |A| = n+1

TRUE

THE PIGEONHOLE PRINCIPLE

If you put 6 pigeons in 5 holes then at least one hole will have

more than one pigeon

HINT 1:

THE PIGEONHOLE PRINCIPLE

If you put 6 pigeons in 5 holes then at least one hole will have

more than one pigeon

HINT 1:

THE PIGEONHOLE PRINCIPLE

If you put n+1 pigeons in n holes then at least one hole will have

more than one pigeon

HINT 1:

HINT 2:Every integer a can be written as

a = 2km, where m is an odd number

PROOF IDEA:

Given: A ⊆ {1, 2, …, 2n} and |A| = n+1

Show: Use PHP to prove There is an integer m and elements a1 = a2 in A

such that a1 = 2im and a2 = 2jm

Suppose A ⊆ {1, 2, …, 2n} with |A| = n+1

Write every number in A as a = 2km, where m is an odd number between 1 and 2n-1

How many odd numbers in {1, …, 2n}? n

Since |A| = n+1, there must be two numbers in A with the same odd part

Say a1 and a2 have the same odd part m. Then a1 = 2im and a2 = 2jm, so one must divide the other

PROOF:

Suppose A ⊆ {1, 2, …, 2n} with |A| = n+1

Write every number in A as a = 2km, where m is an odd number between 1 and 2n-1

Put pigeon a in hole m <= 2n-1 n holes = odd numbers in {1, 3…, 2n-1}

There exists a hole m with 2 pigeons T2im and 2jm, so one must divide the other

PROOF:

DETERMINISTIC FINITE AUTOMATA

and REGULAR LANGUAGES

00,1

00

1

1

1

0111 111

11

1

The automaton accepts a string if the process ends in a double circle

Read string left to right

00,1

00

1

1

1ANATOMY OF A DETERMINISTIC

FINITE AUTOMATONstates

states

q0

q1

q2

q3start state (q0)

accept states (F)

00,1

00

1

1

1

0,1

ANATOMY OF A DETERMINISTIC FINITE AUTOMATON

An alphabet Σ is a finite set (e.g., Σ = {0,1})

A string over Σ is a finite-length sequence of elements of Σ

For string x, |x| is the length of x

The unique string of length 0 will be denoted by ε and will be called the empty or null string

SOME VOCABULARY

A language over Σ is a set of strings over Σ In other words: a language is a subset of Σ*

Σ* = the set of strings over Σ

Q is the set of states (finite)

Σ is the alphabet (finite)

δ : Q × Σ → Q is the transition function

q0 ∈ Q is the start state

F ⊆ Q is the set of accept/final states

A ^ finite automaton ^ is a 5-tuple M = (Q, Σ, δ, q0, F) deterministic DFA

Suppose w1, ... , wn ∈ Σ and w = w1... wn ∈ Σ* Then M accepts w iff there are r0, r1, ..., rn ∈ Q, s.t. • r0 = q0 • δ(ri, wi+1) = ri+1, for i = 0, ..., n-1, and • rn ∈ F

Q is the set of states (finite)

Σ is the alphabet (finite)

δ : Q × Σ → Q is the transition function

q0 ∈ Q is the start state

F ⊆ Q is the set of accept/final states

A ^ finite automaton ^ is a 5-tuple M = (Q, Σ, δ, q0, F)

M accepts ε iff q0 ∈ F

deterministic DFA

Q is the set of states (finite)

Σ is the alphabet (finite)

δ : Q × Σ → Q is the transition function

q0 ∈ Q is the start state

F ⊆ Q is the set of accept/final states

A ^ finite automaton ^ is a 5-tuple M = (Q, Σ, δ, q0, F)

L(M) = set of all strings that M accepts = “the language recognized by M”

deterministic DFA

00,1

00

1

1

1

L(M) = ?

0,1

0,1q0

L(M) = {0,1}*∅

q0 q1

0 0

1

1

L(M) = { w | w has an even number of 1s}

q0 q1

0 0

1

1

L(M) = { w | w has an odd number of 1s}

Q Σ q0 FM = ({p,q}, {0,1}, δ, p, {q}) δ 0 1

p p qq q p0

1

1

0

p qTHEOREM:

L(M) = {w | w has odd number of 1s }

Induction Hypothesis: Suppose for all w ∈ Σ*, |w| = n, w ∈ L(M) iff w has odd number of 1s.

Induction step: Any string of length n+1 has the form w0 or w1. Now w0 has an odd # of 1’s ⇔ w has an odd # of 1’s⇔ M is in state q after reading w (why?) ⇔ M is in state q after reading w0 (why?) ⇔w0 ∈ L(M)

Proof: By induction on n, the length of a string.Base Case: n=0: ε ∉ RHS and ε ∉ L(M). Why?

Q Σ q0 FM = ({p,q}, {0,1}, δ, p, {q}) δ 0 1

p p qq q p0

1

1

0

p qTHEOREM:

L(M) = {w | w has odd number of 1s }

Induction Hypothesis: Suppose for all w ∈ Σ*, |w| = n, w ∈ L(M) iff w has odd number of 1s.

Induction step: Any string of length n+1 has the form w0 or w1. Now w1 has an odd # of 1’s ⇔ w has an even # of 1’s⇔ M is in state p after reading w (why?) ⇔ M is in state q after reading w1 (why?) ⇔w1 ∈ L(M) QED

Proof: By induction on n, the length of a string.Base Case: n=0: ε ∉ RHS and ε ∉ L(M). Why?

Invariant Condition

If M in state p, M has read a W with an even number of 1s. If M in state q, M has read a W with an odd number of 1s.

Base Case, Invariant is true at start: Initially, M has seen 0 1s, and starts in state p.

Inductive step: M has read W with even number of 1s and is in p.

OR M has read W with odd number of 1s and is in q.

Next M sees 0, remains in same state, maintaining parity. OR M sees 1, changing state, maintaining parity invariant.

Thus, the invariant condition is always true.

0

1

1

0

p q

What the little machine is thinking:

If I am in p, I have seen an even number of 1s If I am in q, I have seen an odd number of 1s

q q00

1 0

1q0 q001

0 0 1

0,1

Build a DFA that accepts all and only those strings that contain 001

PROVE

Steven Rudich: www.cs.cmu.edu/~rudich rudich0123456789

L = all strings containing ababb as a consecutive substring

ba,b

a a

b

b

bb

aa

a

a ab aba ababeInvariant: I am state s exactly when s is the longest suffix of the input (so far) that forms a prefix of ababb.

DEFINITION: A language L is regular if it is recognized by a DFA,

i.e. if there is a DFA M s.t. L = L(M).

L = { w | w contains 001} is regular

L = { w | w has an even number of 1s} is regular

L = { w | w has an odd number of 1s} is regular

UNION THEOREMGiven two languages, L1 and L2, define the union of L1 and L2 as

L1 ∪ L2 = { w | w ∈ L1 or w ∈ L2 }

Theorem: The union of two regular languages is also a regular language

Theorem: The union of two regular languages is also a regular language

Proof: Let M1 = (Q1, Σ, δ1, q0, F1) be finite automaton for L1

and M2 = (Q2, Σ, δ2, q0, F2) be finite automaton for L2

We want to construct a finite automaton M = (Q, Σ, δ, q0, F) that recognizes L = L1 ∪ L2

1

2

Idea: Run both M1 and M2 at the same time!

Q = pairs of states, one from M1 and one from M2

= { (q1, q2) | q1 ∈ Q1 and q2 ∈ Q2 }= Q1 × Q2

q0 = (q0, q0)1 2

F = { (q1, q2) | q1 ∈ F1 or q2 ∈ F2 }

δ( (q1,q2), σ) = (δ1(q1, σ), δ2(q2, σ))

Theorem: The union of two regular languages is also a regular language

q0 q1

0 0

1

1

p0 p1

11

0

0

q0,p0 q1,p01

1

q0,p1 q1,p11

1

0000

INTERSECTION?

Intersection THEOREMGiven two languages, L1 and L2, define the intersection of L1 and L2 as

L1 ∩ L2 = { w | w ∈ L1 and w ∈ L2 }

Theorem: The intersection of two regular languages is also a regular language