Finite Automata and Regular Languagesccf.ee.ntu.edu.tw/~yen/courses/toc13/Lec-1.pdfFinite Automata and Regular Languages Topics to be covered in Chapters 1 - 4 include: deterministic

Finite Automata and Regular Languages

Topics to be covered in Chapters 1 - 4 include:

deterministic vs. nondeterministic FA,

regular expressions,

one-way vs. two-way FA,

minimization,

pumping lemma for regular sets,

closure properties,

Myhill-Nerode Theorem, ...

H. Yen (NTUEE) Theory of Computation Fall 2013 1 / 67

Definitions and Notation

An alphabet is finite set of set of symbols or ”letters”. Eg.A = {a, b, c},Σ = {0, 1}.A string or word over an alphabet A is a finite sequence of lettersfrom A. Eg. aaba is string over {a, b, c}.Empty string denoted by ε.

Set of all strings over A denoted by A∗.

What is the ”size” or ”cardinality” of A∗?Infinite but Countable: Can enumerate in lexicographic order:

ε, a, b, c , aa, ab, ...

Operation of concatenation on words: String u followed by string v :written u · v or simply uv .

Eg. aabb · aaa = aabbaaa.


Definitions and Notation

A language over an alphabet Σ is a set L of strings over Σ, i.e.,L ⊆ Σ∗.Eg. for Σ = {a, b, c}:

L = {abc, aaba}.L1 = {ε, b, aa, bb, aab, aba, baa, bbb, ...}.L2 = {}.L3 = {ε}.

How many languages are there over a given alphabet A?

Uncountably infinite

Example (Some languages)

Chinese, Swedish, English, Spanish, French, ...

Any programming language

∅, {ε} and Σ∗ are languages over any Σ

The set of prime natural numbers {1, 3, 5, 7, 11, ...}


Some Operations on Languages

Union, Intersection, ... : As for any set

Concatenation:

L1 · L2 = {u · v | u ∈ L1, v ∈ L2}.

Eg. {abc, aaba} · {ε, a, bb} = {abc, aaba, abca, aabaa, abcbb, aababb}.Closure:

L∗ =⋃n∈N

Ln

where L0 = {ε}, Ln+1 = Ln · L.

Note: We have ∅∗ = {ε} andL∗ = L0 ∪ L1 ∪ L2 ∪ ... = {ε} ∪ {x1...xn | n > 0, xi ∈ L}


Definitions and notation: DFA

A Deterministic Finite Automaton (DFA) A is a 5-tuple (Q,Σ, q0, δ,F )where

Q is a finite set of states,

Σ is the input alphabet,

q0 ∈ Q is the start state (or initial state),

δ : Q × Σ→ Q is the transition function, and

F ⊆ Q is the set of final states (or accepting states).


Example

Let the DFA (Q,Σ, δ, q0,F ) be given by:

Q = {q0, q1, q2}Σ = {0, 1}F = {q2}δ : Q × Σ→ Q

δ(q0, 0) = q1; δ(q1, 0) = q1; δ(q2, 0) = q2;

δ(q0, 1) = q0; δ(q1, 1) = q2; δ(q2, 1) = q2

What does the DFA do?


How to Represent a DFA?

Transition diagram:

Transition table:

The start state is indicated with →.The final states are indicated with ∗


When Does a DFA Accept a Word?

When reading the word the automaton moves according to δ.

Definition: If after reading the input it stops in a final state, it acceptsthe word.

Example

Only the word ”then” is accepted.We have a (non-accepting) stop or dead state q5.


Example: DFA

Let Σ = {0, 1}. We want to accept words in L = {x010y | x , y ∈ Σ∗}, i.e.,words that contain 010 as a subword.


Extended Transition Function

Definition of δ̂

We extend δ : Q × Σ→ Q to δ̂ : Q × Σ∗ → Q inductively by δ̂(q,w) =

δ̂(q, ε) = q w = ε

δ̂(q, ax) = δ̂(δ(q, a), x) w = ax , a ∈ Σ, x ∈ Σ∗

Note: δ̂(q, a) = δ(q, a) since the string a = aε.δ̂(q, a) = δ̂(q, aε) = δ̂(δ(q, a), ε) = δ(q, a)

Another definition δ̂′

δ̂′(q,w) =

δ̂′(q, ε) = q w = ε

δ̂′(q, xa) = δ(δ̂′(q, x), a) w = xa, a ∈ Σ, x ∈ Σ∗


Some Properties

Proposition

For any words x and y , and for any state q we have thatδ̂(q, xy) = δ̂(δ̂(q, x), y).

Proof.

We prove the result by induction on x.Basis case: δ̂(q, εy) = δ̂(q, y) = δ̂(δ̂(q, ε), y).Inductive step: Our IH is that δ̂(q, xy) = δ̂(δ̂(q, x), y) for any word y andany state q. We should prove that δ̂(q, (ax)y) = δ̂(δ̂(q, ax), y).

δ̂(q, (ax)y) = δ̂(q, a(xy)) by def of concat

= δ̂(δ(q, a), xy) by def of δ̂

= δ̂(δ̂(δ(q, a), x), y) by IH

= δ̂(δ̂(q, ax), y) by def of δ̂


Language Accepted by a DFA

Definition

The language accepted by the DFA M = (Q,Σ, δ, q0,F ), denoted byL(M), is the set {x | x ∈ Σ∗, δ̂(q0, x) ∈ F}.

Example

10101 is accepted but 00110 is not.

Definition

A language L ⊆ Σ∗ is called regular if there is a DFA M over Σ such thatL(M) = L.


Configuration

A configuration of a finite automaton A = (Q,Σ, δ, q0,F ) is given by astate of its control unit and the content of its tape that was not read yet.The set of all configurations (Conf) is Q × Σ∗

For an input word w ∈ Σ∗, (q0,w) is the initial configuration, and (qf , ε)is an final configuration where qf ∈ F .

Definition

(q,w) ` (q′,w ′) iff w = aw ′ and q′ = δ(q, a) for some a ∈ Σ.

Definition

A computation of an automaton is a sequence of configurationsC0,C1,C2, ...,Ck where Ci are configurations, C0 is an initial configuration,Ck is a final configuration, and for all i , we have Ci ` Ci+1.


Relation `∗

Definition

The relation `∗ is the reflexive and transitive closure of the relation `, i.e.,it is the smallest reflexive and transitive relation containing the relation `.

We also write qw→ q′ to denote that δ̂(q,w) = q′, i.e., starting in state q

goes to state q′ by reading word w .

NOTE: (q,w) `∗ (q′, ε) iff δ̂(q,w) = q′ iff qw→ q′, for DFA.

Definition

The language accepted by the DFA M = (Q,Σ, δ, q0,F ) is the set

L(M) = {x ∈ Σ∗ | δ̂(q0, x) ∈ F}= {x ∈ Σ∗ | (q0, x) `∗ (qf , ε),∃qf ∈ F}= {x ∈ Σ∗ | q0

x→ qf , ∃qf ∈ F}


Accepting a Word


Product of Automata

Definition

Given two DFA D1 = (Q1,Σ, δ1, q1,F1) and D2 = (Q2,Σ, δ2, q2,F2) withthe same alphabet Σ, we can define the product D = (Q,Σ, δ, q0,F ), alsodenoted by D1 × D2, as follows:

Q = Q1 × Q2

δ((r1, r2), a) = (δ1(r1, a), δ2(r2, a))

q0 = (q1, q2)

F = F1 × F2

Proposition

δ̂(r1, r2), x) = (δ̂1(r1, x), δ̂2(r2, x)).

Proof by Induction.


An Example

Example

⇓


Language Accepted by a Product Automaton

Theorem

Given two DFA D1 and D2, then L(D1 × D2) = L(D1) ∩ L(D2).

Proof.

δ̂(q0, x) = (δ̂1(q1, x), δ̂2(q2, x)) ∈ F iff δ̂1(q1, x) ∈ F1 and δ̂2(q2, x) ∈ F2,that is, x ∈ L(D1) and x ∈ L(D2), so x ∈ L(D1) ∩ L(D2).


Variation of the Product

Definition

We define D1 ⊕ D2 similarly to D1 × D2 but with a different notion ofaccepting state: a state (r1, r2) is accepting iff r1 ∈ F1 or r2 ∈ F2

Theorem

Given two DFA D1 and D2, then L(D1 ⊕ D2) = L(D1) ∪ L(D2).


Complement

Definition

Given the automaton D = (Q,Σ, δ, q0,F ) we define the complement D̄of D as the automaton D̄ = (Q,Σ, δ, q0,Q − F ).

Theorem

Given a DFA D we have that L(D̄) = Σ∗ − L(D).

Proof.

L(D̄) ⊆ Σ∗ − L(D)

w ∈ L(D̄)⇒ δ̂(q0,w) ∈ (Q − F )

⇒ δ̂(q0,w) 6∈ F⇒ w 6∈ L(D)⇒ w ∈ Σ∗ − L(D)

L(D̄) ⊇ Σ∗ − L(D)

Remark: We have that D1 ⊕ D2 = D1 × D2.H. Yen (NTUEE) Theory of Computation Fall 2013 20 / 67

Closure properties

Theorem

The class of regular languages is closed under

1 complement,

2 intersection,

3 union,

4 concatenation,

5 Kleene iteration.

(4) and (5) will be shown later.


Nondeterministic Finite Automata

A nondeterministic finite automaton (NFA) M is a 5-tuple(Q,Σ, q0, δ,F ), where

Q is a finite set of states,

Σ is the input alphabet,

q0 ∈ Q is the start state (or initial state),

δ : Q × Σ→ 2Q is the transition relation, and

F ⊆ Q is the set of final states (or accepting states).

Example

Given a state and the nextsymbol, the automata can”move” to many states.


Extending the Transition Function to Strings

Definition

δ̂ : Q × Σ∗ → 2Q

δ̂(q, ε) = {q}δ̂(q, ax) =

⋃p∈δ(q,a) δ̂(p, x)

That is, if δ(q, a) = {p1, ..., pn} then δ̂(q, ax) = δ̂(p1, x) ∪ ... ∪ δ̂(pn, x)

Definition

The language accepted by the NFA N = (Q,Σ, δ, q0,F ) is the setL(N) = {x ∈ Σ∗ | δ̂(q0, x) ∩ F 6= ∅}.

That is, a word x is accepted if δ̂(q0, x) contains at least one acceptingstate.


DFA and NFA

A DFA can be turned into an NFA that accepts the same language.

If δ(q, a) = p, let the NFA have δ(q, a) = {p}.Then the NFA is always in a set containing exactly one state - thestate the DFA is in after reading the same input.

Surprisingly, for any NFA there is a DFA that accepts the samelanguage.

Proof is the subset construction.

The number of states of the DFA can be exponential in the numberof states of the NFA. Thus, NFA!—s accept exactly the regularlanguages.


Subset Construction

Subset construction

Given an NFA N = (QN ,Σ, δN , q0,FN), we will construct a DFAD = (QD ,Σ, δD , {q0},FD) such that L(D) = L(N)

QD = {S : S ⊆ QN}FD = {S : S ⊆ QN , and S ∩ FN 6= ∅}∀S ⊆ QN , a ∈ Σ,

δD(S , a) =⋃p∈S

δN(p, a)

The DFA states have names that are sets of NFA states.

But as a DFA state, an expression like {p, q} must be read as a singlesymbol, not as a set.

Analogy: a class of objects whose values are sets of objects of anotherclass.


Subset Construction (cont’d)

Example:



Example (cont’d):



Example (cont’d):



Example (cont’d):



Example (cont’d):



Example (cont’d):


Subset Cconstruction (cont’d)

Example (cont’d):



Example (cont’d):


Proof of Equivalence

By Induction on |w | that δN(q0,w) = δD({q0},w)

Basis: w = ε: δN(q0, ε) = δD({q0}, ε) = {q0}.Ind. Hypothesis: Assume IH for strings shorter than w .

Ind. Step: Let w = xa; IH holds for x .


Exponential Blow-Up

There is an NFA N with n + 1 states that has no equivalent DFA withfewer than 2n states.

L(N) = {x1c2c3 · · · cn : x ∈ {0, 1}∗, ci ∈ {0, 1}}.Suppose an equivalent DFA D with fewer than 2n states exists.

D must remember the last n symbols it has read.

There are 2n bitsequences a1a2 · · · an.

∃q, a1a2 · · · an, b1b2 · · · bn : q ∈ δ̂N(q0, a1a2 · · · an), q ∈δ̂N(q0, b1b2 · · · bn), a1a2 · · · an 6= b1b2 · · · bn


Exponential Blow-Up (Cont’d)

a1 · · · ai11ai+1 · · · anb1 · · · bi10bi+1 · · · bn

Now

δ̂N(q0, a1 · · · ai−11ai+1 · · · an0i−1) = δ̂N(q0, b1 · · · bi−10bi+1 · · · bn0i−1)

Andδ̂N(q0, a1 · · · ai−11ai+1 · · · an0i−1) ∈ FD

δ̂N(q0, b1 · · · bi−10bi+1 · · · bn0i−1) 6∈ FD

– A contradiction!


NFA with ε-Transitions

We can allow state-to-state transitions on ε input.

These transitions are done spontaneously, without looking at theinput string.

A convenience at times, but still only regular languages are accepted.

Example


Closure of States

CL(q) = set of states you can reach from state q following only arcslabeled ε.

CL(A) = {A}; CL(E ) = {B,C ,D,E}.

Closure of a set of states = union of the closure of each state.


Extended Delta δ̂

Basis: δ̂(q, ε) = CL(q)

Induction:δ̂(q, xa) =

⋃p∈δ(δ̂(q,x),a)

CL(p)

Example

δ̂(A, ε) = CL(A) = {A}.δ̂(A, 0) = CL({E}) ={B,C ,D,E}.δ̂(A, 01) = CL({C ,D}) ={C ,D}.


Equivalence of ε-NFA and DFA

Given an ε-NFA E = (QE ,Σ, δE , q0,FE ), we will construct a DFAD = (QD ,Σ, δD , qD ,FD) such that L(D) = L(E )

Details of the construction:

QD = {S : S ⊆ QE , and S = CL(S)}qD = CL(q0)

FD = {S : S ∈ QD , and S ∩ FE 6= ∅}δD(S , a) =

⋃{CL(p) : p ∈ δ(t, a), for some t ∈ S}


Regular Expression

Regular expressions are an ”algebraic” way to denote languages.

Syntax of regular expressions

Syntax of regular expresions over an alphabet Σ:

r ::= ∅ | a | r + r | r · r | r∗,

where a ∈ Σ.

Semantics of regular expressions

Semantics: associate a language L(r) ⊆ Σ∗ with regexp r .

L(∅) = {}L(a) = {a}L(r + r ′) = L(r) ∪ L(r ′)

L(r · r ′) = L(r) · L(r ′)

L(r∗) = L(r)∗.


Algebraic Laws for Regular Expressions

The following equalities hold for any RE R, S and T:


Kleene’s Theorem: RE = DFA

Class of languages defined by regular expressions coincides with regularlanguages.Proof.

RE → DFA: Use closure properties of regular languages.

DFA → RE:


DFA → RE: Kleene’s Construction


DFA → RE: Kleene’s Construction (cont’d)


DFA → RE: Kleene’s Construction (cont’d)


DFA → RE: Using State Elimination


DFA → RE: Using State Elimination (cont’d)

For each q ∈ F we’ll be left with an Aq that looks like

Eq = (R + SU∗T )∗SU∗ Eq = R∗

The final expression is⊕

q∈F Eq.


DFA → RE: Using System of Equations


DFA → RE: Using System of Equations (cont’d)



Lq’s are a solution to the system of equations

In general there could be many solutions to equations. Considerx = A∗x .

In this case, Lq’s can be seen to the least solution to the equations.




RE → ε-NFA

Theorem

For every regex R we can construct and ε-NFA A, s.t. L(A) = L(R).


Induction Proof


An Example

We convert (0 + 1)∗1(0 + 1)


Equivalence Relation

Definition

A binary relation R on a set S is a subset of S × S . An equivalencerelation on a set satisfies

1 Reflexivity: For all x in S , xRx

2 Symmetry: For x , y ∈ S xRy ⇔ yRx

3 Transitivity: For x , y , z ∈ S xRy ∧ yRz ⇒ xRz

Every equivalence relation on S partitions S into equivalence classes.

The number of equivalence classes is called the index of the relation.


Right Invariant

Definition

An equivalence relation on Σ∗ is said to be right invariant with respect toconcatenation if ∀x , y ∈ Σ∗ and a ∈ Σ, xRy implies that xaRya.


Equivalence relations induced by DFA

Let M = (Q,Σ, δ, q0,F ) be a DFA.

Define a relation RM as follows:

For x , y ∈ Σ∗, xRMy ⇔ δ(q0, x) = δ(q0, y).

Is this an equivalence relation?

If so, how many equivalence classes does it have? That is, what is itsindex?


An Example

Example


Refinement

Definition

An equivalence relation R1 is a refinement of R2 if R1 ⊆ R2, i.e.(x , y) ∈ R1 ⇒ (x , y) ∈ R2


The Myhill-Nerode Theorem

Theorem

Let L ⊆ Σ∗. The following statements are equivalent:

1 L is recognized by a DFA

2 L is the union of some of the equivalence classes of a right invariantequivalence relation of finite index.

3 Define an equivalence relation RL as follows. Forx , y ∈ Σ∗, (x , y) ∈ RL ⇔ ∀z ∈ Σ∗, xz ∈ L whenever yz ∈ L. Then RL

has finite index.

To prove this, we need to prove that 1⇒ 2⇒ 3⇒ 1.


Proof: 1⇒ 2

(1) L is recognized by a DFA

(2) L is the union of some of the equivalence classes of a right invariantequivalence relation of finite index.

Let M = (Q,Σ, δ, q0,F ) be the DFA that recognizes L.

The relation RM defined earlier as follows:For x , y ∈ Σ∗, xRMy ⇔ δ(q0, x) = δ(q0, y)is an equivalence relation which is of finite index and is also rightinvariant as M is a DFA.

L is just the union of the equivalence classes corresponding to thefinal states of M


Proof: 2⇒ 3

(2) L is the union of some of the equivalence classes of a right invariantequivalence relation of finite index.

(3) Define an equivalence relation RL as follows. Forx , y ∈ Σ∗, (x , y) ∈ RL ⇔ ∀z ∈ Σ∗, xz ∈ L whenever yz ∈ L. Then RL

has finite index.

Let E be the equivalence relation in (2). We show that E is arefinement of RL. Hence since E has finite index, so does RL.

We show that if (x , y) ∈ E then (x , y) ∈ RL, i.e. E ⊆ RL and henceE is a refinement of RL.

(x , y) ∈ E ⇒ ∀z ∈ Σ∗, (xz , yz) ∈ E by repeated use of rightinvariance. Hence xz is in an equivalence class in L iff yz is, therebyimplying that (x , z) ∈ RL. Thus RL has finite index.


Proof: 3⇒ 1

(3) Define an equivalence relation RL as follows. Forx , y ∈ Σ∗, (x , y) ∈ RL ⇔ ∀z ∈ Σ∗, xz ∈ L whenever yz ∈ L. Then RL

has finite index.

(1) L is recognized by a DFA

Construct a DFA ML = (QL,Σ, δL, q0L,FL).

We first show that RL is right invariant. To see this assume that(x , y) ∈ RL. Assume RL is not right invariant. Then there existsa ∈ Σ such that (xa, ya) 6∈ RL. Let z be a ”distinguishing” string forthe pair (xa, ya). Then az is a distinguishing string for (x , y)contradicting the assumption that (x , y) ∈ RL. Therefore RL is rightinvariant.


Proof: 3⇒ 1 (cont’d)

Let [x ] denote the equivalence class containing x . The state setQL = {[x ] : x ∈ Σ∗}. The transition function δL is defined asδL([x ], a) = [xa]. The definition is consistent because had we chosensome other representative [y ] of the equivalence class, [xa] = [ya] byright invariance of RL so the transition is to the same state of ML.

We need to prove that x ∈ L⇔ δL(q0L, x) ∈ FL i.e. x is accepted byML

This is easy as δL(q0L, x) = [x ] and x is accepted by ML iff[x ] ∈ FL ⇔ x ∈ L.


Applications of the Myhill-Nerode Theorem

The MN theorem can be used to show that a particular language is regularwithout actually constructing the automaton or to show conclusively thata language is not regular.Example. Is the following language regular

1 L1 = {xy : |x | = |y |, x , y ∈ Σ∗}?2 Example. What about the language

L2 = {xy : |x | = |y |, x , y ∈ Σ∗ and y ends with a 1 }?3 Example. What about the language

L3 = {xy : |x | = |y |, x , y ∈ Σ∗ and y contains a 1}?


Applications of the Myhill-Nerode Theorem (cont’d)

1 For the language L1 there are two equivalence classes of RL1 . Thefirst C1 contains all strings of even length and the second C2 allstrings of odd length.

2 For L2 we have the additional constraint that y ends with a 1. ClassC2 remains the same as that for L1. Class C1 is refined into classes C ′1which contains all strings of even length that end in a 1 and C ′′1 whichcontains all strings of even length which end in a 0. Thus L1 and L2

are both regular.

3 For L3 we have to distinguish for example, between the even lengthstrings in the sequence 01, 0001, 000001,..., as 00 distinguishes thefirst string from all the others after it in the sequence (concatenationof 00 to 01 gives a string not in L3 but concatenation of 00 to all theothers gives a string in L3), 0000 distinguishes the second from all theothers ...


Documents

Finite Automata and Regular Languagesccf.ee.ntu.edu.tw/~yen/courses/toc13/Lec-1.pdfFinite Automata and Regular Languages Topics to be covered in Chapters 1 - 4 include: deterministic