COMP 335Introduction toTheoretical Computer ScienceSection GInstructor: G. GrahneLectures: Tuesdays and Thursdays,11:45 { 13:00, H 521O_ce hours: Tuesdays,14:00 - 15:00, LB 903-11_ All slides shown here are on the web.www.cs.concordia.ca/~teaching/comp335/2003F1Thanks to David Ford for TEX assistance.Thanks to the following students of comp335Winter 2002, for spotting errors in previousversions of the slides: Omar Khawajkie, CharlesdeWeerdt, Wayne Jang, Keith Kang, BeiWang,Yapeng Fan, Monzur Chowdhury, Pei JennyTse, Tao Pan, Shahidur Molla, Bartosz Adamczyk,Hratch Chitilian, Philippe Legault.Tutor: TBATutorial: Tuesdays, 13:15 { 14:05, H 411_ Tutorials are an integral part of this course.2Course organizationTextbook: J. E. Hopcroft, R. Motwani, andJ. D. Ullman Introduction to Automata Theory,Languages, and Computation, Second Edition,Addison-Wesley, New York, 2001.Sections: There are two parallel sections. Thematerial covered by each instructor is roughlythe same. There are four common assignmentsand a common _nal exam. Each sectionwill have di_erent midterm tests.Assignments: There will be four assignments.Each student is expected to solve the assignmentsindependently, and submit a solution forevery assigned problem.3Examinations: There will be three midtermexaminations, each lasting thirty minutes andcovering the material of the most recent assignment.The _nal examination will be a threehourexamination at the end of the term.Weight distribution:Midterm examinations: 3 _ 15% = 45%;Final examination: = 55%:_ At the end of the term, any midterm exammark lower than your _nal exam mark will bereplaced by your _nal exam mark. To passthe course you must submit solutions for allassigned problems.4Important: COMP 238 and COMP 239 areprerequisites. For a quick refresher course,read Chapter 1 in the textbook._ Spend some time every week on:(1) learning the course content,(2) solving exercises._ Visit the course web site regularly for updatedinformation.5Motivation_ Automata = abstract computing devices_ Turing studied Turing Machines (= computers)before there were any real computers_ We will also look at simpler devices thanTuring machines (Finite State Automata, PushdownAutomata, . . . ), and speci_cation means,such as grammars and regular expressions._ NP-hardness = what cannot be e_cientlycomputed6Finite AutomataFinite Automata are used as a model for_ Software for designing digital cicuits_ Lexical analyzer of a compiler_ Searching for keywords in a _le or on theweb._ Software for verifying _nite state systems,such as communication protocols.7_ Example: Finite Automaton modelling anon/o_ switchPushPushStartoff on_ Example: Finite Automaton recognizing thestring thent th theStart t h e nthen8Structural RepresentationsThese are alternative ways of specifying a machineGrammars: A rule like E ) E+E speci_es anarithmetic expression_ Lineup ) Person:Lineupsays that a lineup is a person in front of alineup.Regular Expressions: Denote structure of data,e.g.'[A-Z][a-z]*[][A-Z][A-Z]'matches Ithaca NYdoes not match Palo Alto CAQuestion: What expression would matchPalo Alto CA9Central ConceptsAlphabet: Finite, nonempty set of symbolsExample: _ = f0; 1g binary alphabetExample: _ = fa; b; c; : : : ; zg the set of all lowercase lettersExample: The set of all ASCII charactersStrings: Finite sequence of symbols from analphabet _, e.g. 0011001Empty String: The string with zero occurrencesof symbols from __ The empty string is denoted _10Length of String: Number of positions forsymbols in the string.jwj denotes the length of string wj0110j = 4; j_j = 0Powers of an Alphabet: _k = the set ofstrings of length k with symbols from _Example: _ = f0; 1g_1 = f0; 1g_2 = f00; 01; 10; 11g_0 = f_gQuestion: How many strings are there in _311The set of all strings over _ is denoted ____ = _0 [ _1 [ _2 [ _ _ _Also:_+ = _1 [ _2 [ _3 [ _ _ ___ = _+ [ f_gConcatenation: If x and y are strings, thenxy is the string obtained by placing a copy ofy immediately after a copy of xx = a1a2 : : : ai; y = b1b2 : : : bjxy = a1a2 : : : aib1b2 : : : bjExample: x = 01101; y = 110; xy = 01101110Note: For any string xx_ = _x = x12Languages:If _ is an alphabet, and L _ __then L is a languageExamples of languages:_ The set of legal English words_ The set of legal C programs_ The set of strings consisting of n 0's followedby n 1'sf_; 01; 0011; 000111; : : :g13_ The set of strings with equal number of 0'sand 1'sf_; 01; 10; 0011; 0101; 1001; : : :g_ LP = the set of binary numbers whose valueis primef10; 11; 101; 111; 1011; : : :g_ The empty language ;_ The language f_g consisting of the emptystringNote: ; 6= f_gNote2: The underlying alphabet _ is always_nite14Problem: Is a given string w a member of alanguage L?Example: Is a binary number prime = is it ameber in LPIs 11101 2 LP ? What computational resourcesare needed to answer the question.Usually we think of problems not as a yes/nodecision, but as something that transforms aninput into an output.Example: Parse a C-program = check if theprogram is correct, and if it is, produce a parsetree.Let LX be the set of all valid programs in proglang X. If we can show that determining membershipin LX is hard, then parsing programswritten in X cannot be easier.Question: Why?15Finite Automata InformallyProtocol for e-commerce using e-moneyAllowed events:1. The customer can pay the store (=sendthe money-_le to the store)2. The customer can cancel the money (likeputting a stop on a check)3. The store can ship the goods to the customer4. The store can redeem the money (=cashthe check)5. The bank can transfer the money to thestore16e-commerceThe protocol for each participant:1 3 42redeem transfercancelStarta bcd fe gStart(a) Store(b) Customer (c) Bankredeem transfership shipredeem transfershippaycancelStart pay17Completed protocols:cancel1 3 42redeem transfercancelStarta bcd fe gStart(a) Store(b) Customer (c) Bankship ship shipredeem transferpay redeem transferpay, cancelship. redeem, transfer,pay,shippay, shippay,cancel pay,cancel pay,cancelpay,cancel pay,cancel pay,cancelcancel, ship cancel, shippay,redeem, pay,redeem,Start18The entire system as an Automaton:C C C C C C CP P P P P PP P P P P PP,C P,CC P,C P,C P,C P,C P,C P,CCP S S SP S S SP S SP S S Sa b c d e f g1234StartP,CP,C P,C P,CRRSTTRRRR19Deterministic Finite AutomataA DFA is a quintupleA = (Q;_; _; q0; F)_ Q is a _nite set of states_ _ is a _nite alphabet (=input symbols)_ _ is a transition function (q; a) 7! p_ q0 2 Q is the start state_ F _ Q is a set of _nal states20Example: An automaton A that acceptsL = fx01y : x; y 2 f0; 1g_gThe automaton A = (fq0; q1; q2g; f0; 1g; _; q0; fq1g)as a transition table:0 1! q0 q2 q0?q1 q1 q1q2 q2 q1The automaton as a transition diagram:1 0q 0 1 0 q2 q1 0, 1Start21An FA accepts a string w = a1a2 _ _ _ an if thereis a path in the transition diagram that1. Begins at a start state2. Ends at an accepting state3. Has sequence of labels a1a2 _ _ _ anExample: The FAStart q 0 1 0 q q0, 11 2accepts e.g. the string 0110122_ The transition function _ can be extendedto ^_ that operates on states and strings (asopposed to states and symbols)Basis: ^_(q; _) = qInduction: ^_(q; xa) = _(^_(q; x); a)_ Now, fomally, the language accepted by AisL(A) = fw : ^_(q0;w) 2 Fg_ The languages accepted by FA:s are calledregular languages23Example: DFA accepting all and only stringswith an even number of 0's and an even numberof 1'sq qq q0 12 3Start00110011Tabular representation of the Automaton0 1? ! q0 q2 q1q1 q3 q0q2 q0 q3q3 q1 q224ExampleMarble-rolling toy from p. 53 of textbookA BC Dxx x32125A state is represented as sequence of three bitsfollowed by r or a (previous input rejected oraccepted)For instance, 010a, meansleft, right, left, acceptedTabular representation of DFA for the toyA B! 000r 100r 011r?000a 100r 011r?001a 101r 000a010r 110r 001a?010a 110r 001a011r 111r 010a100r 010r 111r?100a 010r 111r101r 011r 100a?101a 011r 100a110r 000a 101a?110a 000a 101a111r 001a 110a26Nondeterministic Finite AutomataA NFA can be in several states at once, or,viewded another way, it can \guess" whichstate to go to nextExample: An automaton that accepts all andonly strings ending in 01.Start q 0 1 0 q q0, 11 2Here is what happens when the NFA processesthe input 00101q0q2q0 q0 q0 q0 q0q1 q1 q1q20 0 1 0 1(stuck)(stuck)27Formally, a NFA is a quintupleA = (Q;_; _; q0; F)_ Q is a _nite set of states_ _ is a _nite alphabet_ _ is a transition function from Q _ _ to thepowerset of Q_ q0 2 Q is the start state_ F _ Q is a set of _nal states28Example: The NFA from the previous slide is(fq0; q1; q2g; f0; 1g; _; q0; fq2g)where _ is the transition function0 1! q0 fq0; q1g fq0gq1 ; fq2g?q2 ; ;29Extended transition function ^_.Basis: ^_(q; _) = fqgInduction:^_(q; xa) =[p2^_(q;x)_(p; a)Example: Let's compute ^_(q0; 00101) on theblackboard_ Now, fomally, the language accepted by A isL(A) = fw : ^_(q0;w) \ F 6= ;g30Let's prove formally that the NFAStart q 0 1 0 q q0, 11 2accepts the language fx01 : x 2 __g: We'll doa mutual induction on the three statementsbelow0. w 2 __ ) q0 2 ^_(q0;w)1. q1 2 ^_(q0;w) , w = x02. q2 2 ^_(q0;w) , w = x0131Basis: If jwj = 0 then w = _. Then statement(0) follows from def. For (1) and (2) bothsides are false for _Induction: Assume w = xa, where a 2 f0; 1g;jxj = n and statements (0){(2) hold for x. Wewill show on the blackboard in class that thestatements hold for xa.32Equivalence of DFA and NFA_ NFA's are usually easier to \program" in._ Surprisingly, for any NFA N there is a DFA D,such that L(D) = L(N), and vice versa._ This involves the subset construction, an importantexample how an automaton B can begenerically constructed from another automatonA._ Given an NFAN = (QN;_; _N; q0; FN)we will construct a DFAD = (QD;_; _D; fq0g; FD)such thatL(D) = L(N).33The details of the subset construction:_ QD = fS : S _ QNg.Note: jQDj = 2jQNj, although most states inQD are likely to be garbage._ FD = fS _ QN : S \ FN 6= ;g_ For every S _ QN and a 2 _,_D(S; a) =[p2S_N(p; a)34Let's construct _D from the NFA on slide 270 1; ; ;! fq0g fq0; q1g fq0gfq1g ; fq2g?fq2g ; ;fq0; q1g fq0; q1g fq0; q2g?fq0; q2g fq0; q1g fq0g?fq1; q2g ; fq2g?fq0; q1; q2g fq0; q1g fq0; q2g35Note: The states of D correspond to subsetsof states of N, but we could have denoted thestates of D by, say, A F just as well.0 1A A A! B E BC A D?D A AE E F?F E B?G A D?H E F36We can often avoid the exponential blow-upby constructing the transition table for D onlyfor accessible states S as follows:Basis: S = fq0g is accessible in DInduction: If state S is accessible, so are thestates inSa2_ _D(S; a).Example: The \subset" DFA with accessiblestates only.Start{q {q {q 0 0 0 , q1 , q2 } }0 11 001}37Theorem 2.11: Let D be the \subset" DFAof an NFA N. Then L(D) = L(N).Proof: First we show on an induction on jwjthat^_D(fq0g;w) = ^_N(q0;w)Basis: w = _. The claim follows from def.38Induction:^_D(fq0g; xa)def= _D(^_D(fq0g; x); a)i:h: = _D(^_N(q0; x); a)cst=[p2^_N(q0;x)_N(p; a)def= ^_N(q0; xa)Now (why?) it follows that L(D) = L(N).39Theorem 2.12: A language L is accepted bysome DFA if and only if L is accepted by someNFA.Proof: The \if" part is Theorem 2.11.For the \only if" part we note that any DFAcan be converted to an equivalent NFA by modifyingthe _D to _N by the rule_ If _D(q; a) = p, then _N(q; a) = fpg.By induction on jwj it will be shown in thetutorial that if ^_D(q0;w) = p, then ^_N(q0;w) =fpg.The claim of the theorem follows.40Exponential Blow-UpThere is an NFA N with n+1 states that hasno equivalent DFA with fewer than 2n statesStart0, 10, 1 0, 1 0, 1q q q q 0 1 2 n1 0, 1L(N) = fx1c2c3 _ _ _ cn : x 2 f0; 1g_; ci 2 f0; 1ggSuppose an equivalent DFA D with fewer than2n states exists.D must remember the last n symbols it hasread.There are 2n bitsequences a1a2 _ _ _ an9 q; a1a2 _ _ _ an; b1b2 _ _ _ bn : q 2 ^_N(q0; a1a2 _ _ _ an);q 2 ^_N(q0; b1b2 _ _ _ bn);a1a2 _ _ _ an 6= b1b2 _ _ _ bn41Case 1:1a2 _ _ _ an0b2 _ _ _ bnThen q has to be both an accepting and anonaccepting state.Case 2:a1 _ _ _ ai11ai+1 _ _ _ anb1 _ _ _ bi10bi+1 _ _ _ bnNow ^_N(q0; a1 _ _ _ ai11ai+1 _ _ _ an0i1) =^_N(q0; b1 _ _ _ bi10bi+1 _ _ _ bn0i1)and ^_N(q0; a1 _ _ _ ai11ai+1 _ _ _ an0i1) 2 FD^_N(q0; b1 _ _ _ bi10bi+1 _ _ _ bn0i1) =2 FD42FA's with Epsilon-TransitionsAn _-NFA accepting decimal numbers consistingof:1. An optional + or - sign2. A string of digits3. a decimal point4. another string of digitsOne of the strings (2) are (4) are optionalq q q q qq0 1 2 3 54Start0,1,...,9 0,1,...,90,1,...,90,1,...,9,+,-..43Example:_-NFA accepting the set of keywords febay; webg12 3 45 6 7 8Startweeb a yb44An _-NFA is a quintuple (Q;_; _; q0; F) where _is a function from Q __[ f_g to the powersetof Q.Example: The _-NFA from the previous slideE = (fq0; q1; : : : ; q5g; f:;+;; 0; 1; : : : ; 9g _; q0; fq5g)where the transition table for _ is_ +,- . 0; : : : ; 9! q0 fq1g fq1g ; ;q1 ; ; fq2g fq1; q4gq2 ; ; ; fq3gq3 fq5g ; ; fq3gq4 ; ; fq3g ;?q5 ; ; ; ;45ECLOSEWe close a state by adding all states reachableby a sequence __ _ _ _ _Inductive de_nition of ECLOSE(q)Basis:q 2 ECLOSE(q)Induction:p 2 ECLOSE(q) and r 2 _(p; _) )r 2 ECLOSE(q)46Example of _-closure12 3 64 5 7a bFor instance,ECLOSE(1) = f1; 2; 3; 4; 6g47_ Inductive de_nition of ^_ for _-NFA'sBasis:^_(q; _) = ECLOSE(q)Induction:^_(q; xa) =[p2_(^_(q;x);a)ECLOSE(p)Let's compute on the blackboard in class^_(q0; 5:6) for the NFA on slide 4348Given an _-NFAE = (QE;_; _E; q0; FE)we will construct a DFAD = (QD;_; _D; qD; FD)such thatL(D) = L(E)Details of the construction:_ QD = fS : S _ QE and S = ECLOSE(S)g_ qD = ECLOSE(q0)_ FD = fS : S 2 QD and S \ FE 6= ;g_ _D(S; a) =[fECLOSE(p) : p 2 _(t; a) for some t 2 Sg49Example: _-NFA Eq q q q qq0 1 2 3 54Start0,1,...,9 0,1,...,90,1,...,90,1,...,9,+,-..DFA D corresponding to EStart{ { { {{ {q q q qq q0 1 1, q }1} , q4} 2, q3, q5}2} 3, q5}0,1,...,9 0,1,...,90,1,...,90,1,...,90,1,...,90,1,...,9+,-...50Theorem 2.22: A language L is accepted bysome _-NFA E if and only if L is accepted bysome DFA.Proof: We use D constructed as above andshow by induction that ^_D(q0;w) = ^_E(qD;w)Basis: ^_E(q0; _) = ECLOSE(q0) = qD = ^_(qD; _)51Induction:^_E(q0; xa) =[p2_E(^_E(q0;x);a)ECLOSE(p)=[p2_D(^_D(qD;x);a)ECLOSE(p)=[p2^_D(qD;xa)ECLOSE(p)= ^_D(qD; xa)52Regular expressionsA FA (NFA or DFA) is a \blueprint" for contructinga machine recognizing a regular language.A regular expression is a \user-friendly," declarativeway of describing a regular language.Example: 01_ +10_Regular expressions are used in e.g.1. UNIX grep command2. UNIX Lex (Lexical analyzer generator) andFlex (Fast Lex) tools.53Operations on languagesUnion:L [M = fw : w 2 L or w 2 MgConcatenation:L:M = fw : w = xy; x 2 L; y 2 MgPowers:L0 = f_g; L1 = L; Lk+1 = L:LkKleene Closure:L_ = 1[i=0LiQuestion: What are ;0; ;i, and ;_54Building regex'sInductive de_nition of regex's:Basis: _ is a regex and ; is a regex.L(_) = f_g, and L(;) = ;.If a 2 _, then a is a regex.L(a) = fag.Induction:If E is a regex's, then (E) is a regex.L((E)) = L(E).If E and F are regex's, then E +F is a regex.L(E +F) = L(E) [ L(F).If E and F are regex's, then E:F is a regex.L(E:F ) = L(E):L(F).If E is a regex's, then E? is a regex.L(E?) = (L(E))_.55Example: Regex forL = fw 2 f0; 1g_ : 0 and 1 alternate in wg(01)_ +(10)_ +0(10)_ +1(01)_or, equivalently,(_+1)(01)_(_+0)Order of precedence for operators:1. Star2. Dot3. PlusExample: 01_ +1 is grouped (0(1)_)+156Equivalence of FA's and regex'sWe have already shown that DFA's, NFA's,and _-NFA's all are equivalent.-NFA NFARE DFATo show FA's equivalent to regex's we need toestablish that1. For every DFA A we can _nd (construct,in this case) a regex R, s.t. L(R) = L(A).2. For every regex R there is a _-NFA A, s.t.L(A) = L(R).57Theorem 3.4: For every DFA A = (Q;_; _; q0; F)there is a regex R, s.t. L(R) = L(A).Proof: Let the states of A be f1; 2; : : : ; ng,with 1 being the start state._ Let R(k)ij be a regex describing the set oflabels of all paths in A from state i to statej going through intermediate states f1; : : : ; kgonly.ikj58R(k)ij will be de_ned inductively. Note thatL0@Mj2FR1j(n)1A = L(A)Basis: k = 0, i.e. no intermediate states._ Case 1: i 6= jR(0)ij =Mfa2_:_(i;a)=jga_ Case 2: i = jR(0)ii =0B@Mfa2_:_(i;a)=iga1CA+_59Induction:R(k)ij=R(k1)ij+R(k1)ik_R(k1)kk__R(k1)kjR kj(k-1)R kk(k-1) R ik(k-1)i k k k kZero or more strings inIn Inj60Example: Let's _nd R for A, whereL(A) = fx0y : x 2 f1g_ and y 2 f0; 1g_g1Start 0 0,11 2R(0)11 _+1R(0)12 0R(0)21 ;R(0)22 _+0+161We will need the following simpli_cation rules:_ (_+R)_ = R__ R+RS_ = RS__ ;R = R; = ; (Annihilation)_ ;+R = R+; = R (Identity)62R(0)11 _+1R(0)12 0R(0)21 ;R(0)22 _+0+1R(1)ij = R(0)ij +R(0)i1_R(0)11__R(0)1jBy direct substitution Simpli_edR(1)11 _+1+(_+1)(_+1)_(_+1) 1_R(1)12 0+(_+1)(_+1)_0 1_0R(1)21 ;+;(_+1)_(_+1) ;R(1)22 _+0+1+;(_+1)_0 _+0+163Simpli_edR(1)11 1_R(1)12 1_0R(1)21 ;R(1)22 _+0+1R(2)ij = R(1)ij +R(1)i2_R(1)22__R(1)2jBy direct substitutionR(2)11 1_ +1_0(_+0+1)_;R(2)12 1_0+1_0(_+0+1)_(_+0+1)R(2)21 ;+(_+0+1)(_+0+1)_;R(2)22 _+0+1+(_+0+1)(_+0+1)_(_+0+1)64By direct substitutionR(2)11 1_ +1_0(_+0+1)_;R(2)12 1_0+1_0(_+0+1)_(_+0+1)R(2)21 ;+(_+0+1)(_+0+1)_;R(2)22 _+0+1+(_+0+1)(_+0+1)_(_+0+1)Simpli_edR(2)11 1_R(2)12 1_0(0+1)_R(2)21 ;R(2)22 (0+1)_The _nal regex for A isR(2)12 = 1_0(0+1)_65ObservationsThere are n3 expressions R(k)ijEach inductive step grows the expression 4-foldR(n)ij could have size 4nFor all fi; jg _ f1; : : : ; ng, R(k)ij uses R(k1)kkso we have to write n2 times the regex R(k1)kkWe need a more e_cient approach:the state elimination technique66The state elimination techniqueLet's label the edges with regex's instead ofsymbolsqqpp1 1k msQQP1Pmk111 RR 1mR kmR k1S67Now, let's eliminate state s.11 R Q1 P1R 1mR k1R kmQ1 PmQkQkP1Pmqqpp1 1k m+ S*+++S*S*S*For each accepting state q eliminate from theoriginal automaton all states exept q0 and q.68For each q 2 F we'll be left with an Aq thatlooks likeStartRSTUthat corresponds to the regex Eq = (R+SU_T)_SU_or with Aq looking likeRStartcorresponding to the regex Eq = R__ The _nal expression isMq2FEq69Example: A, where L(A) = fW : w = x1b; or w =x1bc; x 2 f0; 1g_; fb; cg _ f0; 1ggStart0,11 0,1 0,1A B C DWe turn this into an automaton with regexlabels0 + 1Start 0 + 1 0 + 1A B C D1700 + 1Start 0 + 1 0 + 1A B C D1Let's eliminate state B0 + 1C DStart (0 + 1) 0 + 1A1Then we eliminate state C and obtain AD0 + 1DStart (0 + 1)(0 + 1)A1with regex (0+1)_1(0+1)(0+1)71From0 + 1C DStart (0 + 1) 0 + 1A1we can eliminate D to obtain AC0 + 1CStart (0 + 1)A1with regex (0+1)_1(0+1)_ The _nal expression is the sum of the previoustwo regex's:(0+1)_1(0+1)(0+1)+(0+1)_1(0+1)72From regex's to _-NFA'sTheorem 3.7: For every regex R we can constructand _-NFA A, s.t. L(A) = L(R).Proof: By structural induction:Basis: Automata for _, ;, and a.a(a)(b)(c)73Induction: Automata for R+S, RS, and R_(a)(b)(c)RSR SR74Example: We convert (0+1)_1(0+1)01011 Start(a)(b)(c)010175Algebraic Laws for languages_ L [M = M [ L.Union is commutative._ (L [M) [ N = L [ (M [ N).Union is associative._ (LM)N = L(MN).Concatenation is associativeNote: Concatenation is not commutative, i.e.,there are L and M such that LM 6= ML.76_ ; [ L = L [ ; = L.; is identity for union._ f_gL = Lf_g = L.f_g is left and right identity for concatenation._ ;L = L; = ;.; is left and right annihilator for concatenation.77_ L(M [ N) = LM [ LN.Concatenation is left distributive over union._ (M [ N)L = ML [ NL.Concatenation is right distributive over union._ L [ L = L.Union is idempotent._ ;_ = f_g, f_g_ = f_g._ L+ = LL_ = L_L, L_ = L+ [ f_g78_ (L_)_ = L_. Closure is idempotentProof:w 2 (L_)_ () w 21[i=01[j=0Lj!i() 9k;m 2 N : w 2 (Lm)k() 9p 2 N : w 2 Lp() w 21[i=0Li() w 2 L_ _79Algebraic Laws for regex'sEvidently e.g. L((0+1)1) = L(01+11)Also e.g. L((00+101)11) = L(0011+10111).More generallyL((E +F)G) = L(EG+FG)for any regex's E, F, and G._ How do we verify that a general identity likeabove is true?1. Prove it by hand.2. Let the computer prove it.80In Chapter 4 we will learn how to test automaticallyif E = F, for any concrete regex'sE and F.We want to test general identities, such asE +F = F +E, for any regex's E and F.Method:1. \Freeze" E to a1, and F to a22. Test automatically if the frozen identity istrue, e.g. if L(a1 +a2) = L(a2 +a1)Question: Does this always work?81Answer: Yes, as long as the identities use onlyplus, dot, and star.Let's denote a generalized regex, such as (E +F)EbyE(E;F)Now we can for instance make the substitutionS = fE=0;F=11g to obtainS (E(E;F)) = (0+11)082Theorem 3.13: Fix a \freezing" substitution = fE1=a1; E2=a2; : : : ; Em=amg.Let E(E1; E2; : : : ; Em) be a generalized regex.Then for any regex's E1;E2; : : : ;Em,w 2 L(E(E1;E2; : : : ;Em))if and only if there are strings wi 2 L(Ei), s.t.w = wj1wj2 _ _ _wjkandaj1aj2 _ _ _ ajk 2 L(E(a1; a2; : : : ; am))83For example: Suppose the alphabet is f1; 2g.Let E(E1; E2) be (E1 + E2)E1, and let E1 be 1,and E2 be 2. Thenw 2 L(E(E1;E2)) = L((E1 +E2)E1) =(f1g [ f2g)f1g = f11; 21gif and only if9w1 2 L(E1) = f1g; 9w2 2 L(E2) = f2g : w = wj1wj2andaj1aj2 2 L(E(a1; a2))) = L((a1+a2)a1) = fa1a1; a2a1gif and only ifj1 = j2 = 1, or j1 = 1, and j2 = 284Proof of Theorem 3.13: We do a structuralinduction of E.Basis: If E = _, the frozen expression is also _.If E = ;, the frozen expression is also ;.If E = a, the frozen expression is also a. Noww 2 L(E) if and only if there is u 2 L(a), s.t.w = u and u is in the language of the frozenexpression, i.e. u 2 fag.85Induction:Case 1: E = F+G.Then (E) = (F)+(G), andL((E)) = L((F)) [ L((G))Let E and and F be regex's. Then w 2 L(E +F)if and only if w 2 L(E) or w 2 L(F), if and onlyif a1 2 L((F)) or a2 2 L((G)), if and only ifa1 2 (E), or a2 2 (E).Case 2: E = F:G.Then (E) = (F):(G), andL((E)) = L((F)):L((G))Let E and and F be regex's. Then w 2 L(E:F )if and only if w = w1w2, w1 2 L(E) and w2 2 L(F),and a1a2 2 L((F)):L((G)) = (E)Case 3: E = F_.Prove this case at home.86Examples:To prove (L+M)_ = (L_M_)_ it is enough todetermine if (a1+a2)_ is equivalent to (a_1a_2)_To verify L_ = L_L_ test if a_1 is equivalent toa_1a_1.Question: Does L+ML = (L+M)L hold?87Theorem 3.14: E(E1; : : : ; Em) = F(E1; : : : ; Em) ,L((E)) = L((F))Proof:(Only if direction) E(E1; : : : ; Em) = F(E1; : : : ; Em)means that L(E(E1; : : : ;Em)) = L(F(E1; : : : ;Em))for any concrete regex's E1; : : : ;Em. In particularthen L((E)) = L((F))(If direction) Let E1; : : : ;Em be concrete regex's.Suppose L((E)) = L((F)). Then by Theorem3.13,w 2 L(E(E1; : : :Em)) ,9wi 2 L(Ei);w = wj1 _ _ _wjm; aj1 _ _ _ ajm 2 L((E)) ,9wi 2 L(Ei);w = wj1 _ _ _wjm; aj1 _ _ _ ajm 2 L((F)) ,w 2 L(F(E1; : : :Em))88Properties of Regular Languages_ Pumping Lemma. Every regular languagesatis_es the pumping lemma. If somebodypresents you with fake regular language, usethe pumping lemma to show a contradiction._ Closure properties. Building automata fromcomponents through operations, e.g. given Land M we can build an automaton for L \M._ Decision properties. Computational analysisof automata, e.g. are two automata equivalent._ Minimization techniques. We can save moneysince we can build smaller machines.89The Pumping Lemma InformallySuppose L01 = f0n1n : n _ 1g were regular.Then it would be recognized by some DFA A,with, say, k states.Let A read 0k. On the way it will travel asfollows:_ p00 p100 p2: : : : : :0k pk)9i < j : pi = pj Call this state q.90Now you can fool A:If ^_(q; 1i) 2 F the machine will foolishly accept0j1i.If ^_(q; 1i) =2 F the machine will foolishly reject0i1i.Therefore L01 cannot be regular._ Let's generalize the above reasoning.91Theorem 4.1.The Pumping Lemma for Regular Languages.Let L be regular.Then 9n; 8w 2 L : jwj _ n ) w = xyz such that1. y 6= _2. jxyj _ n3. 8k _ 0; xykz 2 L92Proof: Suppose L is regularThe L is recognized by some DFA A with, say,n states.Let w = a1a2 : : : am 2 L; m > n.Let pi = ^_(q0; a1a2 _ _ _ ai).)9i < j : pi = pj93Now w = xyz, where1. x = a1a2 _ _ _ ai2. y = ai+1ai+2 _ _ _ aj3. z = aj+1aj+2 : : : amStartpi p0a1 . . . aiai+1 . . . ajaj+1 . . . amx = z =y =Evidently xykz 2 L, for any k _ 0. Q:E:D:94Example: Let Leq be the language of stringswith equal number of zero's and one's.Suppose Leq is regular. Then w = 0n1n 2 L.By the pumping lemma w = xyz, jxyj _ n,y 6= _ and xykz 2 Leqw = 0| 00{z_ _}_x _|_{_z0}y|0111{z_ _ _ 11}zIn particular, xz 2 Leq, but xz has fewer 0'sthan 1's.95Suppose Lpr = f1p : p is prime g were regular.Let n be given by the pumping lemma.Choose a prime p _ n+2.w =z }p| {|111{z_ _}_x _|_{_z1}yjyj=m|1111{z_ _ _ 11}zNow xypmz 2 Lprjxypmzj = jxzj+(p m)jyj =p m+(p m)m = (1+m)(p m)which is not prime unless one of the factorsis 1._ y 6= _ ) 1+m > 1_ m = jyj _ jxyj _ n; p _ n+2) p m _ n+2 n = 2.96Closure Properties of Regular LanguagesLet L and M be regular languages. Then thefollowing languages are all regular:_ Union: L [M_ Intersection: L \M_ Complement: N_ Di_erence: L nM_ Reversal: LR = fwR : w 2 Lg_ Closure: L_._ Concatenation: L:M_ Homomorphism:h(L) = fh(w) : w 2 L; h is a homom. g_ Inverse homomorphism:h1(L) = fw 2 _ : h(w) 2 L; h : _ ! _ is a homom. g97Theorem 4.4. For any regular L and M, L[Mis regular.Proof. Let L = L(E) and M = L(F). ThenL(E +F) = L [M by de_nition.Theorem 4.5. If L is a regular language over_, then so is L = __ n L.Proof. Let L be recognized by a DFAA = (Q;_; _; q0; F):Let B = (Q;_; _; q0;Q n F). Now L(B) = L.98Example:Let L be recognized by the DFA belowStart{q {q {q 0 0 0 , q1 , q2 } }0 11 001}Then L is recognized by1 0Start{q {q {q 0 0 0 , q1 , q2 } }0 1}10Question: What are the regex's for L and L99Theorem 4.8. If L and M are regular, thenso is L \M.Proof. By DeMorgan's law L \ M = L [M.We already that regular languages are closedunder complement and union.We shall shall also give a nice direct proof, theCartesian construction from the e-commerceexample.100Theorem 4.8. If L and M are regular, thenso in L \M.Proof. Let L be the language ofAL = (QL;_; _L; qL; FL)and M be the language ofAM = (QM;_; _M; qM; FM)We assume w.l.o.g. that both automata aredeterministic.We shall construct an automaton that simulatesAL and AM in parallel, and accepts if andonly if both AL and AM accept.101If AL goes from state p to state s on reading a,and AM goes from state q to state t on readinga, then AL\M will go from state (p; q) to state(s; t) on reading a.StartInputAND AcceptaLMAA102FormallyAL\M = (QL_QM;_; _L\M; (qL; qM); FL_FM);where_L\M((p; q); a) = (_L(p; a); _M(q; a))It will be shown in the tutorial by and inductionon jwj that^_L\M((qL; qM);w) =_^_L(qL;w); ^_M(qM;w)_The claim then follows.Question: Why?103Example: (c) = (a) _ (b)StartStart10 0,11 0,10(a)(b)Start0,1p qr spr psqr qs011001(c)104Theorem 4.10. If L and M are regular languages,then so in L nM.Proof. Observe that L n M = L \ M. Wealready know that regular languages are closedunder complement and intersection.105Theorem 4.11. If L is a regular language,then so is LR.Proof 1: Let L be recognized by an FA A.Turn A into an FA for LR, by1. Reversing all arcs.2. Make the old start state the new sole acceptingstate.3. Create a new start state p0, with _(p0; _) = F(the old accepting states).106Theorem 4.11. If L is a regular language,then so is LR.Proof 2: Let L be described by a regex E.We shall construct a regex ER, such thatL(ER) = (L(E))R.We proceed by a structural induction on E.Basis: If E is _, ;, or a, then ER = E.Induction:1. E = F +G. Then ER = FR +GR2. E = F:G. Then ER = GR:FR3. E = F_. Then ER = (FR)_We will show by structural induction on E onblackboard in class thatL(ER) = (L(E))R107HomomorphismsA homomorphism on _ is a function h : __ ! __,where _ and _ are alphabets.Let w = a1a2 _ _ _ an 2 __. Thenh(w) = h(a1)h(a2) _ _ _ h(an)andh(L) = fh(w) : w 2 LgExample: Let h : f0; 1g_ ! fa; bg_ be de_ned byh(0) = ab, and h(1) = _. Now h(0011) = abab.Example: h(L(10_1)) = L((ab)_).108Theorem 4.14: h(L) is regular, whenever Lis.Proof:Let L = L(E) for a regex E. We claim thatL(h(E)) = h(L).Basis: If E is _ or ;. Then h(E) = E, andL(h(E)) = L(E) = h(L(E)).If E is a, then L(E) = fag, L(h(E)) = L(h(a)) =fh(a)g = h(L(E)).Induction:Case 1: L = E + F. Now L(h(E + F)) =L(h(E)+h(F)) = L(h(E))[L(h(F)) = h(L(E))[h(L(F)) = h(L(E) [ L(F)) = h(L(E +F)).Case 2: L = E:F . Now L(h(E:F )) = L(h(E)):L(h(F))= h(L(E)):h(L(F)) = h(L(E):L(F))Case 3: L = E_. Now L(h(E_)) = L(h(E)_) =L(h(E))_ = h(L(E))_ = h(L(E_)).109Inverse HomomorphismLet h : __ ! __ be a homom. Let L _ __,and de_neh1(L) = fw 2 __ : h(w) 2 LgL h(L)h-1 (L) L(a)(b)hh110Example: Let h : fa; bg ! f0; 1g_ be de_ned byh(a) = 01, and h(b) = 10. If L = L((00+1)_),then h1(L) = L((ba)_).Claim: h(w) 2 L if and only if w = (ba)nProof: Let w = (ba)n. Then h(w) = (1001)n 2L.Let h(w) 2 L, and suppose w =2 L((ba)_). Thereare four cases to consider.1. w begins with a. Then h(w) begins with01 and =2 L((00+1)_).2. w ends in b. Then h(w) ends in 10 and=2 L((00+1)_).3. w = xaay. Then h(w) = z0101v and =2L((00+1)_).4. w = xbby. Then h(w) = z1010v and =2L((00+1)_).111Theorem 4.16: Let h : __ ! __ be a homom.,and L _ __ regular. Then h1(L) isregular.Proof: Let L be the language of A = (Q;_; _; q0; F).We de_ne B = (Q;_; ; q0; F), where(q; a) = ^_(q; h(a))It will be shown by induction on jwj in the tutorialthat ^(q0;w) = ^_(q0; h(w))Start h(a) to AAccept/rejectInput ahAInput112Decision PropertiesWe consider the following:1. Converting among representations for regularlanguages.2. Is L = ;?3. Is w 2 L?4. Do two descriptions de_ne the same language?113From NFA's to DFA'sSuppose the _-NFA has n states.To compute ECLOSE(p) we follow at most n2arcs.The DFA has 2n states, for each state S andeach a 2 _ we compute _D(S; a) in n3 steps.Grand total is O(n32n) steps.If we compute _ for reachable states only, weneed to compute _D(S; a) only s times, where sis the number of reachable states. Grand totalis O(n3s) steps.114From DFA to NFAAll we need to do is to put set brackets aroundthe states. Total O(n) steps.From FA to regexWe need to compute n3 entries of size up to4n. Total is O(n34n).The FA is allowed to be a NFA. If we _rstwanted to convert the NFA to a DFA, the totaltime would be doubly exponentialFrom regex to FA's We can build an expressiontree for the regex in n steps.We can construct the automaton in n steps.Eliminating _-transitions takes O(n3) steps.If you want a DFA, you might need an exponentialnumber of steps.115Testing emptinessL(A) 6= ; for FA A if and only if a _nal stateis reachable from the start state in A. TotalO(n2) steps.Alternatively, we can inspect a regex E and tellif L(E) = ;. We use the following method:E = F + G. Now L(E) is empty if and only ifboth L(F) and L(G) are empty.E = F:G. Now L(E) is empty if and only ifeither L(F) or L(G) is empty.E = F_. Now L(E) is never empty, since _ 2L(E).E = _. Now L(E) is not empty.E = a. Now L(E) is not empty.E = ;. Now L(E) is empty.116Testing membershipTo test w 2 L(A) for DFA A, simulate A on w.If jwj = n, this takes O(n) steps.If A is an NFA and has s states, simulating Aon w takes O(ns2) steps.If A is an _-NFA and has s states, simulatingA on w takes O(ns3) steps.If L = L(E), for regex E of length s, we _rstconvert E to an _-NFA with 2s states. Then wesimulate w on this machine, in O(ns3) steps.117Equivalence and Minimization of AutomataLet A = (Q;_; _; q0; F) be a DFA, and fp; qg _ Q.We de_nep _ q , 8w 2 __ : ^_(p;w) 2 F i_ ^_(q;w) 2 F_ If p _ q we say that p and q are equivalent_ If p 6_ q we say that p and q are distinguishableIOW (in other words) p and q are distinguishablei_9w : ^_(p;w) 2 F and ^_(q;w) =2 F; or vice versa118Example:Start00110101101 001 10A B C DE F G H^_(C; _) 2 F; ^_(G; _) =2 F ) C 6_ G^_(A; 01) = C 2 F; ^_(G; 01) = E =2 F ) A 6_ G119What about A and E?Start00110101101 001 10A B C DE F G H^_(A; _) = A =2 F; ^_(E; _) = E =2 F^_(A; 1) = F = ^_(E; 1)Therefore ^_(A; 1x) = ^_(E; 1x) = ^_(F; x)^_(A; 00) = G = ^_(E; 00)^_(A; 01) = C = ^_(E; 01)Conclusion: A _ E.120We can compute distinguishable pairs with thefollowing inductive table _lling algorithm:Basis: If p 2 F and q 62 F, then p 6_ q.Induction: If 9a 2 _ : _(p; a) 6_ _(q; a),then p 6_ q.Example: Applying the table _lling algo to A:BCDEFGHA B C D E F Gxxxxxxxxxxxxxxxxxxxxxxxx x121Theorem 4.20: If p and q are not distinguishedby the TF-algo, then p _ q.Proof: Suppose to the contrary that that thereis a bad pair fp; qg, s.t.1. 9w : ^_(p;w) 2 F; ^_(q;w) =2 F, or vice versa.2. The TF-algo does not distinguish betweenp and q.Let w = a1a2 _ _ _ an be the shortest string thatidenti_es a bad pair fp; qg.Now w 6= _ since otherwise the TF-algo wouldin the basis distinguish p from q. Thus n _ 1.122Consider states r = _(p; a1) and s = _(q; a1).Now fr; sg cannot be a bad pair since fr; sgwould be indenti_ed by a string shorter than w.Therefore, the TF-algo must have discoveredthat r and s are distinguishable.But then the TF-algo would distinguish p fromq in the inductive part.Thus there are no bad pairs and the theoremis true.123Testing Equivalence of Regular LanguagesLet L and M be reg langs (each given in someform).To test if L = M1. Convert both L and M to DFA's.2. Imagine the DFA that is the union of thetwo DFA's (never mind there are two startstates)3. If TF-algo says that the two start statesare distinguishable, then L 6= M, otherwiseL = M.124Example:StartStart001101 0110A BC DEWe can \see" that both DFA acceptL(_+(0+1)_0). The result of the TF-algo isBCDEA B C Dxxxxx xTherefore the two automata are equivalent.125Minimization of DFA'sWe can use the TF-algo to minimize a DFAby merging all equivalent states. IOW, replaceeach state p by p=_.Example: The DFA on slide 119 has equivalenceclasses ffA;Eg; fB;Hg; fCg; fD; Fg; fGgg.The \union" DFA on slide 125 has equivalenceclasses ffA;C;Dg; fB;Egg.Note: In order for p=_to be an equivalenceclass, the relation _ has to be an equivalencerelation (reexive, symmetric, and transitive).126Theorem 4.23: If p _ q and q _ r, then p _ r.Proof: Suppose to the contrary that p 6_ r.Then 9w such that ^_(p;w) 2 F and ^_(r;w) 62 F,or vice versa.OTH, ^_(q;w) is either accpeting or not.Case 1: ^_(q;w) is accepting. Then q 6_ r.Case 1: ^_(q;w) is not accepting. Then p 6_ q.The vice versa case is proved symmetricallyTherefore it must be that p _ r.127To minimize a DFA A = (Q;_; _; q0; F) constructa DFA B = (Q=_;_; ; q0=_; F=_), where(p=_; a) = _(p; a)=_In order for B to be well de_ned we have toshow thatIf p _ q then _(p; a) _ _(q; a)If _(p; a) 6_ _(q; a), then the TF-algo would concludep 6_ q, so B is indeed well de_ned. Notealso that F=_contains all and only the acceptingstates of A.128Example: We can minimizeStart00110101101 001 10A B C DE F G Hto obtainStart100110101A,E 0G D,FB,H C129NOTE:We cannot apply the TF-algo to NFA's.For example, to minimizeStart0,101 0A BCwe simply remove state C.However, A 6_ C.130Why the Minimized DFA Can't Be BeatenLet B be the minimized DFA obtained by applyingthe TF-algo to DFA A.We already know that L(A) = L(B).What if there existed a DFA C, withL(C) = L(B) and fewer states than B?Then run the TF-algo on B \union" C.Since L(B) = L(C) we have qB0 _ qC0 .Also, _(qB0 ; a) _ _(qC0 ; a), for any a.131Claim: For each state p in B there is at leastone state q in C, s.t. p _ q.Proof of claim: There are no inaccessible states,so p = ^_(qB0 ; a1a2 _ _ _ ak), for some string a1a2 _ _ _ ak.Now q = ^_(qC0 ; a1a2 _ _ _ ak), and p _ q.Since C has fewer states than B, there must betwo states r and s of B such that r _ t _ s, forsome state t of C. But then r _ s (why?)which is a contradiction, since B was constructedby the TF-algo.132Context-Free Grammars and Languages_ We have seen that many languages cannotbe regular. Thus we need to consider largerclasses of langs._ Contex-Free Languages (CFL's) played a centralrole natural languages since the 1950's,and in compilers since the 1960's._ Context-Free Grammars (CFG's) are the basisof BNF-syntax._ Today CFL's are increasingly important forXML and their DTD's.We'll look at: CFG's, the languages they generate,parse trees, pushdown automata, andclosure properties of CFL's.133Informal example of CFG'sConsider Lpal = fw 2 __ : w = wRgFor example otto 2 Lpal; madamimadam 2 Lpal.In Finnish language e.g. saippuakauppias 2 Lpal(\soap-merchant")Let _ = f0; 1g and suppose Lpal were regular.Let n be given by the pumping lemma. Then0n10n 2 Lpal. In reading 0n the FA must makea loop. Omit the loop; contradiction.Let's de_ne Lpal inductively:Basis: _; 0, and 1 are palindromes.Induction: If w is a palindrome, so are 0w0and 1w1.Circumscription: Nothing else is a palindrome.134CFG's is a formal mechanism for de_nitionssuch as the one for Lpal.1: P ! _2: P ! 03: P ! 14: P ! 0P05: P ! 1P10 and 1 are terminalsP is a variable (or nonterminal, or syntacticcategory)P is in this grammar also the start symbol.1{5 are productions (or rules)135Formal de_nition of CFG'sA context-free grammar is a quadrupleG = (V; T; P; S)whereV is a _nite set of variables.T is a _nite set of terminals.P is a _nite set of productions of the formA ! _, where A is a variable and _ 2 (V [ T)_S is a designated variable called the start symbol.136Example: Gpal = (fPg; f0; 1g; A; P), where A =fP ! _; P ! 0; P ! 1; P ! 0P0; P ! 1P1g.Sometimes we group productions with the samehead, e.g. A = fP ! _j0j1j0P0j1P1g.Example: Regular expressions over f0; 1g canbe de_ned by the grammarGregex = (fEg; f0; 1g; A;E)where A =fE ! 0;E ! 1;E ! E:E;E ! E+E;E ! E?;E ! (E)g137Example: (simple) expressions in a typical proglang. Operators are + and *, and argumentsare ident_ers, i.e. strings inL((a+b)(a+b+0+1)_)The expressions are de_ned by the grammarG = (fE; Ig; T; P;E)where T = f+; _; (; ); a; b; 0; 1g and P is the followingset of productions:1: E ! I2: E ! E +E3: E ! E _ E4: E ! (E)5: I ! a6: I ! b7: I ! Ia8: I ! Ib9: I ! I010: I ! I1138Derivations using grammars_ Recursive inference, using productions frombody to head_ Derivations, using productions from head tobody.Example of recursive inference:String Lang Prod String(s) used(i) a I 5 -(ii) b I 6 -(iii) b0 I 9 (ii)(iv) b00 I 9 (iii)(v) a E 1 (i)(vi) b00 E 1 (iv)(vii) a+b00 E 2 (v), (vi)(viii) (a+b00) E 4 (vii)(ix) a _ (a+b00) E 3 (v), (viii)139Let G = (V; T; P; S) be a CFG, A 2 V ,f_; _g _ (V [ T)_, and A ! 2 P.Then we write_A_ )G__or, if G is understood_A_ ) __and say that _A_ derives __.We de_ne _) to be the reexive and transitiveclosure of ), IOW:Basis: Let _ 2 (V [ T)_. Then _ _) _.Induction: If _ _) _, and _ ) , then _ _) .140Example: Derivation of a _ (a+b00) from E inthe grammar of slide 138:E ) E _ E ) I _ E ) a _ E ) a _ (E) )a_(E+E) ) a_(I+E) ) a_(a+E) ) a_(a+I) )a _ (a+I0) ) a _ (a+I00) ) a _ (a+b00)Note: At each step we might have several rulesto choose from, e.g.I _ E ) a _ E ) a _ (E), versusI _ E ) I _ (E) ) a _ (E):Note2: Not all choices lead to successful derivationsof a particular string, for instanceE ) E +Ewon't lead to a derivation of a _ (a+b00).141Leftmost and Rightmost DerivationsLeftmost derivation )lm: Always replace the leftmostvariable by one of its rule-bodies.Rightmost derivation )rm: Always replace therightmost variable by one of its rule-bodies.Leftmost: The derivation on the previous slide.Rightmost:E )rmE _ E )rmE_(E) )rmE_(E+E) )rmE_(E+I) )rmE_(E+I0))rmE_(E+I00) )rmE_(E+b00) )rmE_(I+b00))rmE _ (a+b00) )rmI _ (a+b00) )rma _ (a+b00)We can conclude that E _)rma _ (a+b00)142The Language of a GrammarIf G(V; T; P; S) is a CFG, then the language ofG isL(G) = fw 2 T_ : S _)Gwgi.e. the set of strings over T_ derivable fromthe start symbol.If G is a CFG, we call L(G) acontext-free language.Example: L(Gpal) is a context-free language.Theorem 5.7:L(Gpal) = fw 2 f0; 1g_ : w = wRgProof: (_-direction.) Suppose w = wR. Weshow by induction on jwj that w 2 L(Gpal)143Basis: jwj = 0, or jwj = 1. Then w is _; 0;or 1. Since P ! _; P ! 0; and P ! 1 areproductions, we conclude that P _)Gw in allbase cases.Induction: Suppose jwj _ 2. Since w = wR,we have w = 0x0, or w = 1x1, and x = xR.If w = 0x0 we know from the IH that P _) x.ThenP ) 0P0 _) 0x0 = wThus w 2 L(Gpal).The case for w = 1x1 is similar.144(_-direction.) We assume that w 2 L(Gpal)and must show that w = wR.Since w 2 L(Gpal), we have P _) w.We do an induction of the length of _).Basis: The derivation P _) w is done in onestep.Then w must be _; 0; or 1, all palindromes.Induction: Let n _ 1, and suppose the derivationtakes n+1 steps. Then we must havew = 0x0 _( 0P0 ( Porw = 1x1 _( 1P1 ( Pwhere the second derivation is done in n steps.By the IH x is a palindrome, and the inductiveproof is complete.145Sentential FormsLet G = (V; T; P; S) be a CFG, and _ 2 (V [T)_.IfS _) _we say that _ is a sentential form.If S )lm_ we say that _ is a left-sentential form,and if S )rm_ we say that _ is a right-sententialformNote: L(G) is those sentential forms that arein T_.146Example: Take G from slide 138. Then E _ (I +E)is a sentential form sinceE ) E_E ) E_(E) ) E_(E+E) ) E_(I+E)This derivation is neither leftmost, nor rightmostExample: a _ E is a left-sentential form, sinceE )lmE _ E )lmI _ E )lma _ EExample: E_(E+E) is a right-sentential form,sinceE )rmE _ E )rmE _ (E) )rmE _ (E +E)147Parse Trees_ If w 2 L(G), for some CFG, then w has aparse tree, which tells us the (syntactic) structureof w_ w could be a program, a SQL-query, an XMLdocument,etc._ Parse trees are an alternative representationto derivations and recursive inferences._ There can be several parse trees for the samestring_ Ideally there should be only one parse tree(the \true" structure) for each string, i.e. thelanguage should be unambiguous._ Unfortunately, we cannot always remove theambiguity.148Constructing Parse TreesLet G = (V; T; P; S) be a CFG. A tree is a parsetree for G if:1. Each interior node is labelled by a variablein V .2. Each leaf is labelled by a symbol in V [ T [ f_g.Any _-labelled leaf is the only child of itsparent.3. If an interior node is lablelled A, and itschildren (from left to right) labelledX1;X2; : : : ;Xk;then A ! X1X2 : : :Xk 2 P.149Example: In the grammar1: E ! I2: E ! E +E3: E ! E _ E4: E ! (E)___the following is a parse tree:EE + EIThis parse tree shows the derivation E _) I+E150Example: In the grammar1: P ! _2: P ! 03: P ! 14: P ! 0P05: P ! 1P1the following is a parse tree:PPP0 01 1It shows the derivation of P _) 0110.151The Yield of a Parse TreeThe yield of a parse tree is the string of leavesfrom left to right.Important are those parse trees where:1. The yield is a terminal string.2. The root is labelled by the start symbolWe shall see the the set of yields of theseimportant parse trees is the language of thegrammar.152Example: Below is an important parse treeEE * EIaEE EIaIIIb( )+00The yield is a _ (a+b00).Compare the parse tree with the derivation onslide 141.153Let G = (V; T; P; S) be a CFG, and A 2 V .We are going to show that the following areequivalent:1. We can determine by recursive inferencethat w is in the language of A2. A _) w3. A _)lmw, and A _)rmw4. There is a parse tree of G with root A andyield w.To prove the equivalences, we use the followingplan.RecursivetreeParseinferenceLeftmostderivationRightmostDerivation derivation154From Inferences to TreesTheorem 5.12: Let G = (V; T; P; S) be aCFG, and suppose we can show w to be inthe language of a variable A. Then there is aparse tree for G with root A and yield w.Proof: We do an induction of the length ofthe inference.Basis: One step. Then we must have used aproduction A ! w. The desired parse tree isthenAw155Induction: w is inferred in n + 1 steps. Supposethe last step was based on a productionA ! X1X2 _ _ _Xk;where Xi 2 V [ T. We break w up asw1w2 _ _ _wk;where wi = Xi, when Xi 2 T, and when Xi 2 V;then wi was previously inferred being in Xi, inat most n steps.By the IH there are parse trees i with root Xiand yield wi. Then the following is a parse treefor G with root A and yield w:AX X Xw w wkk1 21 2 . . .. . .156From trees to derivationsWe'll show how to construct a leftmost derivationfrom a parse tree.Example: In the grammar of slide 6 there clearlyis a derivationE ) I ) Ib ) ab:Then, for any _ and _ there is a derivation_E_ ) _I_ ) _Ib_ ) _ab_:For example, suppose we have a derivationE ) E +E ) E +(E):The we can choose _ = E + ( and _ =) andcontinue the derivation asE +(E) ) E +(I) ) E +(Ib) ) E +(ab):This is why CFG's are called context-free.157Theorem 5.14: Let G = (V; T; P; S) be aCFG, and suppose there is a parse tree withroot labelled A and yield w. Then A _)lmw in G.Proof: We do an induction on the height ofthe parse tree.Basis: Height is 1. The tree must look likeAwConsequently A ! w 2 P, and A )lmw.158Induction: Height is n + 1. The tree mustlook likeAX X Xw w wkk1 21 2 . . .. . .Then w = w1w2 _ _ _wk, where1. If Xi 2 T, then wi = Xi.2. If Xi 2 V , then Xi _)lmwi in G by the IH.159Now we construct A _)lmw by an (inner) inductionby showing that8i : A _)lmw1w2 _ _ _wiXi+1Xi+2 _ _ _Xk:Basis: Let i = 0. We already know thatA )lmX1Xi+2 _ _ _Xk:Induction: Make the IH thatA _)lmw1w2 _ _ _wi1XiXi+1 _ _ _Xk:(Case 1:) Xi 2 T. Do nothing, since Xi = wigives usA _)lmw1w2 _ _ _wiXi+1 _ _ _Xk:160(Case 2:) Xi 2 V . By the IH there is a derivationXi )lm_1 )lm_2 )lm _ _ _ )lmwi. By the contexfreeproperty of derivations we can proceedwithA _)lmw1w2 _ _ _wi1XiXi+1 _ _ _Xk )lmw1w2 _ _ _wi1_1Xi+1 _ _ _Xk )lmw1w2 _ _ _wi1_2Xi+1 _ _ _Xk )lm_ _ _w1w2 _ _ _wi1wiXi+1 _ _ _Xk161Example: Let's construct the leftmost derivationfor the treeEE * EIaEE EIaIIIb( )+00Suppose we have inductively constructed theleftmost derivationE )lmI )lmacorresponding to the leftmost subtree, and theleftmost derivationE )lm(E) )lm(E +E) )lm(I +E) )lm(a+E) )lm(a+I) )lm(a+I0) )lm(a+I00) )lm(a+b00)corresponding to the righmost subtree.162For the derivation corresponding to the wholetree we start with E )lmE _ E and expand the_rst E with the _rst derivation and the secondE with the second derivation:E )lmE _ E )lmI _ E )lma _ E )lma _ (E) )lma _ (E +E) )lma _ (I +E) )lma _ (a+E) )lma _ (a+I) )lma _ (a+I0) )lma _ (a+I00) )lma _ (a+b00)163From Derivations to Recursive InferencesObservation: Suppose that A ) X1X2 _ _ _Xk _) w.Then w = w1w2 _ _ _wk, where Xi _) wiThe factor wi can be extracted from A _) w bylooking at the expansion of Xi only.Example: E ) a _ b+a, andE ) E |{z}X1|{_z}X2E |{z}X3+ |{z}X4E |{z}X5We haveE ) E _ E ) E _ E +E ) I _ E +E ) I _ I +E )I _ I +I ) a _ I +I ) a _ b+I ) a _ b+aBy looking at the expansion of X3 = E only,we can extractE ) I ) b:164Theorem 5.18: Let G = (V; T; P; S) be aCFG. Suppose A _)Gw, and that w is a stringof terminals. Then we can infer that w is inthe language of variable A.Proof: We do an induction on the length ofthe derivation A _)Gw.Basis: One step. If A )Gw there must be aproduction A ! w in P. The we can infer thatw is in the language of A.165Induction: Suppose A _)Gw in n + 1 steps.Write the derivation asA )GX1X2 _ _ _Xk _)GwThe as noted on the previous slide we canbreak w as w1w2 _ _ _wk where Xi _)Gwi. Furthermore,Xi _)Gwi can use at most n steps.Now we have a production A ! X1X2 _ _ _Xk,and we know by the IH that we can infer wi tobe in the language of Xi.Therefore we can infer w1w2 _ _ _wk to be in thelanguage of A.166Ambiguity in Grammars and LanguagesIn the grammar1: E ! I2: E ! E +E3: E ! E _ E4: E ! (E)_ _ _the sentential form E + E _ E has two derivations:E ) E +E ) E +E _ EandE ) E _ E ) E +E _ EThis gives us two parse trees:+**+EE EE EEE EE E(a) (b)167The mere existence of several derivations is notdangerous, it is the existence of several parsetrees that ruins a grammar.Example: In the same grammar5: I ! a6: I ! b7: I ! Ia8: I ! Ib9: I ! I010: I ! I1the string a+b has several derivations, e.g.E ) E +E ) I +E ) a+E ) a+I ) a+bandE ) E +E ) E +I ) I +I ) I +b ) a+bHowever, their parse trees are the same, andthe structure of a+b is unambiguous.168De_nition: Let G = (V; T; P; S) be a CFG. Wesay that G is ambiguous is there is a string inT_ that has more than one parse tree.If every string in L(G) has at most one parsetree, G is said to be unambiguous.Example: The terminal string a+a _ a has twoparse trees:Ia IaIaIaIaIa+**+EE EE EEE EE E(a) (b)169Removing Ambiguity From GrammarsGood news: Sometimes we can remove ambiguity\by hand"Bad news: There is no algorithm to do itMore bad news: Some CFL's have only ambiguousCFG'sWe are studying the grammarE ! I j E +E j E _ E j (E)I ! a j b j Ia j Ib j I0 j I1There are two problems:1. There is no precedence between * and +2. There is no grouping of sequences of operators,e.g. is E + E + E meant to beE +(E +E) or (E +E)+E.170Solution: We introduce more variables, eachrepresenting expressions of same \binding strength."1. A factor is an expresson that cannot bebroken apart by an adjacent * or +. Ourfactors are(a) Identi_ers(b) A parenthesized expression.2. A term is an expresson that cannot be brokenby +. For instance a _ b can be brokenby a1_ or _a1. It cannot be broken by +,since e.g. a1+a _b is (by precedence rules)same as a1+(a _ b), and a _ b+a1 is sameas (a _ b)+a1.3. The rest are expressions, i.e. they can bebroken apart with * or +.171We'll let F stand for factors, T for terms, and Efor expressions. Consider the following grammar:1: I ! a j b j Ia j Ib j I0 j I12: F ! I j (E)3: T ! F j T _ F4: E ! T j E +TNow the only parse tree for a+a _ a will beFIaFIaTFIaT+*EE T172Why is the new grammar unambiguous?Intuitive explanation:_ A factor is either an identi_er or (E), forsome expression E._ The only parse tree for a sequencef1 _ f2 _ _ _ _ _ fn1 _ fnof factors is the one that gives f1 _f2 __ _ __fn1as a term and fn as a factor, as in the parsetree on the next slide._ An expression is a sequencet1 +t2 +_ _ _+tn1 +tnof terms ti. It can only be parsed witht1 +t2 +_ _ _+tn1 as an expression and tn asa term.173***TT FT FTT FF.. .174Leftmost derivations and AmbiguityThe two parse trees for a+a _ aIa IaIaIaIaIa+**+EE EE EEE EE E(a) (b)give rise to two derivations:E )lmE +E )lmI +E )lma+E )lma+E _ E)lma+I _ E )lma+a _ E )lma+a _ I )lma+a _ aandE )lmE _E )lmE+E _E )lmI +E _E )lma+E _E)lma+I _ E )lma+a _ E )lma+a _ I )lma+a _ a175In General:_ One parse tree, but many derivations_ Many leftmost derivation implies many parsetrees._ Many rightmost derivation implies many parsetrees.Theorem 5.29: For any CFG G, a terminalstring w has two distinct parse trees if and onlyif w has two distinct leftmost derivations fromthe start symbol.176Sketch of Proof: (Only If.) If the two parsetrees di_er, they have a node a which differentproductions, say A ! X1X2 _ _ _Xk andB ! Y1Y2 _ _ _ Ym. The corresponding leftmostderivations will use derivations based on thesetwo di_erent productions and will thus be distinct.(If.) Let's look at how we construct a parsetree from a leftmost derivation. It should nowbe clear that two distinct derivations gives riseto two di_erent parse trees.177Inherent AmbiguityA CFL L is inherently ambiguous if all grammarsfor L are ambiguous.Example: Consider L =fanbncmdm : n _ 1;m _ 1g[fanbmcmdn : n _ 1;m _ 1g:A grammar for L isS ! AB j CA ! aAb j abB ! cBd j cdC ! aCd j aDdD ! bDc j bc178Let's look at parsing the string aabbccdd.SA Ba A ba bc B dc d(a)SCa C da D db D cb c(b)179From this we see that there are two leftmostderivations:S )lmAB )lmaAbB )lmaabbB )lmaabbcBd )lmaabbccddandS )lmC )lmaCd )lmaaDdd )lmaabDcdd )lmaabbccddIt can be shown that every grammar for L behaveslike the one above. The language L isinherently ambiguous.180Pushdown AutomataA pushdown automata (PDA) is essentially an_-NFA with a stack.On a transition the PDA:1. Consumes an input symbol.2. Goes to a new state (or stays in the old).3. Replaces the top of the stack by any string(does nothing, pops the stack, or pushes astring onto the stack)StackFinitestatecontrolInput Accept/reject181Example: Let's considerLwwr = fwwR : w 2 f0; 1g_g;with \grammar" P ! 0P0; P ! 1P1; P ! _.A PDA for Lwwr has tree states, and operatesas follows:1. Guess that you are reading w. Stay instate 0, and push the input symbol ontothe stack.2. Guess that you're in the middle of wwR.Go spontanteously to state 1.3. You're now reading the head of wR. Compareit to the top of the stack. If theymatch, pop the stack, and remain in state 1.If they don't match, go to sleep.4. If the stack is empty, go to state 2 andaccept.182The PDA for Lwwr as a transition diagram:1 ,, Z 0 Z 0 Z 0 Z 0 , /1 , 0 / 1 00 , 1 / 0 10 , 0 / 0 01 , Z 0 Z 00 , Z 0 Z 0 / 0, 0 / 0, 1 / 10 , 0 / q q q 0 1 21 / 1 1/Start1 , 1 / / 1183PDA formallyA PDA is a seven-tuple:P = (Q;_; ; _; q0;Z0; F);where_ Q is a _nite set of states,_ _ is a _nite input alphabet,_ is a _nite stack alphabet,_ _ : Q__[f_g_ ! 2Q__ is the transitionfunction,_ q0 is the start state,_ Z0 2 is the start symbol for the stack,and_ F _ Q is the set of accepting states.184Example: The PDA1 ,, Z 0 Z 0 Z 0 Z 0 , /1 , 0 / 1 00 , 1 / 0 10 , 0 / 0 01 , Z 0 Z 00 , Z 0 Z 0 / 0, 0 / 0, 1 / 10 , 0 / q q q 0 1 21 / 1 1/Start1 , 1 / / 1is actually the seven-tupleP = (fq0; q1; q2g; f0; 1g; f0; 1;Z0g; _; q0;Z0; fq2g);where _ is given by the following table (setbrackets missing):0;Z0 1;Z0 0,0 0,1 1,0 1,1 _;Z0 _; 0 _; 1! q0 q0; 0Z0 q0; 1Z0 q0; 00 q0; 01 q0; 10 q0; 11 q1;Z0 q1; 0 q1; 1q1 q1; _ q1; _ q2;Z0?q2185Instantaneous DescriptionsA PDA goes from con_guration to con_gurationwhen consuming input.To reason about PDA computation, we useinstantaneous descriptions of the PDA. An IDis a triple(q;w; )where q is the state, w the remaining input,and the stack contents.Let P = (Q;_; ; _; q0;Z0; F) be a PDA. Then8w 2 __; _ 2 _ :(p; _) 2 _(q; a;X) ) (q; aw;X_) ` (p;w; __):We de_ne `_ to be the reexive-transitive closureof `.186Example: On input 1111 the PDA1 ,, Z 0 Z 0 Z 0 Z 0 , /1 , 0 / 1 00 , 1 / 0 10 , 0 / 0 01 , Z 0 Z 00 , Z 0 Z 0 / 0, 0 / 0, 1 / 10 , 0 / q q q 0 1 21 / 1 1/Start1 , 1 / / 1has the following computation sequences:1870 ) Z0 ) Z0 ) Z0 ) Z0 ) Z0 ) Z0 ) Z0 ) Zq( 2 ,q( 2 ,q( 2 ,0 ) Z0 ) Z0 ) Z0 ) Z0 ) 0 ) Z Z0 ) Z0 ) q Z 1q0q0q0q0q0q1q1q1q1q1q1q1q11111, 0 Z )111, 111, 111, 111, 11111111,111, 111, 111, 1111111,11,11,1, 1, 11 ,,( ,( ,( ,( ,( , , 1111( ,( ,( ,( ,( ,( ,( ,( ,( ,188The following properties hold:1. If an ID sequence is a legal computation fora PDA, then so is the sequence obtainedby adding an additional string at the endof component number two.2. If an ID sequence is a legal computation fora PDA, then so is the sequence obtained byadding an additional string at the bottomof component number three.3. If an ID sequence is a legal computationfor a PDA, and some tail of the input isnot consumed, then removing this tail fromall ID's result in a legal computation sequence.189Theorem 6.5: 8w 2 __; _ 2 _ :(q; x; _) `_ (p; y; _) ) (q; xw; _) `_ (p; yw; _):Proof: Induction on the length of the sequenceto the left.Note: If = _ we have proerty 1, and if w = _we have property 2.Note2: The reverse of the theorem is false.For property 3 we haveTheorem 6.6:(q; xw; _) `_ (p; yw; _) ) (q; x; _) `_ (p; y; _):190Acceptance by _nal stateLet P = (Q;_; ; _; q0;Z0; F) be a PDA. Thelanguage accepted by P by _nal state isL(P) = fw : (q0;w;Z0) `_ (q; _; _); q 2 Fg:Example: The PDA on slide 183 accepts exactlyLwwr.Let P be the machine. We prove that L(P) =Lwwr.(_-direction.) Let x 2 Lwwr. Then x = wwR,and the following is a legal computation sequence(q0;wwR;Z0) `_ (q0;wR;wRZ0) ` (q1;wR;wRZ0) `_(q1; _;Z0) ` (q2; _;Z0):191(_-direction.)Observe that the only way the PDA can enterq2 is if it is in state q1 with an empty stack.Thus it is su_cient to show that if (q0; x;Z0) `_(q1; _;Z0) then x = wwR, for some word w.We'll show by induction on jxj that(q0; x; _) `_ (q1; _; _) ) x = wwR:Basis: If x = _ then x is a palindrome.Induction: Suppose x = a1a2 _ _ _ an, where n > 0,and the IH holds for shorter strings.Ther are two moves for the PDA from ID (q0; x; _):192Move 1: The spontaneous (q0; x; _) ` (q1; x; _).Now (q1; x; _) `_ (q1; _; _) implies that j_j < j_j,which implies _ 6= _.Move 2: Loop and push (q0; a1a2 _ _ _ an; _) `(q0; a2 _ _ _ an; a1_).In this case there is a sequence(q0; a1a2 _ _ _ an; _) ` (q0; a2 _ _ _ an; a1_) ` _ _ _ `(q1; an; a1_) ` (q1; _; _).Thus a1 = an and(q0; a2 _ _ _ an; a1_) `_ (q1; an; a1_):By Theorem 6.6 we can remove an. Therefore(q0; a2 _ _ _ an1; a1_ `_ (q1; _; a1_):Then, by the IH a2 _ _ _ an1 = yyR. Then x =a1yyRan is a palindrome.193Acceptance by Empty StackLet P = (Q;_; ; _; q0;Z0; F) be a PDA. Thelanguage accepted by P by empty stack isN(P) = fw : (q0;w;Z0) `_ (q; _; _)g:Note: q can be any state.Question: How to modify the palindrome-PDAto accept by empty stack?194From Empty Stack to Final StateTheorem 6.9: If L = N(PN) for some PDAPN = (Q;_; ; _N; q0;Z0), then 9 PDA PF , suchthat L = L(PF ).Proof: LetPF = (Q [ fp0; pfg;_; [ fX0g; _F ; p0;X0; fpfg)where _F (p0; _;X0) = f(q0;Z0X0)g, and for allq 2 Q; a 2 _[f_g; Y 2 : _F (q; a; Y ) = _N(q; a; Y ),and in addition (pf ; _) 2 _F (q; _;X0).X 0 Z 0X 0 ,, X 0 / , X 0 / , X 0 / , X 0 / q/PNStartp0 0 pf195We have to show that L(PF ) = N(PN).(_direction.) Let w 2 N(PN). Then(q0;w;Z0) `_N(q; _; _);for some q. From Theorem 6.5 we get(q0;w;Z0X0) `_N(q; _;X0):Since _N _ _F we have(q0;w;Z0X0) `_F(q; _;X0):We conclude that(p0;w;X0) `F(q0;w;Z0X0) `_F(q; _;X0) `F(pf ; _; _):(_direction.) By inspecting the diagram.196Let's design PN for for cathing errors in stringsmeant to be in the if-else-grammar GS ! _jSSjiSjiSe:Here e.g. fieie; iie; ieig _ G; and e.g. fei; ieeiig \ G = ;:The diagram for PN isStartqi, Z/ZZe, Z/ Formally,PN = (fqg; fi; eg; fZg; _N; q;Z);where _N(q; i;Z) = f(q;ZZ)g,and _N(q; e;Z) = f(q; _)g.197From PN we can constructPF = (fp; q; rg; fi; eg; fZ;X0g; _F ; p;X0; frg);where_F (p; _;X0) = f(q;ZX0)g,_F (q; i;Z) = _N(q; i;Z) = f(q;ZZ)g,_F (q; e;Z) = _N(q; e;Z) = f(q; _)g, and_F (q; _;X0) = f(r; _)gThe diagram for PF is, X 0/ZX 0 , X 0 / qi, Z/ZZe, Z/ Startp r198From Final State to Empty StackTheorem 6.11: Let L = L(PF ), for somePDA PF = (Q;_; ; _F ; q0;Z0; F). Then 9 PDAPn, such that L = N(PN).Proof: LetPN = (Q [ fp0; pg;_; [ fX0g; _N; p0;X0)where _N(p0; _;X0) = f(q0;Z0X0)g, _N(p; _; Y )= f(p; _)g, for Y 2 [fX0g, and for all q 2 Q;a 2 _ [ f_g; Y 2 : _N(q; a; Y ) = _F (q; a; Y ),and in addition 8q 2 F, and Y 2 [ fX0g :(p; _) 2 _N(q; _; Y ).any/ any/ any/ , X 0 /Z 0X 0 P p FStartp q 0 0199We have to show that N(PN) = L(PF ).(_-direction.) By inspecting the diagram.(_-direction.) Let w 2 L(PF ). Then(q0;w;Z0) `_F(q; _; _);for some q 2 F; _ 2 _. Since _F _ _N, andTheorem 6.5 says that X0 can be slid underthe stack, we get(q0;w;Z0X0) `_N(q; _;_X0):The PN can compute:(p0;w;X0) `N(q0;w;Z0X0) `_N(q; _;_X0) `_N(p; _; _):200Equivalence of PDA's and CFG'sA language isgenerated by a CFGif and only if it isaccepted by a PDA by empty stackif and only if it isaccepted by a PDA by _nal statePDA byempty stackPDA byGrammar final stateWe already know how to go between null stackand _nal state.201From CFG's to PDA'sGiven G, we construct a PDA that simulates _)lm.We write left-sentential forms asxA_where A is the leftmost variable in the form.For instance,|(a{z+}xE |{z}A|{)z}| {z _}tailLet xA_ )lmx__. This corresponds to the PDA_rst having consumed x and having A_ on thestack, and then on _ it pops A and pushes _.More fomally, let y, s.t. w = xy. Then the PDAgoes non-deterministically from con_guration(q; y;A_) to con_guration (q; y; __).202At (q; y; __) the PDA behaves as before, unlessthere are terminals in the pre_x of _. Inthat case, the PDA pops them, provided it canconsume matching input.If all guesses are right, the PDA ends up withempty stack and input.Formally, let G = (V; T; Q; S) be a CFG. De_nePG as(fqg; T; V [ T; _; q; S);where_(q; _;A) = f(q; _) : A ! _ 2 Qg;for A 2 V , and_(q; a; a) = f(q; _)g;for a 2 T.Example: On blackboard in class.203Theorem 6.13: N(PG) = L(G).Proof:(_-direction.) Let w 2 L(G). ThenS = 1 )lm2 )lm _ _ _ )lmn = wLet i = xi_i. We show by induction on i thatifS _)lmi;then(q;w; S) `_ (q; yi; _i);where w = xiyi.204Basis: For i = 1; 1 = S. Thus x1 = _, andy1 = w. Clearly (q;w; S) `_ (q;w; S).Induction: IH is (q;w; S) `_ (q; yi; _i). We haveto show that(q; yi; _i) ` (q; yi+1; _i+1)Now _i begins with a variable A, and we havethe form|xi{Az_}i)lmxi+1__ | {z }i+1By IH A_ is on the stack, and yi is unconsumed.From the construction of PG is follows that wecan make the move(q; yi; _) ` (q; yi; __):If _ has a pre_x of terminals, we can pop themwith matching terminals in a pre_x of yi, endingup in con_guration (q; yi+1; _i+1), where_i+1 = __, which is the tail of the sententialxi__ = i+1.Finally, since n = w, we have _n = _, and yn =_, and thus (q;w; S) `_ (q; _; _), i.e. w 2 N(PG)205(_-direction.) We shall show by an inductionon the length of `_, that(|) If (q; x;A) _ ` (q; _; _), then A _) x.Basis: Length 1. Then it must be that A ! _is in G, and we have (q; _) 2 _(q; _;A). ThusA _) _.Induction: Length is n > 1, and the IH holdsfor lengths < n.Since A is a variable, we must have(q; x;A) ` (q; x; Y1Y2 _ _ _ Yk) ` _ _ _ ` (q; _; _)where A ! Y1Y2 _ _ _ Yk is in G.206We can now write x as x1x2 _ _ _ xn, accordingto the _gure below, where Y1 = B; Y2 = a, andY3 = C.BaCx x x1 2 3207Now we can conclude that(q; xixi+1 _ _ _ xk; Yi) `_ (q; xi+1 _ _ _ xk; _)is less than n steps, for all i 2 f1; : : : ; kg. If Yiis a variable we have by the IH and Theorem6.6 thatYi _) xiIf Yi is a terminal, we have jxij = 1, and Yi = xi.Thus Yi _) xi by the reexivity of _) .The claim of the theorem now follows by choosingA = S, and x = w. Suppose w 2 N(P).Then (q;w; S) `_ (q; _; _), and by (|), we haveS _) w, meaning w 2 L(G):208From PDA's to CFG'sLet's look at how a PDA can consume x =x1x2 _ _ _ xk and empty the stack.YYYppppkkk-12101...x x x 1 2 kWe shall de_ne a grammar with variables of theform [pi1Yipi] representing going from pi1 topi with net e_ect of popping Yi.209Formally, let P = (Q;_; ; _; q0;Z0) be a PDA.De_ne G = (V;_; R; S), whereV = f[pXq] : fp; qg _ Q;X 2 g [ fSgR = fS ! [q0Z0p] : p 2 Qg[f[qXrk] ! a[rY1r1] _ _ _ [rk1Ykrk] :a 2 _ [ f_g;fr1; : : : ; rkg _ Q;(r; Y1Y2 _ _ _ Yk) 2 _(q; a;X)g210Example: Let's convertStartqi, Z/ZZe, Z/ PN = (fqg; fi; eg; fZg; _N; q;Z);where _N(q; i;Z) = f(q;ZZ)g,and _N(q; e;Z) = f(q; _)g to a grammarG = (V; fi; eg; R; S);where V = f[qZq]; Sg, andR = f[qZq] ! i[qZq][qZq]; [qZq] ! eg.If we replace [qZq] by A we get the productionsS ! A and A ! iAAje.211Example: Let P = (fp; qg; f0; 1g; fX;Z0g; _; q;Z0),where _ is given by1. _(q; 1;Z0) = f(q;XZ0)g2. _(q; 1;X) = f(q;XX)g3. _(q; 0;X) = f(p;X)g4. _(q; _;X) = f(q; _)g5. _(p; 1;X) = f(p; _)g6. _(p; 0;Z0) = f(q;Z0)gto a CFG.212We get G = (V; f0; 1g; R; S), whereV = f[pXp]; [pXq]; [pZ0p]; [pZ0q]; Sgand the productions in R areS ! [qZ0q]j[qZ0p]From rule (1):[qZ0q] ! 1[qXq][qZ0q][qZ0q] ! 1[qXp][pZ0q][qZ0p] ! 1[qXq][qZ0p][qZ0p] ! 1[qXp][pZ0p]From rule (2):[qXq] ! 1[qXq][qXq][qXq] ! 1[qXp][pXq][qXp] ! 1[qXq][qXp][qXp] ! 1[qXp][pXp]213From rule (3):[qXq] ! 0[pXq][qXp] ! 0[pXp]From rule (4):[qXq] ! _From rule (5):[pXp] ! 1From rule (6):[pZ0q] ! 0[qZ0q][pZ0p] ! 0[qZ0p]214Theorem 6.14: Let G be constructed from aPDA P as above. Then L(G) = N(P)Proof:(_-direction.) We shall show by an inductionon the length of the sequence `_ that() If (q;w;X) _ ` (p; _; _) then [qXp] _) w.Basis: Length 1. Then w is an a or _, and(p; _) 2 _(q;w;X). By the construction of G wehave [qXp] ! w and thus [qXp] _) w.215Induction: Length is n > 1, and holds forlengths < n. We must have(q;w;X) ` (r0; x; Y1Y2 _ _ _ Yk) ` _ _ _ ` (p; _; _);where w = ax or w = _x. It follows that(r0; Y1Y2 _ _ _ Yk) 2 _(q; a;X). Then we have aproduction[qXrk] ! a[r0Y1r1] _ _ _ [rk1Ykrk];for all fr1; : : : ; rkg _ Q.We may now choose ri to be the state inthe sequence `_ when Yi is popped. Let w =w1w2 _ _ _wk, where wi is consumed while Yi ispopped. Then(ri1;wi; Yi) `_ (ri; _; _):By the IH we get[ri1; Y; ri] _) wi216We then get the following derivation sequence:[qXrk] ) a[r0Y1r1] _ _ _ [rk1Ykrk] _)aw1[r1Y2r2][r2Y3r3] _ _ _ [rk1Ykrk] _)aw1w2[r2Y3r3] _ _ _ [rk1Ykrk] _)_ _ _aw1w2 _ _ _wk = w217(_-direction.) We shall show by an inductionon the length of the derivation _) that(~) If [qXp] _) w then (q;w;X) `_ (p; _; _)Basis: One step. Then we have a production[qXp] ! w. From the construction of G itfollows that (p; _) 2 _(q; a;X), where w = a.But then (q;w;X) `_ (p; _; _).Induction: Length of _) is n > 1, and ~ holdsfor lengths < n. Then we must have[qXrk] ) a[r0Y1r1][r1Y2r2] _ _ _ [rk1Ykrk] _) wWe can break w into aw2 _ _ _wk such that [ri1Yiri] _)wi. From the IH we get(ri1;wi; Yi) `_ (ri; _; _)218From Theorem 6.5 we get(ri1;wiwi+1 _ _ _wk; YiYi+1 _ _ _ Yk) `_(ri;wi+1 _ _ _wk; Yi+1 _ _ _ Yk)Since this holds for all i 2 f1; : : : ; kg, we get(q; aw1w2 _ _ _wk;X) `(r0;w1w2 _ _ _wk; Y1Y2 _ _ _ Yk) `_(r1;w2 _ _ _wk; Y2 _ _ _ Yk) `_(r2;w3 _ _ _wk; Y3 _ _ _ Yk) `_(p; _; _):219Deterministic PDA'sA PDA P = (Q;_; ; _; q0;Z0; F) is deterministici_1. _(q; a;X) is always empty or a singleton.2. If _(q; a;X) is nonempty, then _(q; _;X) mustbe empty.Example: Let us de_neLwcwr = fwcwR : w 2 f0; 1g_gThen Lwcwr is recognized by the following DPDA1 ,Z 0 Z 0 Z 0 Z 0 , /1 , 0 / 1 00 , 1 / 0 10 , 0 / 0 01 , Z 0 Z 00 , Z 0 Z 0 / 00 , 0 / q q q 0 1 21 / 1 1/Start1 , 1 / / 1,0 / 0, 1 / 1,ccc220We'll show that Regular_ L(DPDA) _ CFLTheorem 6.17: If L is regular, then L = L(P)for some DPDA P.Proof: Since L is regular there is a DFA A s.t.L = L(A). LetA = (Q;_; _A; q0; F)We de_ne the DPDAP = (Q;_; fZ0g; _P ; q0;Z0; F);where_P (q; a;Z0) = f(_A(q; a);Z0)g;for all p; q 2 Q, and a 2 _.An easy induction (do it!) on jwj gives(q0;w;Z0) `_ (p; _;Z0) , ^ _A(q0;w) = pThe theorem then follows (why?)221What about DPDA's that accept by null stack?They can recognize only CFL's with the pre_xproperty.A language L has the pre_x property if thereare no two distinct strings in L, such that oneis a pre_x of the other.Example: Lwcwr has the pre_x property.Example: f0g_ does not have the pre_x property.Theorem 6.19: L is N(P) for some DPDA Pif and only if L has the pre_x property and Lis L(P0) for some DPDA P0.Proof: Homework222_ We have seen that Regular_ L(DPDA):_ Lwcwr 2 L(DPDA)n Regular_ Are there languages in CFLnL(DPDA).Yes, for example Lwwr._ What about DPDA's and Ambiguous Grammars?Lwwr has unamb. grammar S ! 0S0j1S1j_but is not L(DPDA).For the converse we haveTheorem 6.20: If L = N(P) for some DPDAP, then L has an unambiguous CFG.Proof: By inspecting the proof of Theorem6.14 we see that if the construction is appliedto a DPDA the result is a CFG with uniqueleftmost derivations.223Theorem 6.20 can actually be strengthen asfollowsTheorem 6.21: If L = L(P) for some DPDAP, then L has an unambiguous CFG.Proof: Let $ be a symbol outside the alphabetof L, and let L0 = L$.It is easy to see that L0 has the pre_x property.By Theorem 6.20 we have L0 = N(P0) for someDPDA P0.By Theorem 6.20 N(P0) can be generated byan unambiguous CFG G0Modify G0 into G, s.t. L(G) = L, by adding theproduction$ ! _Since G0 has unique leftmost derivations, G0also has unique lm's, since the only new thingwe're doing is adding derivationsw$ )lmwto the end.224Properties of CFL's_ Simpli_cation of CFG's. This makes life easier,since we can claim that if a language is CF,then it has a grammar of a special form._ Pumping Lemma for CFL's. Similar to theregular case. Not covered in this course._ Closure properties. Some, but not all, of theclosure properties of regular languages carryover to CFL's._ Decision properties. We can test for membershipand emptiness, but for instance, equivalenceof CFL's is undecidable.225Chomsky Normal FormWe want to show that every CFL (without _)is generated by a CFG where all productionsare of the formA ! BC; or A ! awhere A;B; and C are variables, and a is aterminal. This is called CNF, and to get therewe have to1. Eliminate useless symbols, those that donot appear in any derivation S _) w, forstart symbol S and terminal w.2. Eliminate _-productions, that is, productionsof the form A ! _.3. Eliminate unit productions, that is, productionsof the form A ! B, where A and Bare variables.226Eliminating Useless Symbols_ A symbol X is useful for a grammar G =(V; T; P; S), if there is a derivationS _)G_X_ _)Gwfor a teminal string w. Symbols that are notuseful are called useless._ A symbol X is generating if X _)Gw, for somew 2 T__ A symbol X is reachable if S _)G_X_, forsome f_; _g _ (V [ T)_It turns out that if we eliminate non-generatingsymbols _rst, and then non-reachable ones, wewill be left with only useful symbols.227Example: Let G beS ! ABja; A ! bS and A are generating, B is not. If we eliminateB we have to eliminate S ! AB, leavingthe grammarS ! a; A ! bNow only S is reachable. Eliminating A and bleaves us withS ! awith language fag.OTH, if we eliminate non-reachable symbols_rst, we _nd that all symbols are reachable.FromS ! ABja; A ! bwe then eliminate B as non-generating, andare left withS ! a; A ! bthat still contains useless symbols228Theorem 7.2: Let G = (V; T; P; S) be a CFGsuch that L(G) 6= ;. Let G1 = (V1; T1; P1; S)be the grammar obtained by1. Eliminating all nongenerating symbols andthe productions they occur in. Let the newgrammar be G2 = (V2; T2; P2; S).2. Eliminate from G2 all nonreachable symbolsand the productions they occur in.The G1 has no useless symbols, andL(G1) = L(G).229Proof: We _rst prove that G1 has no uselesssymbols:Let X remain in V1[T1. Thus X _) w in G1, forsome w 2 T_. Moreover, every symbol used inthis derivation is also generating. Thus X _) win G2 also.Since X was not eliminated in step 2, there are_ and _, such that S _) _X_ in G2. Furthermore,every symbol used in this derivation isalso reachable, so S _) _X_ in G1.Now every symbol in _X_ is reachable and inV2[T2 _ V1[T1, so each of them is generatingin G2.The terminal derivation _X_ _) xwy in G2 involvesonly symbols that are reachable from S,because they are reached by symbols in _X_.Thus the terminal derivation is also a derviationof G1, i.e.,S _) _X_ _) xwyin G1.230We then show that L(G1) = L(G).Since P1 _ P, we have L(G1) _ L(G).Then, let w 2 L(G). Thus S _)Gw. Each symbolis this derivation is evidently both reachableand generating, so this is also a derivationof G1.Thus w 2 L(G1).231We have to give algorithms to compute thegenerating and reachable symbols of G = (V; T; P; S).The generating symbols g(G) are computed bythe following closure algorithm:Basis: g(G) == TInduction: If _ 2 g(G) and X ! _ 2 P, theng(G) == g(G) [ fXg.Example: Let G be S ! ABja; A ! bThen _rst g(G) == fa; bg.Since S ! a we put S in g(G), and becauseA ! b we add A also, and that's it.232Theorem 7.4: At saturation, g(G) containsall and only the generating symbols of G.Proof:We'll show in class on an induction on thestage in which a symbol X is added to g(G)that X is indeed generating.Then, suppose that X is generating. ThusX _)Gw, for some w 2 T_. We prove by inductionon this derivation that X 2 g(G).Basis: Zero Steps. Then X is added in thebasis of the closure algo.Induction: The derivation takes n > 0 steps.Let the _rst production used be X ! _. ThenX ) _ _) wand _ _) w in less than n steps and by the IH_ 2 g(G). From the inductive part of the algoit follows that X 2 g(G).233The set of reachable symbols r(G) of G =(V; T; P; S) is computed by the following closurealgorithm:Basis: r(G) == fSg.Induction: If variable A 2 r(G) and A ! _ 2 Pthen add all symbols in _ to r(G)Example: Let G be S ! ABja; A ! bThen _rst r(G) == fSg.Based on the _rst production we add fA;B; agto r(G).Based on the second production we add fbg tor(G) and that's it.Theorem 7.6: At saturation, r(G) containsall and only the reachable symbols of G.Proof: Homework.234Eliminating _-ProductionsWe shall prove that if L is CF, then Ln f_g hasa grammar without _-productions.Variable A is said to be nullable if A _) _.Let A be nullable. We'll then replace a rulelikeA ! BADwithA ! BAD; A ! BDand delete any rules with body _.We'll compute n(G), the set of nullable symbolsof a grammar G = (V; T; P; S) as follows:Basis: n(G) == fA : A ! _ 2 PgInduction: If fC1C2 _ _ _Ckg _ n(G) and A !C1C2 _ _ _Ck 2 P, then n(G) == n(G) [ fAg.235Theorem 7.7: At saturation, n(G) containsall and only the nullable symbols of G.Proof: Easy induction in both directions.Once we know the nullable symbols, we cantransform G into G1 as follows:_ For each A ! X1X2 _ _ _Xk 2 P with m _ knullable symbols, replace it by 2m rules, onewith each sublist of the nullable symbols absent.Exeption: If m = k we don't delete all m nullablesymbols._ Delete all rules of the form A ! _.236Example: Let G beS ! AB; A ! aAAj_; B ! bBBj_Now n(G) = fA;B; Sg. The _rst rule will becomeS ! ABjAjBthe secondA ! aAAjaAjaAjathe thirdB ! bBBjbBjbBjbWe then delete rules with _-bodies, and end upwith grammar G1 :S ! ABjAjB; A ! aAAjaAja; B ! bBBjbBjb237Theorem 7.9: L(G1) = L(G) n f_g.Proof: We'll prove the stronger statement:(]) A _) w in G1 if and only if w 6= _ and A _) win G._-direction: Suppose A _) w in G1. Thenclearly w 6= _ (Why?). We'll show by and inductionon the length of the derivation thatA _) w in G also.Basis: One step. Then there exists A ! win G1. Form the construction of G1 it followsthat there exists A ! _ in G, where _ is w plussome nullable variables interspersed. ThenA ) _ _) win G.238Induction: Derivation takes n > 1 steps. ThenA ) X1X2 _ _ _Xk _) w in G1and the _rst derivation is based on a productionA ! Y1Y2 _ _ _ Ymwhere m _ k, some Yi's are Xj's and the otherare nullable symbols of G.Furhtermore, w = w1w2 _ _ _wk, and Xi _) wi inG1 in less than n steps. By the IH we haveXi _) wi in G. Now we getA )GY1Y2 _ _ _ Ym _)GX1X2 _ _ _Xk _)Gw1w2 _ _ _wk = w239_-direction: Let A _)Gw, and w 6= _. We'll showby induction of the length of the derivationthat A _) w in G1.Basis: Length is one. Then A ! w is in G,and since w 6= _ the rule is in G1 also.Induction: Derivation takes n > 1 steps. Thenit looks likeA )GY1Y2 _ _ _ Ym _)GwNow w = w1w2 _ _ _wm, and Yi _)Gwi in less thann steps.Let X1X2 _ _ _Xk be those Yj's in order, suchthat wj 6= _. Then A ! X1X2 _ _ _Xk is a rule inG1.Now X1X2 _ _ _Xk _)Gw (Why?)240Each Xj=Yj _)Gwj in less than n steps, so byIH we have that if w 6= _ then Yj _) wj in G1.ThusA ) X1X2 _ _ _Xk _) w in G1The claim of the theorem now follows fromstatement (]) on slide 238 by choosing A = S.241Eliminating Unit ProductionsA ! Bis a unit production, whenever A and B arevariables.Unit productions can be eliminated.Let's look at grammarI ! a j b j Ia j Ib j I0 j I1F ! I j (E)T ! F j T _ FE! T j E +TIt has unit productions E ! T, T ! F, andF ! I242We'll expand rule E ! T and get rulesE ! F; E ! T _ FWe then expand E ! F and getE ! Ij(E)jT _ FFinally we expand E ! I and getE ! a j b j Ia j Ib j I0 j I1 j (E) j T _ FThe expansion method works as long as thereare no cycles in the rules, as e.g. inA ! B; B ! C; C ! AThe following method based on unit pairs willwork for all grammars.243(A;B) is a unit pair if A _) B using unit productionsonly.Note: In A ! BC; C ! _ we have A _) B, butnot using unit productions only.To compute u(G), the set of all unit pairs ofG = (V; T; P; S) we use the following closurealgorithmBasis: u(G) == f(A;A) : A 2 V gInduction: If (A;B) 2 u(G) and B ! C 2 Pthen add (A;C) to u(G).Theorem: At saturation, u(G) contains alland only the unit pair of G.Proof: Easy.244Given G = (V; T; P; S) we can construct G1 =(V; T; P1; S) that doesn't have unit productions,and such that L(G1) = L(G) by settingP1 = fA ! _ : _ =2 V;B ! _ 2 P; (A;B) 2 u(G)gExample: Form the grammar of slide 242 wegetPair Productions(E;E) E ! E +T(E; T) E ! T _ F(E; F) E ! (E)(E; I) E ! a j b j Ia j Ib j I0 j I1(T; T) T ! T _ F(T; F) T ! (E)(T; I) T ! a j b j Ia j Ib j I0 j I1(F; F) F ! (E)(F; I) F ! a j b j Ia j Ib j I0 j I1(I; I) I ! a j b j Ia j Ib j I0 j I1The resulting grammar is equivalent to theoriginal one (proof omitted).245SummaryTo \clean up" a grammar we can1. Eliminate _-productions2. Eliminate unit productions3. Eliminate useless symbolsin this order.246Chomsky Normal Form, CNFWe shall show that every nonempty CFL without_ has a grammar G without useless symbols,and such that every production is of theform_ A ! BC, where fA;B;Cg _ T, or_ A ! _, where A 2 V , and _ 2 T.To achieve this, start with any grammar forthe CFL, and1. \Clean up" the grammar.2. Arrange that all bodies of length 2 or moreconsists of only variables.3. Break bodies of length 3 or more into acascade of two-variable-bodied productions.247_ For step 2, for every terminal a that appearsin a body of length _ 2, create a new variable,say A, and replace a by A in all bodies.Then add a new rule A ! a._ For step 3, for each rule of the formA ! B1B2 _ _ _Bk;k _ 3, introduce new variables C1;C2; : : :Ck2,and replace the rule withA ! B1C1C1 ! B2C2_ _ _Ck3 ! Bk2Ck2Ck2 ! Bk1Bk248Illustration of the e_ect of step 3B 1B 2B k-1 B kB 1 B 2 B kACCCk12-2...A. . .(a)(b)249Example of CNF conversionLet's start with the grammar (step 1 alreadydone)E ! E +T j T _ F j (E) j a j b j Ia j Ib j I0 j I1T ! T _ F j (E)a j b j Ia j Ib j I0 j I1F ! (E) a j b j Ia j Ib j I0 j I1I ! a j b j Ia j Ib j I0 j I1For step 2, we need the rulesA ! a;B ! b;Z ! 0;O ! 1P ! +;M ! _;L ! (;R !)and by replacing we get the grammarE ! EPT j TMF j LER j a j b j IA j IB j IZ j IOT ! TMF j LER j a j b j IA j IB j IZ j IOF ! LER j a j b j IA j IB j IZ j IOI ! a j b j IA j IB j IZ j IOA ! a;B ! b;Z ! 0;O ! 1P ! +;M ! _;L ! (;R !)250For step 3, we replaceE ! EPT by E ! EC1;C1 ! PTE ! TMF; T ! TMF byE ! TC2; T ! TC2;C2 ! MFE ! LER; T ! LER; F ! LER byE ! LC3; T ! LC3; F ! LC3;C3 ! ERThe _nal CNF grammar isE ! EC1 j TC2 j LC3 j a j b j IA j IB j IZ j IOT ! TC2 j LC3 j a j b j IA j IB j IZ j IOF ! LC3 j a j b j IA j IB j IZ j IOI ! a j b j IA j IB j IZ j IOC1 ! PT;C2 ! MF;C3 ! ERA ! a;B ! b;Z ! 0;O ! 1P ! +;M ! _;L ! (;R !)251Closure Properties of CFL'sConsider a mappings : _ ! 2__where _ and _ are _nite alphabets. Let w 2__, where w = a1a2 _ _ _ an, and de_nes(a1a2 _ _ _ an) = s(a1):s(a2): _ _ _ :s(an)and, for L _ __,s(L) =[w2Ls(w)Such a mapping s is called a substitution.252Example: _ = f0; 1g;_ = fa; bg;s(0) = fanbn : n _ 1g; s(1) = faa; bbg.Let w = 01. Then s(w) = s(0):s(1) =fanbnaa : n _ 1g [ fanbn+2 : n _ 1gLet L = f0g_. Then s(L) = (s(0))_ =fan1bn1an2bn2 _ _ _ ankbnk : k _ 0; ni _ 1gTheorem 7.23: Let L be a CFL over _, and sa substitution, such that s(a) is a CFL, 8a 2 _.Then s(L) is a CFL.253We start with grammarsG = (V;_; P; S)for L, andGa = (Va; Ta; Pa; Sa)for each s(a). We then constructG0 = (V 0; T0; P0; S0)whereV 0 = (Sa2_ Va) [ VT0 =Sa2_ TaP0 =Sa2_ Pa plus the productions of Pwith each a in a body replaced with symbolSa.254Now we have to show that_ L(G0) = s(L).Let w 2 s(L). Then 9x = a1a2 _ _ _ an in L, and9xi 2 s(ai), such that w = x1x2 _ _ _ xn.A derivation tree in G0 will look likeSS Sx x xnSa1 a2 an1 2Thus we can generate Sa1Sa2 _ _ _ San in G0 andform there we generate x1x2 _ _ _ xn = w. Thusw 2 L(G0).255Then let w 2 L(G0). Then the parse tree for wmust again look likeSS Sx x xnSa1 a2 an1 2Now delete the dangling subtrees. Then youhave yieldSa1Sa2 _ _ _ Sanwhere a1a2 _ _ _ an 2 L(G). Now w is also equalto s(a1a2 _ _ _ an), which is in S(L).256Applications of the Substitution TheoremTheorem 7.24: The CFL's are closed under(i) : union, (ii) : concatenation, (iii) : Kleeneclosure and positive closure +, and (iv) : homomorphism.Proof: (i): Let L1 and L2 be CFL's, let L =f1; 2g, and s(1) = L1; s(2) = L2.Then L1 [ L2 = s(L).(ii) : Here we choose L = f12g and s as before.Then L1:L2 = s(L)(iii) : Suppose L1 is CF. Let L = f1g_; s(1) =L1. Now L_1 = s(L). Similar proof for +.(iv) : Let L1 be a CFL over _, and h a homomorphismon _. Then de_ne s bya 7! fh(a)gThen h(L) = s(L).257Theorem: If L is CF, then so in LR.Proof: Suppose L is generated b G = (V; T; P; S).Construct GR = (V; T; PR; S), wherePR = fA ! _R : A ! _ 2 PgShow at home by inductions on the lengths ofthe derivations in G (for one direction) and inGR (for the other direction) that (L(G))R =L(GR).258Let L1 = f0n1n2i : n _ 1; i _ 1g. The L1 is CFwith grammarS ! ABA ! 0A1j01B ! 2Bj2Also, L2 = f0i1n2n : n _ 1; i _ 1g is CF withgrammarS ! ABA ! 0Aj0B ! 1B2j12However, L1 \ L2 = f0n1n2n : n _ 1g which isnot CF (see the handout on course-page).259Theorem 7.27: If L is CR, and R regular,then L \ R is CF.Proof: Let L be accepted by PDAP = (QP ;_; ; _P ; qP ;Z0; FP )by _nal state, and let R be accepted by DFAA = (QA;_; _A; qA; FA)We'll construct a PDA for L \ R according tothe pictureAccept/rejectStackANDPDAstateFAstateInput260Formally, de_neP0 = (QP _ QA; ;_; ; _; (qP ; qA);Z0; FP _ FA)where_((q; p); a;X) = f((r; ^_A(p; a)); ) : (r; ) 2 _P (q; a;X)gProve at home by an induction `_, both for Pand for P0 that(qP ;w;Z0) `_ (q; _; ) in Pif and only if((qP ; qA);w;Z0) `__(q; ^_(pA;w)); _; _in P0The claim the follows (Why?)261Theorem 7.29: Let L;L1;L2 be CFL's and Rregular. Then1. L n R is CF2. _L is not necessarily CF3. L1 n L2 is not necessarily CFProof:1. _R is regular, L \ _R is regular, and L \ _R =L n R.2. If _L always was CF, it would follow thatL1 \ L2 = L1 [ L2always would be CF.3. Note that __ is CF, so if L1nL2 was alwaysCF, then so would __ n L = _L.262Inverse homomorphismLet h : _ ! __ be a homom. Let L _ __, andde_neh1(L) = fw 2 __ : h(w) 2 LgNow we haveTheorem 7.30: Let L be a CFL, and h ahomomorphism. Then h1(L) is a CFL.Proof: The plan of the proof isAccept/rejectStackstatePDABufferInput ha h(a)263Let L be accepted by PDAP = (Q;_; ; _; q0;Z0; F)We construct a new PDAP0 = (Q0;_; ; _0; (q0; _);Z0; F _ f_g)whereQ0 = f(q; x) : q 2 Q; x 2 su_x(h(a)); a 2 _g_0((q; _); a;X) = f((q; h(a));X) : _ 6= a 2_; q 2 Q;X 2 g_0((q; bx); _;X) = f((p; x); ) : (p; ) 2 _(q; b;X); b 2T [ f_g; q 2 Q;X 2 gShow at home by suitable inductions that_ (q0; h(w);Z0) `_ (p; _; ) in P if and only if((q0; _);w;Z0) `_ ((p; _); _; ) in P0.264Decision Properties of CFL'sWe'll look at the following:_ Complexity of converting among CFA's andPDAQ's_ Converting a CFG to CNF_ Testing L(G) 6= ;, for a given G_ Testing w 2 L(G), for a given w and _xed G._ Preview of undecidable CFL problems265Converting between CFA's and PDA's_ Input size is n._ n is the total size of the input CFG or PDA.The following work in time O(n)1. Converting a CFG to a PDA (slide 203)2. Converting a \_nal state" PDAto a \null stack" PDA (slide 199)3. Converting a \null stack" PDAto a \_nal state" PDA (slide 195)266Avoidable exponential blow-upFor converting a PDA to a CFG we have(slide 210)At most n3 variables of the form [pXq]If (r; Y1Y2 _ _ _ Yk) 2 _(q; a;X)g, we'll have O(nn)rules of the form[qXrk] ! a[rY1r1] _ _ _ [rk1Ykrk]_ By introducing k2 new states we can modifythe PDA to push at most one symbol pertransition. Illustration on blackboard in class.267_ Now, k will be _ 2 for all rules._ Total length of all transitions is still O(n)._ Now, each transition generates at most n2productions_ Total size (and time to calculate) the grammaris therefore O(n3).268Converting into CNFGood news:1. Computing r(G) and g(G) and eliminatinguseless symbols takes time O(n). This willbe shown shortly(slides 229,232,234)2. Size of u(G) and the resulting grammarwith productions P1 is O(n2)(slides 244,245)3. Arranging that bodies consist of only variablesis O(n)(slide 248)4. Breaking of bodies is O(n) (slide 248)269Bad news:_ Eliminating the nullable symbols can makethe new grammar have size O(2n)(slide 236)The bad news are avoidable:Break bodies _rst before eliminating nullablesymbols_ Conversion into CNF is O(n2)270Testing emptiness of CFL'sL(G) is non-empty if the start symbol S is generating.A naive implementation on g(G) takes timeO(n2).g(G) can be computed in time O(n) as follows:CountGenerating?32A BCc D BB AAB?yes271Creation and initialzation of the array is O(n)Creation and initialzation of the links and countsis O(n)When a count goes to zero, we have to1. Finding the head variable A, checkin if italready is \yes" in the array, and if not,queueing it is O(1) per production. TotalO(n)2. Following links for A, and decreasing thecounters. Takes time O(n).Total time is O(n).272w 2 L(G)?Ine_cient way:Suppose G is CNF, test string is w, with jwj =n. Since the parse tree is binary, there are2n 1 internal nodes.Generate all binary parse trees of G with 2n1internal nodes.Check if any parse tree generates w273CYK-algo for membership testingThe grammar G is _xedInput is w = a1a2 _ _ _ anWe construct a triangular table, where Xij containsall variables A, such thatA _)Gaiai+1 _ _ _ aja a a a a 1 2 3 4 5X X X X XX X X XX X XX XX11 22 33 44 5512 23 34 4513 24 3514 2515274To _ll the table we work row-by-row, upwardsThe _rst row is computed in the basis, thesubsequent ones in the induction.Basis: Xii == fA : A ! ai is in GgInduction:We wish to compute Xij, which is in row j i+1.A 2 Xij, ifA _) aiai +1 _ _ _ aj, iffor some k < j, and A ! BC, we haveB _) aiai+1 _ _ _ ak, and C _) ak+1ak+2 _ _ _ aj, ifB 2 Xik, and C 2 Xkj275Example:G has productionsS ! ABjBCA ! BAjaB ! CCjbC ! ABjaS,A,C--BS,A BB BA,CS,CA,CS,AB A,C{ }{{{S,A,C{{{{{{{{ {}}}}}}}}}}} }b a a b a276To compute Xij we need to compare at mostn pairs of previously computed sets:(Xii;Xi=1;j); (Xi;i+1;Xi+2;j); : : : ; (Xi;j1;Xjj)as suggested belowFor w = a1 _ _ _ an, there are O(n2) entries Xijto compute.For each Xij we need to compare at most npairs (Xik;Xk+1;j).Total work is O(n3).277Preview of undecidable CFL problemsThe following are undecidable:1. Is a given CFG G ambiguous?2. Is a given CFL inherently ambiguous?3. Is the intersection of two CFL's empty?4. Are two CFL's the same?5. Is a given CFL universal (equal to __)?