Chomsky Article 2

8/3/2019 Chomsky Article 2

1/31

I N F O R M A T I O N A N D C O N T R O L 9. , 1 3 7 - 1 6 7 (1959)

On Cer ta in Fo rm a l Properties o f G ram m ars*NOAM CHOMSKY

Massachusetts Institute of Technology, Cambridge, Massachusetts and The Institutefor Advanced Study, Princeton, New JerseyA g r a m m a r c a n b e r e g a r d e d a s a d e v i c e t h a t e n u m e r a t e s t h e s e n -

t e n c e s o f a l a n g u a g e . W e s t u d y a s e q u e n c e o f r e s t r i c t i o n s t h a t l i m i tg r a m m a r s f i r s t t o T u r i n g m a c h i n e s , t h e n t o t w o t y p e s o f s y s t e m f r o mw h i c h a p h r a s e s t r u c t u r e d e s c r i p t i o n o f t h e g e n e r a t e d l a n g u a g e c a nb e d r a w n , a n d f i n a l l y t o f i n i t e s t a t e I V [ a r k o v s o u r c e s ( f i n i t e a u t o -m a t a ) . T h e s e r e s t r i c t i o n s a r e s h o w n t o b e i n c r e a s i n g l y h e a v y i n t h es e n s e t h a t t h e l a n g u a g e s t h a t c a n b e g e n e r a t e d b y g r a m m a r s m e e t i n ga g i v e n r e s t r i c t i o n c o n s t i t u t e a p r o p e r s u b s e t o f t h o s e t h a t c a n b eg e n e r a t e d b y g r a m m a r s m e e t i n g t h e p r e c e d i n g r e s t r i c t i o n . V a r i o u sf o r m u l a t i o n s o f p h r a s e s t r u c t u r e d e s c r i p t i o n a r e c o n s i d e r e d , a n d t h es o u r c e o f t h e i r e x c e s s g e n e r a t i v e p o w e r o v e r f i n i t e s t a t e s o u r c e s i si n v e s t i g a t e d i n g r e a t e r d e t a i l .

S E C T I O N 1A l a n g u a g e i s a c o l l e c t i o n o f s e n t e n c e s o f f i n i t e l e n g t h a l l c o n s t r u c t e d

f r o m a f i n i t e a l p h a b e t ( o r , w h e r e o u r c o n c e r n i s l i m i t e d t o s y n t a x , a f i n i t ev o c a b u l a r y ) o f s y m b o l s . S i n c e a n y l a n g u a g e L i n w h i c h w e a r e l i k e l y t ob e i n t e r e s t e d i s a n i n f i n i t e s e t , w e c a n i n v e s t i g a t e t h e s t r u c t u r e o f L o n l yt h r o u g h t h e s t u d y o f t h e f i n i t e d e v i c e s ( g r a m m a r s ) w h i c h a r e c a p a b l e o fe n u m e r a t i n g i t s s e n t e n c e s . A g r a m m a r o f L c a n b e r e g a r d e d a s a f u n c t i o nw h o s e r a n g e i s e x a c t l y L . S u c h d e v i c e s h a v e b e e n c a l l e d " s e n t e n c e - g e n -e r a t i n g g r a m m a r s . ' ' z A t h e o r y o f l a n g u a g e w i l l c o n t a i n , t h e n , a s p e c i f i c a -

* T h i s w o r k w a s s u p p o r t e d i n p a r t b y t h e U . S . A r m y ( S i g n a l C o r p s ) , t h e U . S .A i r F o r c e ( O f f i c e o f S c i e n t i f i c R e s e a r c h , A i r R e s e a r c h a n d D e v e l o p m e n t C o m -m a n d ) , a n d t h e U . S . N a v y ( O f f i c e o f N a v a l R e s e a r c h ) . T h i s w o r k w a s a l s o s u p -p o r t e d i n p a r t b y t h e T r a n s f o r m a t i o n s P r o j e c t o n I n f o r m a t i o n R e t r i e v a l o f t h eU n i v e r s i t y o f P e n n s y l v a n i a . I a m i n d e b t e d t o G e o r g e A . M i l l e r f o r s e v e r a l i m -p o r t a n t o b s e r v a t i o n s a b o u t t h e s y s t e m s u n d e r c o n s i d e r a t i o n h e r e , a n d t o I ~ . B .L e e s f o r m a t e r i a l i m p r o v e m e n t s i n p r e s e n t a t i o n .

i F o l l o w i n g a f a m i l i a r t e c h n i c a l u s e o f t h e t e r m " g e n e r a t e , " c f . P o s t ( 1 9 4 4 ) .T h i s l o c u t i o n h a s , h o w e v e r , b e e n m i s l e a d i n g , s i n c e i t h a s e r r o n e o u s l y b e e n i n t e r -p r e t e d a s i n d i c a t i n g t h a t s u c h s e n t e n c e - g e n e r a t i n g g r a m m a r s c o n s i d e r l a n g u a g e

1 3 7


2/31

138 CHOMSKYt i o n o f t h e c la ss F o f f u n c t i o n s f r o m w h i c h g r a m m a r s f o r p a r t i c u l a r l a n -g u a g e s m a y b e d ra w n .

T h e w e a k e s t c o n d i t i o n t h a t c a n s ig n i f ic a n t ly b e p la c e d o n g r a m m a r s i st h a t F b e in c l u d e d in t h e c l a s s o f g e n e r a l , u n r e s t r i c t e d T u r i n g m a c h i n e s .T h e s t r o n g e s t , m o s t l im i t i n g c o n d i t i o n t h a t h a s b e e n s u g g e s t e d is t h a te a c h g r a m m a r b e a f in i t e M a r k o v i a n s o u r ce ( f in i t e a u t o m a t o n ) . 2

T h e l a t t e r c o n d i t i o n is k n o w n t o b e t o o s t r o n g ; if F is l i m i t e d i n t h i sw a y i t w i ll n o t c o n t a i n a g r a m m a r f o r E n g l i s h ( C h o m s k y , 1 9 56 ) . T h ef o r m e r c o n d i t io n , o n t h e o t h e r h a n d , h a s n o i n t e re s t . W e l e a r n n o t h i n ga b o u t a n a t u r a l l a n g u a g e f r o m t h e f a c t t h a t i ts s e n t e n c e s c a n b e e ff ec -t i v e l y d i s p l a y e d , i.e ., t h a t t h e y c o n s t i t u t e a r e e u r s i v e l y e n u m e r a b l e s e t.T h e r e a s o n f o r t h i s is d e a r . A l o n g w i t h a s p e c i f i c a t i o n o f t h e c la s s F o fg r a m m a r s , a t h e o r y o f l a n g u a g e m u s t a ls o i n d i c a te h o w , i n ge n e ra l , r e le -v a n t s t r u c t u r a l i n f o r m a t i o n c a n b e o b t a i n e d f o r a p a r t i c u l a r s e n t e n c eg e n e r a t e d b y a p a r t i c u l a r g r a m m a r . T h a t is , t h e t h e o r y m u s t s p e c i fy ac l as s ~ o f " s t r u c t u r a l d e s c r i p t i o n s " a n d a f u n c t i o n a l s u c h t h a t g i v e nf 6 F a n d x i n t h e r a n g e o f f , ~ ( f , x ) 6 Z is a s t r u c t u r a l d e s c r i p t i o n o f x( w i t h r e s p e c t t o t h e g r a m m a r f ) g i v in g c e r ta i n i n f o r m a t i o n w h i c h w i llf a c i l i ta t e a n d s e r v e a s t h e b a s i s f o r a n a c c o u n t o f h o w x is u s e d a n d u n -d e r s t o o d b y s p e a k e r s o f t h e l a n g u a g e w h o s e g r a m m a r is f ; i .e ., w h i c h w i l li n d i c a te w h e t h e r x is a m b i g u o u s , t o w h a t o t h e r s e n t e n c e s i t is s t r u c t u r a l l ys i m i la r , e tc . T h e s e e m p i r i c a l c o n d i t i o n s t h a t l e a d u s t o c h a r a c t e r i z e Fi n o n e w a y o r a n o t h e r a r e o f c r i t ic a l i m p o r t a n c e . T h e y w i ll n o t b e f u r t h e rd i s c u s s e d i n t h i s p a p e r , 3 b u t i t is c l e a r t h a t w e w i l l n o t b e a b l e t o d e -f r o m t h e p o i n t o f v i e w o f t h e s p e a k e r r a t h e r t h a n t h e h e a r e r . A c t u a l l y , s u c h g r a m -m a r s t a k e a c o m p l e t e l y n e u t r a l p o i n t o f v i ew . C o m p a r e C h o m s k y (1 95 7, p . 4 8) .W e c a n c o n s i d e r a g r a m m a r o f L t o b e a f u n c t i o n m a p p i n g t h e i n t e g e r s o n t o L ,o r d e r o f e n u m e r a t i o n b e i n g im m a t e r i a l ( a n d e a s i ly s p e c if ia b le , i n m a n y w a y s ) t ot h is p u r e l y s y n t a c t i c s t u d y , t h o u g h t h e q u e s t io n o f t h e p a r t i c u l a r " i n p u t s " r e -q u i r e d t o p r o d u c e a p a r t i c u l a r s e n t e n c e m a y b e o f g r e a t i n t e r e s t f o r o t h e r i n v e s -t i g a t i o n s w h i c h c a n b u i l d o n s y n t a c t i c w o r k o f t h i s m o r e r e s t r i c t e d k i n d .2 Co mp are De f in i t i on 9 , See . 5 .Ex ce p t b r i e f ly i n 2. In Ch om sky (1956 , 1957 ), an a pp r op r i a t e ~ a nd ~ ( i .e . , ana p p r o p r i a t e m e t h o d f o r d e t e r m i n i n g s t r u c t u r a l i n f o r m a t i o n i n a u n i f o r m m a n -n e r f r o m t h e g r a m m a r ) a r e d e s c r ib e d in f o r m a l l y f o r s e v e r a l t y p e s o f g r a m m a r ,i n c l u d i n g t h o s e t h a t w i l l b e s t u d i e d h e r e . I t i s , i n c i d e n t a l l y , i m p o r t a n t t o r e c o g -n i z e t h a t a g r a m m a r o f a l a n g u a g e t h a t s u c c e e d s i n e n u m e r a t i n g t h e s e n t e n c e swi ll ( a l t houg h i t i s f a r f rom easy to ob ta in e ven th i s r esu lt ) never the l ess be o fq u i t e l i m i t e d i n t e r e s t u n l es s t h e u n d e r l y i n g p r i n c i pl e s o f c o n s t r u c t i o n a r e s u c ha s t o p r o v i d e a u s e f u l s t r u c t u r a l d e s c r i p t i o n .


3/31

ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 139

velop an adequate formulat ion of and % if the elements of F are speci-fied only as such "unstructured" devices as general Turing machines.

Interest in structural properties of natural language thus serves as anempirical motivation for investigation of devices with more generativepower than finite automata, and more special structure than Turingmachines. This paper is concerned with the effects of a sequence of in-creasing heavy restrictions on the class F which limit it first to Turingmachines and finally to finite automata and, in the intermediate stages,to devices which have linguistic significance in that generation of a sen-tence automatically provides a meaningful structural description. Weshall find that these restrictions are increasingly heavy in the sense thateach limits more severely the set of languages that can be generated.The intermediate systems are those that assign a phrase structure de-scritption to the resulting sentence. Given such a classification of specialkinds of Turing machines, the main problem of immediate relevance tothe theory of language is that of determining where in the hierarchy ofdevices the grammars of natural languages lie. It would, for example, beextremely interesting to know whether it is in principle possible to con-struct a phrase structure grammar for English (even though there isgood motivation of other kinds for not doing so). Before we can hopeto answer this, it will be necessary to discover the structural propertiestha t characterize the languages that can be enumerated by grammars ofthese various types. If the classification of generating devices is reason-able (from the point of view of the empirical motivation), such purelymathematical investigation may provide deeper insight into the formalproperties that distinguish natural languages, among all sets of finitestrings in a finite alphabet. Questions of this nature appear to be quitedifficult in the case of the special classes of Turing machines that havethe required linguistic significance. 4 This paper is devoted to a prelimi-nary s tudy of the properties of such special devices, viewed as grammars.It should be mentioned that there appears to be good evidence thatdevices of the kinds studied here are not adequate for formulation of afull grammar for a natural language (see Chomsky, 1956, 4; 1957,Chapter 5). Left out of consideration here are what have elsewhere been

4 In Chom sky and Miller (1958), a struc tural charac teriza tion theore m isstated for languages that can be enumerated by finite automata, in terms of thecycl ical s t ructure of these automata. The basic character izat ion theorem forfinite automata is proven in Kleene (1956).


4/31

140 CHOMSKYcalled "grammatical transformat ions" (Harris, 1952a, b, 1957; Chom-sky, 1956, 1957). These are complex operations that convert sentenceswith a phrase structure description into other sentences with a phrasestructure description. Nevertheless, it appears that devices of the kindstudied in the following pages must function as essential components inadequate grammars for natural languages. Hence investigation of thesedevices is important as a preliminary to the far more difficult study ofthe generative power of transformational grammars (as well as, nega-tively, for the information it should provide about what it is in naturallanguage that makes a transformational grammar necessary).

SECTION 2A phrase structure grammar consists of a finite set of "rewriting rules"of the form ~ --* , where e and ~b are strings of symbols. It contains a

special "initial" symbol S (standing for "sentence") and a boundarysymbol # indicating the beginning and end of sentences. Some of thesymbols of the grammar stand for words and morphemes (grammaticallysignificant parts of words). These const itute the "terminal vocabulary."Other symbols stand for phrases, and constitute the "nonterminal vo-cabulary" (S is one of these, standing for the "longest" phrase). Givensuch a grammar, we generate a sentence by writing down the initialstring #S#, applying one of the rewriting rules to form a new string#~1# (that is, we might have applied the rule #S# --~ #el# or the ruleS --~ ~ ), applying another rule to form a new string #e2#, and so on, untilwe reach a string #~# which consists solely of terminal symbols andcannot be further rewritten. The sequence of strings constructed in thisway will be called a "derivation" of #e~#.Consider, for example, a grammar containing the rules: S ~ AB ,A --~ C, CB ~ Cb, C --> a, and hence providing the derivation D =(#S#, #AB#, #CB#, #Cb#, #ab#). We can represent D diagrammaticallyin the form S/ \A BI I ( 1 )C bI f app ropr ia te rest r ic t ions are p laced on the fo rm o f the ru les m --~ ~ ( inpa r t i cu la r , t he c ond i t ion tha t ~ d i f fe r f rom m by rep lacemen t o f a s ingle


5/31

ON CERTAIN FORMAL PROP ERTI ES OF GRAMMARS 14%1s y m b o l o f ~ b y a n o n - n u l l s t r i n g ) , i t w i ll a l w a y s b e p o s s i b le t o a s s o c i a t ew i t h a d e r i v a t i o n a l a b e l e d t r e e i n t h e s a m e w a y . T h e s e t r e e s c a n b et a k e n a s t h e s t r u c t u r a l d e s c r i p t i o n s d i s c u s s e d i n S e c . 1 , a n d t h e m e t h o do f c o n s t r u c t i n g t h e m , g i v e n a d e r i v a t i o n , w i ll ( w h e n s t a t e d p r e c is e l y )b e a d e f i n i t io n o f t h e f u n c t i o n a l ~ . A s u b s t r i n g x o f t h e t e r m i n a l s t r i n g o fa g i v e n d e r i v a t i o n w i l l b e c a l le d a p h r a s e o f t y p e A j u s t i n c a s e i t c a nb e t r a c e d b a c k t o a p o i n t l a b e l e d A i n t h e a s s o c i a t e d t r e e ( t h u s , f o r ex -a m p l e , t h e s u b s t r i n g e n c l o s e d w i t h i n t h e b o u n d a r i e s i s a p h r a s e o f t h et y p e " s e n t e n c e " ) . I f in t h e e x a m p l e g i v e n a b o v e w e in t e r p r e t A a s N o u nPhrase, B a s Verb Phrase , C a s S i n g u la r N o u n , a a s J o h n , a n d b a s comes,w e c a n r e g a r d D a s a d e r i v a t i o n o f John comes p r o v i d i n g t h e s t r u c t u r a ld e s c r i p ti o n ( 1 ) , w h i c h i n d i c a te s t h a t J o h n is a S i n g u l a r N o u n a n d aN o u n P h r a s e , t h a t comes is a Verb Phrase , a n d t h a t J o h n c o m e s is aSentence. G r a m m a r s c o n t a i n i n g ru l e s f o r m u l a t e d i n s u c h a w a y t h a t t r e e sc a n b e a s s o c i a t e d w i t h d e r i v a t i o n s w i l l t h u s h a v e a c e r t a i n l i n g u i s t i cs i g n if i ca n c e i n t h a t t h e y p r o v i d e a p r e c is e r e c o n s t r u c t i o n o f l a rg e p a r t so f t h e t r a d i t i o n a l n o t i o n o f " p a r s i n g " o r, in i ts m o r e m o d e r n v e r s io n ,i m m e d i a t e c o n s t i t u e n t a n a ly s is . ( C f . C h o m s k y ( 1 95 6 , 1 95 7 ) fo r f u r t h e r] i s e u s s i o n . )

T h e b a s i c s y s t e m o f d e s c r i p t i o n t h a t w e s h a ll c o n s i d e r i s a s y s t e m Go f t h e f o l l o w i n g f o r m : G is a s e m i - g r o u p u n d e r c o n c a t e n a t i o n w i t h s t r in g si n a fi n i te s e t V o f s y m b o l s a s i t s e l e m e n t s , a n d I a s t h e i d e n t i t y e l e m e n t .V is ca l le d t h e " v o c a b u l a r y " o f G . V = V r u V N ( V r , V N d i s j o i n t ), w h e r eV r is t h e " t e r m i n a l v o c a b u l a r y " a n d VN t h e " n o n t e r m i n a l v o c a b u l a r y . "V r c o n t a in s I a n d a " b o u n d a r y " e l e m e n t #. V ~ c o n t a i n s a n e l e m e n t S( s e n t e n c e ) . A t w o - p l a c e r e l a t i o n - ~ i s d e f i n e d o n e l e m e n t s o f G , r e a d" c a n b e r e w r i t t e n a s . " T h i s r e l a t i o n s a t i s f i e s t h e f o l l o w i n g c o n d i t i o n s :

A x io m 1 . - -* i s i r r e f l e x ive .AXIOM 2. A C VN i f an d o nl y i f th e r e a re ~ ,, , co su ch t h a t ~ ,A - -+ ~co.A xi om 3 . T h e r e a r e n o ~ , , co suc h th a t ~ --+ #c o.A x l o ~ 4 . T h e r e i s a f i n i te s e t o f p a i r s ( X i , c ol), " ' " , ( x ~ , c o s) s u c h

th a t f o r a ll ~ , , ~ --~ i f a n d on ly i f t h e r e a r e ~ 1 , ~ 2 , a n d j _= n su c hth a t ~ = ~ ix j~2 an d = ~coj~2

T h u s t h e p a ir s ( x J , coJ) w h o s e e x i st e n ce is g u a r a n t e e d b y A x i o m 4g i v e a f i n it e s p e c i fi c a ti o n o f t h e r e l a t i o n - -~ . I n o t h e r w o r d s , w e m a y t h i n ko f t h e g r a m m a r a s c o n t a i n i n g a f i n i te n u m b e r o f r u l e s x ; - -~ coi w h i c hc o m p l e t e l y d e t e r m i n e a ll p o s s ib l e d e r i v a t io n s .

T h e p r e s e n t a t i o n w i ll b e g r e a t l y f a c i l it a t e d b y t h e a d o p t i o n o f t h ef o ll o w in g n o t a t i o n a l c o n v e n t i o n ( w h i c h w a s i n f a c t fo l lo w e d a b o v e ) .


6/31

1 4 2 C H O M S K Y

C ON VEN TIO N 1 : W e s h a l l u s e c a p i t a l l e t t e r s f o r s t r i n g s i n V ~ ; s m a l lL a t i n l e t t e r s f o r s t r in g s i n V r ; G r e e k l e t t e r s f o r a r b i t r a r y s t r i n g s ; e a r l yl e t t e r s o f a l l a l p h a b e t s f o r s in g le s y m b o l s ( m e m b e r s o f V ) ; l a t e l e t t e r so f al l a l p h a b e t s f o r a r b i t r a r y s t ri n g s.

DEFINITION 1 . (91 , " ' " , 9n ) (n > 1) i s a J-der iva t ion o f~ i f ~b = ~i ,= 9 ~ , a n d 9 ~ - + 9 i + 1 ( 1 =< i < n ) .D E F IN IT IO N 2 . A 9 - de r iv a t i on i s t e r m i n a t e d i f i t i s n o t a p r o p e r in i t i a l

s u b s e q u e n c e o f a n y 9 - d e r i v a ti o n . ~DEFINITION 3 . T h e t e rm i n a l l a n g u a g e L a g e n e ra t e d b y G i s t h e s e t o f

s t ri n g s x s u c h t h a t t h e r e i s a t e r m i n a t e d # S # - d e r i v a t i o n o f x . 6DEFINITION 4. G is equiva len t t o G * i f L a = L a , .D E F I N I T I O N 5 . 9 ~ ~ b i f t h e r e i s a 9 - d e r i v a t i o n o f ~ .( w h i c h i s t h e o r d i n a r y a n c e s t r a l o f - - ~ ) i s t h u s a p a r t i a l o r d e r i n g o f

s t r i n g s i n G . T h e s e n o t i o n s a p p e a r , i n s l i g h t l y d i f f e r e n t f o r m , i n C h o m s k y( 1 9 5 6 , 1 9 5 7 ) .

T h i s p a p e r w i l l b e d e v o t e d t o a s t u d y o f t h e e f f e c t o f i m p o s i n g t h ef o l l o w i n g a d d i t i o n a l r e s t r i c t i o n s o n g r a m m a r s o f t h e t y p e d e s c r i b e da b o v e .

R E S T R I C T I O N i . I f 9 - - * ~ b , t h e n t h e r e a r e A , 9 1 , 9 2 , ~ s u c h t h a t 9 - -9 1 A 9 2 , ~ b = 9 1 w 9 2 , a n d ~ ~ I .

R E S T R I C T I O N 2 . I f 9 - ~ ~ b , t h e n t h e r e a r e A , 9 J , 9 2 , ~ s u c h t h a t 9 =9 1 A 9 2 , ~ b - - 9 1 ~ 9 2 , ~ 0 # I , b u t A - ~ w .

R E S T R I C T I O N 3 . I f 9 - ~ # , t h e n t h e r e a r e A , 9 1 , 9 2 , w , a , B s u c h t h a t9 - 9 1 A 9 2 , ~ b - 9 1 ~ 9 2 , ~ 0 ~ I , A - - ~ , b u t o ~ - a B o r ~ o = a .

T h e n a t u r e o f t h e s e r e s t r i c t i o n s i s c l a r i f i e d b y c o m p a r i s o n w i t h A x i o m4 , a b o v e . R e s t r i c t i o n 1 r e q u i r e s t h a t t h e r u l e s o f t h e g r a m m a r [ i . e . , t h em i n i m a l p a i r s ( x ~ , w ~ ) o f A x i o m 4 ] a l l o f b e t h e f o r m 9 1 A 9 2 - - + 9 1 ~ q ~ 2 ,w h e r e A i s a s i n g l e s y m b o l a n d w ~ I . S u c h a r u l e a s s e r t s t h a t A - ~i n t h e c o n t e x t 9 1 - - ~ 2 ( w h i c h m a y b e n u l l ) . R e s t r i c t i o n 2 r e q u i r e s t h a tt h e l i m i t i n g c o n t e x t i n d e e d b e n u l l ; t h a t i s , t h a t t h e r u l e s a l l b e o f t h ef o r m A - + o ~ , w h e r e A i s a s i n g l e s y m b o l , a n d t h a t e a c h s u c h r u l e m a y b ea p p l i e d i n d e p e n d e n t l y o f t h e c o n t e x t i n w h i c h A a p p e a r s . R e s t r i c t i o n 3

5 N o t e t h a t a t e r m i n a t e d d e r i v a t i o n n e e d n o t t e r m i n a t e i n a s t r i n g o f V r ( i . e . ,i t m a y b e " b l o c k e d " a t a n o n t e r m i n a l s t r i n g ) , a n d t h a t a d e r i v a t i o n e n d i n g w i t ha s t r i n g o f V T n e e d n o t b e t e r m i n a t e d ( i f , e . g . , t h e g r a m m a r c o n t a i n s s u c h r u l e sas a b - ~ c d ) .6 Thus the term inal la nguage LG consists only of those strin gs of Vr w hich arederivable from #S# but which cannot head a derivati on (of >2 lines).


7/31

ON CERTAIN FORMAL PROP ERTI ES OF GRAMMARS 14 3l i m i t s t h e r u l e s t o t h e f o r m A ---> a B o r A --+ a ( w h e r e A , B a r e s i n g l en o n t e r m i n a l s y m b o l s , a n d a is a s i ng le t e r m i n a l s y m b o l ) .

D E F I N I T I O N 6 . F o r i = 1 , 2 , 3 , a t y p e i g r a m m a r i s o n e m e e t i n g r e s t r ic -t i o n i , a n d a type i language i s o n e w i t h a t y p e i g r a m m a r . A t y pe 0 g ram-m a r ( l a n g u a g e ) is o n e t h a t i s u n r e s t r i c t e d .

T y p e 0 g r a m m a r s a r e e s se n t i a l ly T u r i n g m a c h i n e s ; t y p e 3 g r a m m a r s ,f in i t e a u t o m a t a . T y p e 1 a n d 2 g r a m m a r s c a n b e i n te r p r e t e d a s s y s t e m so f p h r a s e s t r u c t u r e d e s c r i p t i o n .

S E C T I O N 3T h e o r e m 1 fo ll ow s i m m e d i a t e l y f r o m t h e d e f i n it io n s .TH EO REM 1. F o r b o t h g r a m m a r s a n d l a n g u a g e s , t y p e 0 D t y p e 1t y p e 2 ___ t y p e 3 .T h e f o l lo w i n g is , f u r t h e r m o r e , w e l l k n o w n .T tIE O R EM 2 . E v e r y r e c u r s i v e l y e n u m e r a b l e s e t of s t r i n g s is a t y p e 0

l a n g u a g e ( a n d c o n v e r s e l y ) , vT h a t is , a g r a m m a r o f t y p e 0 is a d e v i ce w i t h t h e g e n e r a t i v e p o w e r o f a

T u r i n g m a c h i n e . T h e t h e o r y o f t y p e 0 g r a m m a r s a n d t y p e 0 l a n g u a g e sis t h u s p a r t o f a r a p i d l y d e v e l o p i n g b r a n c h o f m a t h e m a t i c s ( r e c u r s i v ef u n c t i o n t h e o r y ) . C o n c e p t u a l l y , a t l e as t, t h e t h e o r y o f g r a m m a r c a n b ev i e w e d a s a s t u d y o f s p e c i al c l as s es o f r e c u r s i v e f u n c t i o n s .T HE OR EM 3 . E a c h t y p e 1 l a n g u a g e is a d e c i d a b l e s e t o f s t r i n g s . 7~

T h a t is , g i v e n a t y p e 1 g r a m m a r G , t h e r e i s a n e f fe c t i v e p r o c e d u r e f o rd e t e r m i n i n g w h e t h e r a n a r b i t r a r y s t r in g x i s i n t h e l a n g u a g e e n u m e r a t e db y G . T h i s f o l l o w s f r o m t h e f a c t t h a t i f ~ , ~ + 1 a r e s u c c e s s iv e l in e s o f ad e r i v a ti o n p r o d u c e d b y a t y p e 1 g r a m m a r , t h e n ~ + 1 c a n n o t c o n t a i nf e w e r s y m b o l s t h a n ~ , s in c e ~ + 1 is f o r m e d f r o m ~ b y r e p l a c in g a s i ng les y m b o l A o f ~ b y a n o n - n u l l s t r in g ~ . C l e a r ly a n y s t r i n g x w h i c h h a s a

7 See , fo r exam ple , D av i s (1958 , Ch ap . 6 , 2 ) . I t i s eas i ly shown tha t t he fu r th ers t r u c t u r e i n ty p e 0 g r a m m a r s o v e r t h e c o m b i n a t o r i a l s y s t e m s t h e r e d e s c r i b e d d o e sno t a f fec t t h i s r esu l t .7~ B u t n o t c o n v e r s e l y . F o r s u p p o s e w e g i v e a n e f f e c ti v e e n u m e r a t i o n o f t y p e 1g r a m m a r s , t h u s e n u m e r a t i n g t y p e 1 l a n g u ag e s a s L 1 , L ~ , - . . . L e t sl ,s ~ , . . - b ea n e f f e c t i v e e n u m e r a t i o n o f a l l f i n i t e s t r i n g s i n w h a t w e c a n a s s u m e ( w i t h o u tr e s t r ic t i o n ) t o b e t h e c o m m o n , fi n it e a l p h a b e t o f L 1 , L 2 , - - - . G i v e n t h e i n d e x o ia l a n g u a g e i n t h e e n u m e r a t i o n L ~ ,L 2 , . - . , w e h a v e i m m e d i a t e l y a d e c i s io n p r o c e -d u r e f o r t h i s la n g u a g e . L e t M b e th e " d i a g o n a l " l a n g u a g e c o n t a i n i n g j u s t t h o s es t r in g s s l s u c h t h a t si@ L i . T h e n M i s a d e c id a b l e l a n g u a g e n o t i n t h e e n u m e r a -t ion .I a m i n d e b t e d t o H i l a r y P u t n a m f o r t h i s o b s e r v a t i o n .


8/31

144 CHOMSKY#S#-derivation, has a #S#-derivation in which no line repeats, since linesbetween repetitions can be deleted. Consequently, given a grammar Gof type 1 and a string x, only a finite number of derivations (those withno repetitions and no lines longer than x) need be investigated to deter-mine whether x C Lo.

We see, therefore, that Restriction 1 provides an essentially morelimited type of grammar than type 0.

The basic relation -~ of a type 1 grammar is specified completelyby a finite set of pairs of the for m (1A@~, @~~). Suppose tha t ~ =ax a~ . We can then associate with this pair the element

A

(T 1 O~2 O/ m_ 10 /m

(2)

Corresponding to any derivation D we can construct a tree formed fromthe elements (2) associated with the transitions between successive linesof D, adding elements to the tree from the appropriate node as thederi vation progresses, s We can thus associate a labeled tree with eachderivatio n as a structural description of the generated sentence. Th e re-striction on the rules ~ -+ ~ which leads to type 1 grammars thus has acertain linguistic significance since, as pointed out in Sec. 1, these gram-mars provide a precise reconstruction of much of what is traditionallycalled "parsing" or "immediate constituent analysis." Type 1 grammarsare the phrase structure grammars considered in Chomsky (1957,Chap. 4).

SECTION 4LEMMA 1. Suppose that G is a type 1 gra mmar, and X , B are particu-lar strings of G. Let G' be the grammar formed by adding X B ~ B X

to G. Then there is a type 1 grammar G* equivalent to G '.P~ooF. Suppose th a t X = A1 An . Choose C1, , Cn+l new and

distinct. Let Q be the sequence of rules8 This associated tree might not be unique, if, for example, there were a deriva-tion containing the successive lines ,p1AB~,2, ~IA CB ~2, since this step in the deriva-tion might have used either of the rules A --~ A or B --~ CB. It is possible to addconditions on G that guarantee uniqueness without affecting the set of generated

languages.


9/31

O N C E R T A I N F O R M A L P R O P E R T I E S O F G R A M M A R S

A 1 " '" A n B - ~ C 1 A 2 " ' " A,~ B145

C1 . . . C ~ BB C 2 . . . C n+ l

B A 1 " " A,~where the left-hand side of each rule is the right-hand side of the im-mediately preceding rule. Let G* be formed by adding the rules of Q toG. It is obvious that if there is a #S#-derivation of x in G* using rules ofQ, then there is a #S#-derivation of x in G* in which the rules are ap-plied only in the sequence Q, with no other rules interspersed (note tha tx is a terminal st ring). Consequently the only effect of adding the rulesof Q to G is to permit a string ~ , X B ~ to be rewritten ~ B X , and La.contains only sentences of L~,. It is clear that La* contains all the sen-tences of Lo, and that G* meets Restriction 1.By a similar argument it can easily be shown that type 1 languagesare those whose grammars meet the condition that if ~ --~ ~b, then ~b isat least as long as ~. That is, weakening Restriction 1 to this extent willnot increase the class of generated languages.LEMMA 2. Let L be the language containing all and only the sentencesof the form # a ~b m a %' ~c c c # ( m , n ~ 1). Then L is a type 1 language.PRoof. Consider the grammar G with V r = l a , b , c , I , # } ,

VN = {S, $1 , $2, A, .4, B,/~, C, D, E, F},and the following rules:(I) ( a ) S ~ C D S ~ S 2 F(b ) S~ -+ S:S~

(c) [S 2 B ---> B B J(d) $1 "-+ S1S~

~S l u =---+A B ~( e ) [ S I A - - - + A A /


10/31

146 C~OM S XY{CD A --+ CE ~A( I I ) ( a ) I C D B --+ C E B B S

(b ) [ C E ~ ~ ~ C E J( c) E ~ - ~ ~ E a( d ) E a # --+ D a #( e ) ~ --~ D a

( I I I ) C D F a ~ a C D F( IV ) (a) ~B, 3 - ~ bJ

~ C D F # - - - + C D c # ]( b ) ~ C D c ~ C c cL C c - ~ c c J

w h e r e a , f~ r a n g e o v e r { A , B , F } .I t c a n n o w b e d e t e r m i n e d t h a t t h e o n l y # S # - d e r i v a t io n s o f G t h a t

t e r m i n a t e i n s t r in g s o f V T a r e p r o d u c e d i n t h e f o l lo w i n g m a n n e r :( 1 ) t h e r u le s o f ( I ) a r e a p p l i e d a s f o l lo w s : ( a ) o n c e , (b ) m - 1 t i m e s

f o r s o m e m = 1 , ( c) m t i m e s , ( d ) n - - 1 t i m e s f o r s o m e n => 1 , a n d ( e )n t i m e s , g i v i n g

# C D o ~ , . . . , ~ , ,+ , , F #w h e r e a t = A f o r i ~ n , a ~ = B f o r i > n

( 2 ) t h e r u le s o f ( I I ) a r e a p p l i e d a s f o l lo w s : ( a ) o n c e a n d ( b ) o n c e ,g i v i n g

# a l C E a l . . . , ~ , ~ + m F #( c ) n + m t i m e s a n d ( d ) o n c e , g i v i n g

# a l C a ~ . . . o ~ n+ ~ F D a l #( e ) n + m t i m e s , g i v i n g

# a l C D a ~ . . . o l~ F a l#( 3 ) t h e r u l e s o f ( I I ) a r e a p p l i e d , a s i n (2 ) , n + m - 1 m o r e ti m e s ,

g i v i n g# a l " ' " a , ~ + ~ C D F a l . . . a ,~ +,~ #

9 W here he re a nd hen cefo r th , a~ = fi~ i f a~ = A , ~ = /~ i f a~ = B . N ote t h u tu s e o f r u l e s o f t h e t y p e o f ( I I ) , ( b ), ( c) , ( e), a n d ( I I I ) i s j u s t if i e d b y L e m m a 1 .


11/31

ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 147

( 4 ) t h e r u l e ( I I I ) i s a p p l i e d n + m t i m e s , g i v i n g#~ "'" ~,~+,~1 "'" a,~+,~CDF#

( 5 ) t h e r u l e s o f ( I V ) a r e a p p l ie d , ( a ) 2 ( n + m ) t i m e s , ( b ) o n c e ,g i v i n g

#anb ma % %cc#A n y o t h e r s e q u e n c e o f r u le s ( e x c e p t fo r a c e r t a i n fr e e d o m i n p o i n t o f

a p p l i c a t i o n o f [ IV a ] ) w i l l f a i l t o p r o d u c e a d e r i v a t i o n t e r m i n a t i n g i n as t ri n g o f V r . N o t i c e t h a t t h e f o r m o f t h e t e r m i n a l s t r in g is c o m p l e t e l yd e t e r m i n e d b y s t e p ( 1 ) a b o v e , w h e r e n a n d m a r e s el e ct e d. R u l e s ( I I )a n d ( I I I ) a r e n o t h i n g b u t a c o p y i n g d e v i ce t h a t c a rr ie s a n y s t r i n g o f t h ef r o m #CDXF# ( w h e r e X i s a n y s t ri n g o f A ' s a n d B ' s ) t o t h e c o r r e s p o n d -i n g s t r i n g #XXCDF#, w h i c h is c o n v e r t e d b y ( I V ) i n t o t e r m i n a l f o r m .

B y L e m m a 1, t h e r e i s a t y p e 1 g r a m m a r G * e q u i v a l e n t t o G , a s w a st o b e p r o v e n .

T I~E OR EM 4 . T h e r e a r e t y p e 1 l a n g u a g e s w h i c h a r e n o t t y p e 2 la n -g u a g e s .

P R o o F . W e h a v e s e e n t h a t t h e l a n g u a g e L c o n s is t in g o f a ll a n d o n l yt h e s t r i n g s #a%'~a%%cc# i s a t y p e 1 l a n g u a g e . S u p p o s e t h a t G i s a t y p e2 g r a m m a r o f L . W e c a n a s s u m e f o r e a c h A i n th e v o c a b u l a r y o f G t h a tt h e r e a r e i n f i n i t e l y m a n y x ' s s u c h t h a t A ~ x ( o t h e r w i s e A c a n b e e l im i -n a t e d f r o m G in f a v o r o f a f in i te n u m b e r o f r u le s o f t h e f o r m B - + ~ l z ~w h e n e v e r G c o n t a i n s t h e r u l e B --~ ~ 1A ~ 2 a n d A ~ z ) . L c o n t a i n s i n t l-n i t e l y m a n y s e n te n c e s, b u t G c o n t a in s o n l y f i n it e ly m a n y s y m b o l s . T h e r e -f o r e w e c a n f i n d a n A s u c h t h a t f o r i n f in i t e ly m a n y s e n t e n c e s of L t h e r ei s a n # S # - d e r i v a t i o n t h e n e x t - t o - l a s t l in e o f w h i c h is o f t h e f o r m xAy( i. e. , A i s i ts o n l y n o n t e r m i n a l s y m b o l ) . F r o m a m o n g t h e s e , s e l e ct as e n t e n c e s = #a~b'~a'b'%cc# s u c h t h a t m - t- n > r , w h e r e a l . . . an i st h e l o n g e s t s t r in g z s u c h t h a t A --+ z ( n o t e t h a t t h e r e m u s t b e a z s u c ht h a t A - + z , s i n ce A a p p e a r s i n t h e n e x t - t o - l a s t l i n e of a d e r i v a t i o n o f at e r m i n a l s t ri n g ; a n d , b y A x i o m 4 , t h e r e a r e o n l y fi n i te l y m a n y s u c h z ' s ) .B u t n o w i t is i m m e d i a t e l y c l e a r t h a t f f ( ~ , - . , ~ t+ ~) is a # S # - d e r i v a t i o no f s f o r w h i c h ~t = #xAy#, t h e n n o m a t t e r w h a t x a n d y m a y b e ,

( ~ 1 , " ' " , ~ t )i s t h e i n it ia l p a r t o f i n f in i t e ly m a n y d e r i v a t i o n s o f t e r m i n a l s t ri n g s n o ti n L . H e n c e G is n o t a g r a m m a r o f L .

W e s e e, t h e r e f o r e , t h a t g r a m m a r s m e e t i n g R e s t r i c t i o n 2 ar e e s se n t i a ll y


12/31

148 CHOMSKY

l es s p o w e r f u l t h a n t h o s e m e e t i n g o n l y R e s t r i c t i o n 1. H o w e v e r , t h e e x t r ap o w e r o f g r a m m a r s t h a t d o n o t m e e t R e s t r i c t i o n 2 a p p e a rs , f r o m t h ea b o v e r e su l ts , t o b e a d e f e c t o f s u c h g r a m m a r s , w i t h r e g a r d t o t h e i n -t e n d e d i n t e rp r e t a t i o n . T h e e x t r a p o w e r o f t y p e 1 g r a m m a r s c o m e s ( inp a r t , a t l e a s t ) f r o m t h e f a c t t h a t e v e n t h o u g h o n l y a s in g le s y m b o l i sr e w r i t t e n w i t h e a c h a d d i t i o n o f a n e w l in e t o a d e r i v a t i o n , i t i s n e v e r -t h e l e s s p o s s ib l e i n ef f ec t t o i n c o r p o r a t e a p e r m u t a t i o n s u c h a s AB ~ BA( L e m m a 1 ) . T h e p u r p o s e o f p e r m i t t i n g o n l y a s in g le s y m b o l t o b e re -w r i t t e n w a s t o p e r m i t t h e c o n s t r u c t i o n o f a t re e ( a s i n S ec . 2 ) a s as t r u c t u r a l d e s c r i p t i o n w h i c h s p e c if ie s t h a t a c e r t a i n s e g m e n t x o f t h eg e n e r a t e d s e n t e n c e is a n A ( e .g . , i n t h e e x a m p l e i n S e c . 2, John i s aNoun Phrase) . T h e t r e e a s s o c i a te d w i t h a d e r i v a t i o n s u c h a s t h a t i n t h ep r o o f o f L e m m a 1 w i ll , w h e r e i t in c o r p o r a t e s a p e r m u t a t i o n AB --~ BA ,s p e c i f y t h a t t h e s e g m e n t d e r i v e d u l t i m a t e l y f r o m t h e B o f - BA i sa n A , a n d t h e s e g m e n t d e r i v e d f r o m th e A o f . . - B A . . . i s a B . F o re x a m p l e , a ty p e 1 g r a m m a r i n w h i c h b o t h John will come a n d will Johncome a r e d e r i v e d f r o m a n e a r l i e r l i n e Noun Phrase-Modal-Verb, w h e r ewill John come is p r o d u c e d b y a p e r m u t a t i o n , w o u l d s p e c i fy t h a t willi n will John come is a Noun Phrase a n d John a Modal, c o n t r a r y t o i n -t e n t i o n . T h u s t h e e x t r a p o w e r o f t y p e 1 g r a m m a r s i s a s m u c h a d e fe c ta s w a s t h e s t il l g r e a t e r p o w e r o f u n r e s t r i c t e d T u r i n g m a c h i n e s ( t y p e 0g r a m m a r s ) .

A t y p e 1 g r a m m a r m a y c o n t a in m i n i m a l ru l es o f t h e f o r m ~ I A ~~ 1 ~ 2 , w h e r e a s i n a t y p e 2 g r a m m a r , ~ a n d ~2 m u s t b e n u l l i n t h i s c as e .A r u l e o f t h e t y p e 1 f o r m a s s e r t s , i n e f f e c t, t h a t A --~ o~ i n t h e c o n t e x t~ - - ~ . C o n t e x t u a l r e s t ri c t i o n s o f t h i s t y p e a r e o f t e n f o u n d n e c e s s a r yi n c o n s t r u c t i o n o f p h r a s e s t r u c t u r e d e s c r ip t i o n s f o r n a t u r a l l a n g u a g es .C o n s e q u e n t l y t h e e x t r a f le x i bi li ty p e r m i t t e d i n t y p e 1 g r a m m a r s i s i m -p o r t a n t . I t s e e m s c le a r, t h e n , t h a t n e i t h e r R e s t r i c t io n 1 n o r R e s t r i c t i o n 2is e x a c t l y w h a t i s r e q u i r e d f o r t h e c o m p l e t e r e c o n s t r u c t i o n o f i m m e d i a t ec o n s t i t u e n t a n a l y s is . I t i s n o t o b v i o u s w h a t f u r t h e r q u a l if i ca t io n w o u l db e a p p r o p r i a te .

I n t y p e 2 g r a m m a r s , t h e a n o m a l ie s m e n t i o n e d i n f o o tn o t e 5 ar ea v o i d e d . T h e f in a l l in e o f e a c h t e r m i n a t e d d e r i v a t i o n i s a s t r in g i n V r ,a n d n o s t r in g i n V r c a n h e a d a d e r i v a t i o n o f m o r e t h a n o n e li ne .

S E C T I O N 5W e c o n s id e r n o w g r a m m a r s m e e t i n g R e s t r i c t i o n 2 .D E FIN IT IO N 7 . A g r a m m a r is self-embedding ( s . e . ) i f i t c o n t a i n s a n As u c h t h a t f o r s o m e ~ ,~ b(~ ~ I ~ ) , A ~ ~ A~ b.


13/31

O N C E R T A I N F O R M A L P R O P E R T I E S O F G R A M M A R S 1 4 9

D E FIN IT IO N 8 . A g r a m m a r G i s r e g u l a r i f i t c o n t a i n s o n l y r u l e s o f t h ef o r m A --+ a o r A ---+ B C , w h e r e B ~ C ; a n d i f w h e n e v e r A - + ~ 1B ~2 a n dA --+ ~blB~2 ar e ru le s of G , th e n ~o~ = ~b~(i = 1, 2) .T HE OR EM 5 . I f G is a t y p e 2 g r a m m a r , t h e r e i s a r e g u l a r g r a m m a r G *w h i c h is e q u i v a l e n t t o G a n d w h i c h , f u r t h e r m o r e , i s n o n - s . c , if G i sn o n - s . c .

P R O OF . D e f i n e L ( ~ ) ( i. e ., l e n g t h o f ~ ) t o b e m i f ~ = a l a m , w h e r ea ~ 7 I .

G i v e n a t y p e 2 g r a m m a r G , c o n s i d e r a l l d e r i v a t i o n s D = ( 9 1 , " - , 9 t )m e e t i n g t h e f o l l o w i n g f o u r c o n d i t i o n s :

( a ) f o r s o m e A , 91 = A( b ) D c o n t a i n s n o r e p e a t i n g l in e s( c ) L ( ~ o t _ ~ ) < 4(d ) L ( t ) _-__ 4 or ~ot i s te rm in a l .

C l e a r l y t h e r e is a f in i te n u m b e r o f s u c h d e r i v a t i o n s . L e t G 1 b e t h e g r a m -m a r c o n t a i n i n g t h e m i n i m a l r u l e ~ - + ~b j u s t i n c a se f o r s o m e s u c h d e r i v a -t i o n D , ~ = ~ a n d ~ = c t . C l e a r l y G~ i s a t y p e 2 g r a m m a r e q u i v a l e n tto G, a n d i s non -s .c , i f G i s no n-s .c . , s ince ~o -+ ~b in G1 on ly i f ~o ~ in G.

S u p p o s e t h a t G1 c o n t a i n s r u l e s R~ a n d R 2 :R1 : A - + ~iB~o~ = o01c02~a~(~ ~ I )R~ : A - -~ ~ B ~

w h e r e 9~ ~ ~bl o r 9 2 # ~ 2 R e p l a c e R ~ b y t h e t h r e e r u l e sRI~ : A - --+ C DR ~ : C - - + ~

w h e r e C a n d D a r e n e w a n d d i s t i n c t. C o n t i n u i n g i n t h i s w a y , a l w a y sa d d i n g n e w s y m b o l s , f o r m G 2 e q u i v a l e n t t o G~ , n o n - s . e , i f G~ i s n o n - s . e .,a n d m e e t i n g t h e s e c o n d o f t h e r e g u l a r i t y c o n d i t i o n s .

I f G ~ c o n t a i n s a r u l e A --+ a~ oe,~(a~ ~ I , n > 2 ) , r e p l a c e i t b y t h er u l e s

R ~ : A --- + a l . . . a , ~ _~ B

w h e r e B i s n e w . C o n t i n u i n g i n t h is w a y , f o r m G a .I f G a c o n t a i n s A -- + a b ( a ~ I ~ b ) , r e p l a c e i t b y A ~ B C , B - -- + a ,C - + b , w h e r e B a n d C a r e n e w . I f G a c o n t a i n s A - -- + a B , r e p l a c e i t b y


14/31

150 CHOMSKYA -+ CB, C --> a, w h e r e C is n e w . I f i t c o n t a i n s A --+ Ba, r e p l a c e t h i s b yA ~ BC, C --+ a, w h e r e C i s n e w . C o n t i n u i n g i n t h i s w a y f o r m G 4 . G 4t h e n i s t h e g r a m m a r G* r e q u i r e d f o r t h e t h e o r e m .T h e o r e m 5 a s s er t s in p a r t i c u l a r t h a t a ll t y p e 2 l a n g u a g e s c a n b e g e n-e r a t e d b y g r a m m a r s w h i c h y i el d o n l y t re e s w i t h n o m o r e t h a n t w ob r a n c h e s f r o m e a c h n o d e . T h a t is , f r o m t h e p o i n t o f v i e w o f g e n e r a t i v ep o w e r , w e d o n o t r e s t ri c t g r a m m a r s b y r e q u i r i n g t h a t e a c h p h r a s e h a v ea t m o s t tw o i m m e d i a t e c o n s t i t u e n t s ( n o t e t h a t i n a r e g u l a r g r a m m a r , a" p h r a s e " h a s o n e i m m e d i a t e c o n s t i t u e n t j u s t i n e a s e i t is i n t e r p r e t e d a sa w o r d o r m o r p h e m e c l as s, i. e. , a l o w e s t l ev e l p h r a s e ; a n i m m e d i a t ec o n s t i t u e n t i n t h i s c a s e i s a m e m b e r o f t h e c l a s s) .

D E F I N I T I O N 9 . S u p p o s e t h a t 2 ~ i s a f i n i t e s t a t e M a r k o v s o u r c e w i t h as y m b o l e m i t t e d a t e a c h i n t e r - s t a t e t r a n s i t i o n ; w i t h a d e s i g n a t e d i n i t i a ls t a t e S o a n d a d e s i g n a t e d f i n a l s t a t e S y ; w i t h # e m i t t e d o n t r a n s i t i o nf r o m S o a n d f r o m S f t o S o , a n d n o w h e r e e l s e ; a n d w i t h n o t r a n s i t i o nf r o m S f e x c e p t t o S o . D e f i n e a s e n t e n c e a s a s t r i n g o f s y m b o l s e m i t t e da s t h e s y s t e m m o v e s f r o m S o t o a f i r s t r e c u r r e n c e o f S o . T h e n t h e s e to f s e n t e n c e s t h a t c a n b e e m i t t e d b y Z i s a f i n i t e s t a t e l a n g u a g e , z

S i n c e R e s t r i c t i o n 3 l i m i t s t h e r u l e s t o t h e f o r m A - - + a B o r A - ~ a ,w e i m m e d i a t e l y c o n c l u d e t h e f o l l o w i n g .

T H E O R E M 6 . T h e t y p e 3 l a n g u a g e s a r e t h e f i n i t e s t a t e l a n g u a g e s .P R O O F . S u p p o s e t h a t G i s a t y p e 3 g r a m m a r . W e i n t e r p r e t t h e s y m b o l s

o f V ~ a s d e s i g n a t i o n s o f s t a t e s a n d t h e s y m b o l s o f V r a s t r a n s i t i o n s y m -b o l s . T h e n a r u l e o f t h e f o r m A - - ~ a B i s i n t e r p r e t e d a s m e a n i n g t h a t ais e m i t t e d o n t r a n s i t io n f r o m A t o B . A n # S # - d e r i v a t i o n o f G c a n in -v o l v e o n l y o n e a p p l i c a t i o n o f a r u l e o f t h e f o r m A --+ a . T h i s c a n b e i n -t e r p r e t e d a s i n d i c a t i n g t r a n s i t i o n f r o m A t o a f i n al s t a t e w i t h a e m i t t e d .T h e f a c t t h a t # b o u n d s e a c h s e n t en c e o f L ~ c an b e u n d e r s t o o d a s in d i c a t -i n g t h e p r e s e n c e o f a n i n i t i a l s t a t e S o w i t h # e m i t t e d o n t r a n s i t i o n f r o mSo t o S , a n d a s a re q u i r e m e n t t h a t t h e o n l y t ra n s i t i o n f r o m t h e f i na ls t a t e i s t o S o , w i t h # e m i t t e d . T h u s G c a n b e i n t e r p r e t e d a s a s y s t e mo f t h e t y p e d e s c r i b e d in D e f i n i t i o n 9. S im i l a r l y , e a c h s u c h s y s t e m c a nb e d e s c r i b e d a s a t y p e 3 g r a m m a r .

lO A l t e r n a t i v e l y , ~ c a n b e c o n s i d e r e d a s a f i n i t e a u t o m a t o n , a n d t h e g e n e r a t e df i n i t e s t a t e l a n g u a g e , a s t h e s e t o f i n p u t s e q u e n c e s t h a t c a r r y i t f r o m S o t o a f i r s tr e c u r r e n c e o f S 0 . C f . C h o m s k y a n d M i l l e r ( 1 9 5 8 ) f o r a d i s c u s s i o n o f p r o p e r t i e s o ff i n i t e s t a t e l a n g u a g e s a n d s y s t e m s t h a t g e n e r a t e t h e m f r o m a p o i n t o f v i e w r e -l a t e d t o t h a t o f t h i s p a p e r . A f i n i t e s t a t e l a n g u a g e i s e s s e n t i a l l y w h a t i s c a l l e d i nK l e e n e ( 1 9 5 6 ) a " r e g u l a r e v e n t . "


15/31

O N C E R T A I N F O R M A L P R O P E R T I E S O F G R A M M A R S 151Restriction 3 limits the rules to the form A ~ a B or A --~ a. From

Theorem 5 we see that Restriction 2 amounts to a limitation of the rulesto the form A ~ a B , A - -~ a , or A - -~ B C (with the first type dispensable).Hence the fundamental feature distinguishing type 2 grammars (sys-tems of phrase structure) from type 3 grammars (finite automata) isthe possibility of rules of the form A ~ B C in the former. This leads toan important difference in generative power.

THEORE~ 7. There exist type 2 languages th at are not type 3 lan-guages. (Cf. Chomsky, 1956, 1957.)In Chomsky (1956), three examples of non-type 3 languages werepresented. Let L1 be the language containing just the strings a ' b ~ ; L ~ ,the language containing just the strings x y , where x is a string of a'sand b's and y is the mirror image of x; L~, the language consisting of allstrings x x where x is a string of a's and b's. Then L1, L2, and L3 arenot type 3 languages. LI and L2 are type 2 languages (cf. Chomsky,1956). L3 is a type 1 language but not a type 2 language, as can be shownby proofs similar to those of Lemma 2 and Theorem 4.1~

Suppose that we extend the power of a finite automaton by equippingit with a finite number of counters, each of which can assume infinitelymany positions. We permit each counter to shift position in a fixed waywith each inter-state transition, and we permit the next transition to bedetermined by the present state and the present readings of the counters.A language generated (as in Definition 9) by a system of this sort (whereeach counter begins in a fixed position) will be called a c o u n t e r l a n g u a g e .Clearly L1, though not a finite state (type 3) language, is a counterlanguage. Several different systems of this general type are studied bySchiitzenberger, (1957), where the following, in particular, is proven.THEOREM 8. L2 is not a counter language.

Thus there are type 2 languages that are not counter languages.TMTosummarize, L~ is a counter language and a type 2 language, but not atype 3 (finite state) language; L2 is a type 2 language but not a counterlanguage (hence not a type 3 language) ; and L3 is a type 1 language butnot a type 2 language.

11 I n C h o m s k y ( 19 5 6, p . 1 19 ) a n d C h o m s k y ( 19 57 , p . 3 4) , i t w a s e r r o n e o u s l ys t a t e d t h a t L a c a n n o t b e g e n e r a t e d b y a p h r a s e s t r u c t u r e s y s t e m . T h i s i s t r u e f o ra ty p e 2 , b u t n o t a t y p e 1 p h r a s e s t r u c t u r e s y s t e m .

12 T h e f u r t h e r q u e s t i o n w h e t h e r a l l c o u n t e r l a n g u a g e s a r e t y p e 2 l a n g u a g e s ( i. e. ,w h e t h e r c o u n t e r l a n g u a g e s c o n s t i t u t e a s t e p b e t w e e n t y p e s 2 a n d 3 i n t h e h i e r-a r c h y being considered here) has not been investigated.


16/31

152 CHOMSKYF r o m T h e o r e m s 2 , 3 , 4 a n d 7 , w e c o n c l u d e :T HE OR EM 9 . R e s t r i c t i o n s 1 , 2 a n d 3 a re i n c r e a s i n g l y h e a v y . T h a t i s,

t h e i n cl u si o n in T h e o r e m 1 is p r o p e r i nc lu s io n , b o t h f o r g r a m m a r s ( t r iv -i a l l y ) a n d f o r l a n g u a g e s .T h e f a c t t h a t L~ is a t y p e 2 l an g u a g e b u t n e i t h e r a t y p e 3 n o r a c o u n t e r

l a n g u a g e i s i m p o r t a n t , s in c e E n g l i s h h a s t h e e s s e n t i a l p r o p e r t i e s o f L ~( C h o m s k y , 1 95 6, 1 9 5 7 ). W e c a n c o n c l u d e f r o m t h i s t h a t f in i te a u t o -m a t a ( e v e n w i t h a fi n it e n u m b e r o f i n fi n it e c o u n t e r s ) t h a t p r o d u c e s e n -t e n c e s f r o m " l e f t t o r i g h t " i n t h e m a n n e r o f D e f i n i t io n 9 c a n n o t c o n s t i-t u t e t h e c l a ss F ( c f. S e c. 1 ) f r o m w h i c h g r a m m a r s a r e d r a w n ; i .e ., t h ed e v i c es t h a t g e n e r a t e l a n g u a g e c a n n o t b e o f t h i s c h a r a c t e r .

S E C T I O N 6T h e i m p o r t a n c e o f g a i n in g a b e t t e r u n d e r s t a n d i n g o f t h e d i ff er e nc e i n

g e n e r a t i v e p o w e r b e t w e e n p h r a s e s t r u c t u r e g r a m m a r s a n d f i n i t e s t a t es o u r c e s i s c l e a r f r o m t h e c o n s i d e r a t i o n s r e v i e w e d i n S e c . 5 . W e s h a l ln o w s h o w t h a t t h e s o u r ce o f t h e e x ce ss of p o w e r o f t y p e 2 g r a m m a r s o v e rt y p e 3 g r a m m a r s l ie s i n t h e f a c t t h a t t h e f o r m e r m a y b e s e lf - e m b e d d i n g( D e f i n i t io n 7 ) . B e c a u s e o f T h e o r e m 5 w e c a n r e s tr i c t o u r a t t e n t i o n t or e g u l a r g r a m m a r s .

Construction: L e t G b e a n o n -s .e , r e g u l a r ( t y p e 2 ) g r a m m a r . L e tK = { ( A 1 , . . . , A ~ ) [ m = 1 o r,

f o r 1 ~Ai+l~ a n d A ~ # A ~ } .W e c o n s t r u c t t h e g r a m m a r G ' w i t h e a c h n o n t e r m i n a l s y m b o l r ep r e s e n te di n t h e f o r m [ B 1 . . " B~]~(i = 1, 2 ) , w h e r e t h e B / s a r e i n t u r n n o n t e r m i -h a l s y m b o l s o f G , a s f o l lo w s : 13

S u p p o s e t h a t ( B I , . . . , B n ) C K .( i ) I f B n --+ a in G , th en [B1 . . " B~]~ -+ a[B1 . . . B~]2.

( i i ) If B~ ---+ CD w h e r e C # B~ ~ D ( i


17/31

O N C E R T A I N F O R M A L P R O P E R T I E S O F G R A M M A R S 153

(iii) If B , ~ C D where B~ = D for some i < n, then(a) [B1 ... B~h-+ [B~ ... B,~C]I(b) [B1 ... B . C ] 2 -- ~ [ B~ . . . B , ] I .(iv) If B ~ ---> C D where B~ -- C for some i ~ n, then( a ) [B ~ . . . B ~] ~ ~ [ B I . . . B , D ] ~(b) [B~ ... B ,~ D ]2 ~ [ B I . . . B~]2.

We shall prove that G' is equivalent to G (when sfightly modified).The character of this construction can be clarified by consideration

of the trees generated by a grammar (cf. Sec. 3). Since G is regular andnon-s.e., we have to consider only the following configurations:

(a) (b) (c) (d)B1 B1 B1 B1

/ 1 \ / 1 \ / ! \ / 1 \B2 B2 B~ B2/ 1 \ / 1 \ / 1 \ / 1 \

: : :

B~ B~ B~ B~i / \ / \a C D E1 B~+I B~+I E1

E~ Bi+~ B~+~ E2

( 3 )

Bn Bn/ \ / \C B ~ B i D

where at most two of the branches proceeding from a given node arenon-null; in case (b), no node dominated by B~ is labeled B i ( i


18/31

154 CI-IOMSKYB ~ i n a p a r t i c u l a r b r a n c h o f a t r e e, t h e c o n s t r u c t i o n a s s o c ia t es t w o n o n -t e r m i n a l s y m b o l s [B 1 . . . B ~ ] I a n d [B 1 . " B ~ ]2 , w h e r e B1 , , . . , B ~ - I a r et h e l ab e ls o f t h e n o d e s d o m i n a t i n g t h i s o c c u r re n c e o f B ~ . T h e d e r i v a -t i o n i n G ' c o r r e s p o n d i n g t o t h e g i v e n t r e e w i l l c o n t a i n a s u b s e q u e n c e( z[B1 . . . Bn]~ , . . . z x [B~ . . . B~ ]2) , w h e re B~ ~ x and z i s t he s t r i n gp r e c e d i n g t h i s o c c u r r e n c e o f x i n t h e g i v e n d e r i v a t i o n i n G . F o r e x a m p l e ,c o r r e s p o n d i n g t o a t r e e o f t h e f o r m

SA B ( 4 )

I Ja bg e n e r a t e d b y a g r a m m a r G , t h e c o r r e s p o n d i n g G ' w il l g e n e r a t e t h e d e r i v a -t i o n ( 5 ) w i t h t h e a c c o m p a n y i n g t r e e:

1. [S] 1 [S]1I2. [SA]t ( i i a ) [ S A h/ \3. a[SA]2 ( i ) a [SA ]2] ( 5 )4. a[SB]I ( l i b ) [SB]I5 . ab[SB]2 ( i ) b [SB]2

I6. ab[S]2 (iic) [S]2w h e r e t h e s t e p o f t h e c o n s t r u c t i o n p e r m i t t i n g e a c h l i n e i s i n d i c a t e d a tt h e r i g h t .

W e n o w p r o ce e d to e s t a b li sh t h a t t h e g r a m m a r G ' g iv e n b y t h i s c o n-s t r u c t i o n i s a c t u a l l y e q u i v a l e n t ( w i t h s l i g h t m o d i f i c a t i o n ) t o t h e g i v e ng r a m m a r G . T h i s r e su l t , w h i c h r e q u ir e s a l o n g s e q u e n ce o f i n t r o d u c t o r yl e m m a s , is s t a t e d i n a f o ll o w i n g p a r a g r a p h a s T h e o r e m 1 0. F r o m t h i sw e w i l l c o n c l u d e t h a t g i v e n a n y n o n - s . c , t y p e 2 g r a m m a r , w e c a n c o n -s t r u c t a n e q u i v a l e n t t y p e 3 g r a m m a r ( w i t h m a n y v a c u o u s tr a n s i ti o n sw h i c h a r e, h o w e v e r , e l im i n a b l e ; cf . C h o m s k y a n d M i ll e r, 1 9 5 8) . F r o mt h i s fo l lo w s t h e m a i n r e s u l t o f t h e p a p e r ( T h e o r e m 1 1), n a m e l y , t h a tt h e e x t r a p o w e r o f p h r a s e s t r u c t u r e g r a m m a r s o v e r fi n it e a u t o m a t a a sl a n g u a g e g e n e r a to r s lie s i n th e f a c t t h a t p h r a s e s t r u c t u r e g r a m m a r s m a yb e s e l f- e m b e d d i n g .


19/31

ON CERT AIN FORMAL PROPE RTIE S OF GRAMMARS 155

L E M M A 3 . I f ( A 1 , " , A m ) ~ K , w h e r e K is a s i n t h e c o n s t r u c t i o n ,t h e n A j ~ A k ~ b , f o r 1 =< j


20/31

156 CHOMSKY, t~ , Q i - - 1 = [ B 1 " ' " B k ] l , Q i = [ B 1 " ' " B k + i l a , Q+I = [B 1 . . - B k + l ] 2 ,.Q i+~ = [B1 Bk ]2 . . '. B k - -~ B k+ ID f o r s o m e D , B k --+ E B k + a , f o r s o m eE , w h i c h c o n t r a d i c t s t h e a s s u m p t i o n t h a t G is r e g u l a r . . ' , i = 1 .( b ) C o n s i d e r n o w ( I ) . S i n c e i = 1 , r = 2 . . ' . B , --+ z 2 . B y a s s u m p t i o na b o u t t h e C ? s a n d m a p p l i c a t io n s o f L e m m a 4 , a n d ( i) o f t h e c o n s t r u c -t i o n , [C a " ' " C,~B1]a - -* z2[C~ " " CraB1]2 . S i n c e C i --~ A i + a C i + l ( C i ~ C jf o r 1 _-< i < j _< m + 1 , s i n c e ( C a , - . - , C ~ + ~ ) C K b y a s s u m p t i o n ) , i tf o l l o w s t h a t [C a - . . C,~Ba]2 ~ [Ca . . . C , ,]~ - -~ [CI . . . Cm-~]2 . . . - --* [Ca]2.. ' . ( [C~ . . . C m B a ]a , z 2 [ C a " " C raB s]2, z 2 [ C ~ . " C m ] 2 , " ' , z_~[Ca]2) is t h er e q u i r e d d e r i v a t i o n .

( e ) C o n s i d e r n o w ( I I ) . S i n c e i = 1 , B ~ --~ z2 a n d [ B n ]a --~ z2[Bn]~, b y( i ) o f c o n s t r u c t i o n . . ' . ([B n]~ , z2 [B n]2) i s t h e r e q u i r e d d e r i v a t i o n .T h i s p r o v e s t h e l e m m a f o r t h e e a s e o f z~ o f l e n g t h 1 .

S u p p o s e i t is t r u e i n a l l c a s e s w h e r e z~ i s o f l e n g t h < t.C o n s i d e r ( I ) . L e t D b e s u c h t h a t z~ i s o f l e n g t h t. I f n o n e o f C a , ,

C ~ a p p e a r s i n a n y o f t h e Q 's i n D , t h e n t h e p r o o f i s j u s t l ik e ( a ) , a b o v e .S u p p o s e t h a t ~ j is th e e a r l i e s t l in e in w h i c h o n e o f C a , , C ~ , s a y C k ,a p p e a r s i n Q j . j > 1 , s in c e C 1, . - ' , C ~ ~ B a . B y a s s u m p t i o n o f n o n -s . c . , t h e ru l e Q~ '- I - -+ a j Q j u s e d t o f o r m ~ - c a n o n l y h a v e b e e n i n t ro -d u c e d b y ( l i b ) . ~6 . '. Q~-- j = [Ba " " B n E ] 2 , Q j = [ B a ' . . B ,~ C ~]a , B n" - " E C k .B u t C a , " ' " , C ~ d o n o t o c c u r i n Q 1 , " ' " , Q ~-a a n d

( C a , - ' - , C ~ , B ~ ) ~ K .. ' . , b y L e m m a 4 ,

( [C~ . - . C ~ B a] ~ , . . . , zj_~ [C ~ . . . C r a B s . . . B ,~ E ]~ ) ( 6 )i s a d e r i v a t i o n . F u r t h e r m o r e z j _l i s n o t n u l l , s in c e t h e r e i s a t l e a s t o n et r a n s i t i o n f r o m [ . . . ] 1 t o [ . - . ] 2 i n ( 6 ), w h i c h m u s t t h e r e f o r e h a v e b e e ni n t r o d u c e d b y ( i) o f t h e c o n s t r u c t io n . B u t B ,~ ---+ E C ~ . . ' .

[C ~ . . . C r a B s . . . B ,E ]~ ----> [ C a ' . . C ~]~ ( 7 )[ b y ( ii ib )] . F u r t h e r m o r e w e k n o w t h a t

( [ B ~ . . - B ~ C ~ ] I , . . . , g + a [ B , ] 2 ) ( S )~ I t can on ly have been in t rodu ced b y ( i ia ) , ( l ib ) , ( i li a ) , ( i va ) , o r C~ wi l l ap -p e a r i n Q~._, . Su ppo se (i ia ) . . ' . Qi-* = [B~ . . . Bq] , , Qi = [B~ . . . BqC~], , a n d

B q ~ C ~ D . B ut C~ ~ eb b . C on t ra . by Lem m a 5 ( a ). Suppose ( il ia ) . Same. Suppose( i v a ) . . ' . Qi-* = [B~ . . . Bd= , Qi = [B ~. . . Bi+q]~ (q >= 1), B~+q_t ~ B~B~+q, w h e r eC~ = B ~ (1 < s =< q). B u t C~ ~ B~ , # I . . ' . C~ ~ ~colBi+q_lW2 - - > o ~ l B i B i + q C O 2:=~ ~bwaCte.o~Bi+qo~ , c o n t r a . . ' , i n t r o d u c e d b y ( l ib ) .


21/31


m u s t b e a d e r i v a t i o n , s i n c e [B 1 . . . B~C~]I = Q j ; i .e . , ( 8 ) i s j u s t t h e t a i le n d ( 5 , " " , C r) o f D , w i t h t h e i n i t i a l s e g m e n t z j d e l e t e d f r o m e a c h o f ~ ., , C r . S i n c e z j _ l i s n o t n u l l , z~ +~ i s s h o r t e r t h a n z ~ , h e n c e i s o fl e n g t h < t . A l s o , C k ~ x B 1 , b y a s s u m p t i o n . . ' , b y i n d u c t i v e h y p o t h e s i s( I I ) , t h e r e i s a d e r i v a t i o n

( [c k ]l , . . - , d + ' [ c & ) ( 9). ' . b y i n d u c t i v e h y p o t h e s i s ( I ) , t h e r e is a d e r i v a t i o n

( I V 1 ' ' ' C k ] l , . ' . , z~-t -1 [C1 12 ) (1 0 )C o m b i n i n g ( 6 ), ( 7 ) , ( 1 0 ) , w e h a v e t h e r e q u i r e d d e r i v a t i o n .

C o n s i d e r n o w ( I I ) . I f n - 1 o r t h e r e i s n o s u c h d e r i v a t i o n of l e n g t hl , t h e p r o o f i s t r i v i a l . A s s u m e n > 1 .L e t ~ j c o n t a i n t h e f i r s t Q o f t h e f o r m [ B , . . - B m ],(j > 1, m


22/31

158 CItOMSKY

S u p p o s e q = j . . ' . Qq-1 = [B 1 . - . B . + , ] ~ - - ~ a ~ [B 1 . . . B , ~] z = Q j , w h e r em < n < n + v . . ' . t h i s r u l e c a n o n l y h a v e b e e n i n t r o d u c e d b y ( i i ib )o f t h e c o n s t r u c t i o n . . ' , i = 2 a n d B ~+ ~ -I - ~ B ~ + ~ B ~ .C a s e 1. S u p p o s e m = n . . ' .

[B ,~ . . . B ,+ ~]~ - -~ [B , ] I = [B , ]~ (1 5 )C o m b i n i n g ( 1 4 ) , ( 1 5 ) , ( 1 2 ) , W e h a v e t h e r e q u i r e d d e r i v a t i o n .C a s e 2 . S u p p o s e m < n . . ' . B , ~ B n , - . . , B n + ~ . . ' .

[ B , . . . B ,+ ~]2 ~ [ B , . . . B , + ~ _ ~ B , ] I ( 1 6 )b y ( i i b ) . W e h a v e s e en t h a t B , ~ y B ~ . .'. B , + ~ _ ~ w ~ B ~ . .'. f o rs < v - 1 , B , + , - -~ E ~ B , + , + ~ , b y L e m m a 5 .

B u t ( 1 2 ) i s a d e r i v a t i o n w h e r e _s+~, is o f l e n g t h < t . . ' . b y i n d u c t i v eh y p o t h e s i s ( I ) t h e r e is a d e r i v a t i o n

( l B , . . . B ~ + ~ _I B ~] ~ , . . . , z~+*[Bn]2) (17)C o m b i n i n g ( 1 4 ) , ( 1 6 ) , ( 1 7 ) , w e h a v e t h e re q u i r e d d e r i v a t io n .S u p p o s e , f i n a ll y , t h a t q = k . W e h a v e s e e n t h a t i n t h i s c a s e z~ = z k .

B u t Q q - 1 = [ B I . . . B ~ + ~ ] - -~ a k [ B i . . . B m ]~ , w h e r e m =< n < n + v . . ' .t h i s ru l e c a n o n l y h a v e b e e n i n t r o d u c e d b y ( ii c) o r ( i v b ) . I n e i t h e r e as e ,i = 2 , m = n , v = 1 , a n d O ~_ , = [ B~B , + ~] 2 - - ~ a k [B , ]2 . C o m b i n i n g t h i sw i t h ( 1 4 ) w e h a v e t h e re q u i r e d d e r i v a t i o n .

W e h a v e t h u s s h o w n t h a t t h e l e m m a i s t r u e i n c a s e z~ i s of l e n g t h 1 ,a n d t h a t i t is t r u e f o r z~ o f l e n g t h t o n t h e a s s u m p t i o n t h a t i t h o l d s f o rz , o f l e n g t h < t . T h e r e f o r e i t h o ld s f o r e v e r y d e r i v a t i o n D .

L EI~ MA 7 . S u p p o s e t h a t D = ( ~ , , ~ ) i s a d e r i v a t i o n i n G ' w h e r eQI = [B111 . T h e n

( I ) if ~ = [ B ~ h , (C~, . . . , C~+~) C K , C~ ~ C~+~A,+~ (1 < i < m ) ,a n d C ~ + I = B ~ , t h e n t h e r e i s a d e r i v a t i o n( [C~]~ , . - . , z ~[ C ~ . . . C m B~]~) i n G ' .

( I I ) i f , = [B ~ . . . B . ]2 a n d B ~ ~ B ~ x , t h e n t h e r e is a d e r i v a t i o n( [ B , ] I , . . - , z ~[ Bn ]~ ) i n G ' .

T h e p r o o f is a n a l o g o u s t o t h a t o f L e m m a 6 . I n t h e i n d u c t i v e s t e p ,c a s e ( I ) , w e t a k e Q ~ a s t h e l a s t o f t h e Q ' s in w h i c h o n e o f C ~ , . - . , C ~


23/31

O N C E R T A I N F O R M A L P R O P E R T I E S O F G R A M M A R S 159a p p e a r s , a n d i n s t e a d o f (i i ib ) i n ( 7 ) , w e f o r m

[ C i " " G ] 2 - - * [ G " " C ,~ B I . . . B ~E ]~b y ( i r a ) . T h e p r o o f go e s t h r o u g h a s a b o v e , w i t h s i m i l a r m o d i f i c a ti o n st h r o u g h o u t . I n c a s e ( I I ) o f t h e i n d u c t i v e s t e p w e le t Q j b e t h e l as t Q o ft h e f o r m [ B 1 . . . B , ~ ]2 ( j < r , m


24/31

160 CHOMSKP RO O F. B y h y p o t h e s i s , t h e r e a r e d e r i v a t i o n s

( [ A ] I , . . . , x [A ]= ) ( 1 8 )( [ B ] I , . . . , y [B]~) ( 19 )

i n G ' .C a s e 1. S u p p o s e A ~ C ~ B . T h e n b y L e m m a 8 , t h e r e a r e d e r i v a t i o n s

( [ C ] 1 , . . . , x[CA ]~) (20 )( [ C B ] I , ' ' - , y [C ]2 ) ( 2 1 )

i n G '. B y ( i i b ) o f t h e c o n s t r u c t i o n ,[ C A ] ~ - - . [ C B ] I ( 2 2 )

C o m b i n i n g ( 2 0 ) , ( 2 2 ), a n d ( 2 1 ) , w e h a v e t h e r e q u i r e d d e r i v a t i o n .C a s e 2 . C = A . .', C ~ B b y a s s u m p t i o n o f r e g u l a r it y o f G . B y L e m m a

8 , c a se ( a ) , w e h a v e a g a in t h e d e r i v a t i o n ( 2 1 ) . B y ( i r a ) o f t h e c o n -s t r u c t i o n ,

[A]~ = [C]2 - * [ C B ] ~ . ( 2 3 )C o m b i n i n g ( 1 8 ) , ( 2 3 ) , ( 2 1 ) w e h a v e t h e re q u i r e d d e r i v a t i o n .

C a s e 3 . C = B . .'. C ~ A . B y L e m m a 8 , c as e ( b ) , w e h a v e ( 2 0 ).B y ( i i i b ) ,[CA]~ - -~ [C]~ = [B]~. (24 )

C o m b i n i n g ( 2 0 ) , ( 2 4 ) , ( 1 9 ) , w e h a v e t h e r e q u i r e d d e r i v a t i o n .S i n c e C ---+ C C is ru l e d o u t b y a s s u m p t i o n o f r e g u l a r i ty , t h e s e a r e t h e

o n l y p o s s i b l e c a s e s .L E ~M A 1 0 . I f D 1 = ( ~ , " - , ~ r ) i s a X l ~ l - d e r i v a ti o n , w h e r e X l

I ~ ~ , t h e n t h e r e i s a d e r i v a t i o n D 2 = D ~ * D ~ = ( g , ~ , . . . , ~r ) s u c ht h a t t r = ~ r , D 3 i s a x ~ - d e r i v a t i o n a n d D ~ i s a n w l - d e r i v a t io n .])R O O F. S i n c e f o r i > 1 , q~ i s f o r m e d f r o m ~ _ ~ b y r e p l a c e m e n t o f as i n g l e s y m b o l o f ~ _ ~ ,~7 w e c a n c l e a r l y f i n d X ~ , ~ s . t. ~ = x ~ w h e r ee i th e r ( a ) x i = x~- i an d w~-1 --~ ~ or ( b ) x i-~ --~ x~ an d ~i = o~_~( X ~ - ~ - ~ = ~ - ~ ) . T h e n D ~ i s t h e s u b s e q u e n c e o f (X ~ , " ' " , X ,) f o r m e db y d r o p p i n g r e p e ti t i o n s a n d D4 is t h e su b s e q ue n c e o f ( ~ , . . . , ~ )f o r m e d b y d r o p p i n g r e p e t i t i o n s .

LEY~MA 1 1. I f G ' i s f o r m e d f r o m G b y t h e c o n s t r u c t i o n , t h e n e v e r ya - d e r i v a t i o n D i n G i s r e p r e s e n t e d i n G ' .

~ Which , how ever , m ay not be un ique ly de te rmined . C ompare foo tno te 8 .


25/31

O N C E R T A I N F O R M A L P R O P E R T I E S O F G R A M M A R S 1 6 1P RO O F. O b v i o u s , i n c a s e D c o n t a i n s 2 l in e s. S u p p o s e t r u e f o r a l l d e r i v a -

t i o n s o f f e w e r t h a n r l in e s ( r > 2 ) . L e t D = ( ~ i , " ' " , ~ ) , w h e r e ~1 = a .S i n c e r > 2 , a = A , ~2 = B C . . ' . ( ~ , . . , ~ ) i s a B C - d e r i v a t i o n . B yL e l n m a 1 0, t h e r e i s a D z = D ~ * D ~ = (~2 , " " , ~k~) s . t . D , i s a B -d e r i v a -t i o n , D 4 a C - d e r i v a t i o n , a n d ~kr = ~ r B y i n d u c t i v e h y p o t h e s i s , b o t h D 3a n d D 4 a r e r e p r e s e n t e d i n G ' . B y L e m m a 9 , D i s r e p r e s e n t e d i n G ' .

I t r e m a i n s t o s h o w t h a t if ( J A I l , - . , x [A ] 2) is a d e r i v a t i o n i n G ' , t h e nt h e r e i s a d e r i v a t i o n ( A , . - . , x ) i n G .

L EM M A 1 2 . ~s S u p p o s e t h a t G ' i s f o r m e d b y t h e c o n s t r u c t i o n f r o m G ,r e g u l a r a n d n o n - s . c . , a n d t h a t

( a ) D = ( ~ l , - . . , ~ l , - - . , ~ q , - - - , ~ m 2 , . . . , ~ , ) is a d e r i v a t i o ni n G ' , w h e r e Q~ = [A 1 ], Q m l = Q ~ = [A 1 - . . A ~ ] n , Q ~ = [A 1 . . - A j ] ~ ,

( b ) t h e r e i s n o u , v s . t . u v , Q ~ = Q ~ = [ B1 . . - B ~ ] t , a n d s < k 19( c ) f o r m l < u < m 2 , i f Q ~ = [ A 1 . . . A , ] t , t h e n s > f 0

T h e n i t f o ll o w s t h a t( A ) i f n - - 2 , t h e r e i s a n m 0 < m ~ s u c h t h a t Q ~0 = [A ~ . - . A k ] t( B ) i f n = 1 , t h e r e is a n m ~ > m 2 s u c h t h a t Q m = [ A j . . . A k]2( C ) j = ~P R OO F . ( A ) S u p p o s e n = 2 . A s s u m e ~ t o b e t h e e a r li e s t l in e t o c o n -

t a i n [ A 1 . . . A ~ ] 2 . C l e a r l y t h e r e i s a n ~ =< m ~ s . t. Q ~ --- [A 1 . . . A k + ~ ]~ ,Q .a -~ = [ A 1 . . . A k + t ] ~ ( t > 0 ) . I f t h e r e i s n o m0 < ~ s . t.Q ~ 0 = [ A 1 . - . A k ] l ,

t h e n t h e r e m u s t b e a u < r ~ s . t . Q ~ = [A ~ A ~ ] : ,Q ~ + I = [ A 0 . . " A ~ _ I B o . . . B , ~ ]I ,

w h e r e ~ + ~ is f o r m e d b y ( i r a ) o f t h e c o n s t r u c t i o n , A 0 = I , B0 = A k ,m = 1 , a n d s < /c ( s i n c e ( i r a ) g i v e s t h e o n l y p o s s i b i l i t y f o r i n c r e a s i n gt h e l e n g t h o f Q b y m o r e t h a n 1 ) . . ' . B ,~ _ I --> A ~ B . . . . ". A ~ ~ ~ A ~ B ~ .

B u t Q ~ = [A ~ . . . A ~ ]2 c a n n o t r e c u r in a n y l in e f o l l o w i n g f ~ [ t h isw o u l d c o n t r a d i c t a s s u m p t i o n ( b ) ] . T h e r e f o r e , iu s t a s a b o v e , th e r e m u s tb e a v > m2 s . t . Q~_~ = [A0 . . . A , _ ~ C o . . . C , ~, ]~ , Q , = [ A ~ . . . A ~ ] ~ ,w h e r e ~ i s f o r m e d b y ( ii ib ) o f t h e c o n s t r u c t i o n , A 0 = I , Co = A ~ ,m ' > 1 , p < s ( s in c e ( i i ib ) g i v e s t h e o n l y p o s s i b i l i t y f o r d e c r e a s i n g t h e

~s W e c o n t i n u e t o e m p l o y C o n v e n t i o n 2 , a b o v e .19 T h a t i s , Q ~ I = Q ~ : i s t h e s h o r t e s t Q o f D t h a t r e p e a t s .~0 T h a t i s , Qq s t h e s h o r t e s t Q o f t h i s f o r m b e t w e e n ~ i a n d f ~ : .~ T h a ~ i s , Qq s n o t s h o r t e r t h a n Q ~ I = Q ~ e -


26/31

162 CttOMSKl e n g t h o f Q b y m o r e t h a n 1 ) . . ' . C m , -1 ---> C m , A ~ . B u t A ~ ~ ~ 1 C , ~ , -1 ~ 2( L e m m a 3 ) . . ' . A ~ ~ e l C ~ , A p ~ 2 ~ ~ 1 C ~ , ~ A ~ 4 ~ ~ I C ~ , ~ 3 ~ A ~ B , ~ 4 .C o n t r a . , s i n c e G i s a s s u m e d t o b e n o n - s . c .. '. th er e i s an m0 < r~ _= m~ s . t . Q~0 = [At . . Ak]~

( B ) S u p p o s e n = 1 . P r o o f i s a n a l o g o u s .( C ) ( I ) . S u p p o s e n = 2 . S u p p o s e j < k . S u p p o s e i ( i n Q q) i s 2 . C l e a r l y

t h e r e m u s t b e a v > m ~ s . t . e i t h e r Q ~ = [A ~ - . . A ~.]2 [ w h i c h c o n t r a d i c t sa s s u m p t i o n ( b ) ] o r Q ,_ ~ = [A 0 . . . A j - ~ C o . . . C ~ ]~ , Q ~, = [ A 1 . . . A ~ ,] ~ ,w h e r e f~ i s f o r m e d b y ( i ii b ) o f t h e c o n s t r u c t i o n , A o .~- I , C o = A i ,m => 1 , p < j [ as i n t h e s e c o n d p a r a g r a p h o f t h e p r o o f o f ( A ) ] . S u p p o s et h e l a t t e r . . ' . C ~ _ ~ - -* C m A ~ . . '. A j ~ ~ C m A , . F u r t h e r m o r e , s i n c ep < j , A ~ ~ ~ C ~ A ~ .F r o m a s s u m p t i o n ( c ) a n d a s s u m p t i o n o f r e g u l a r i t y o f G , i t f o ll o w s t h a t~q+~ c a n o n l y h a v e b e e n f o r m e d b y ( i v a ) o f t h e c o n s t r u c t i o n . . ' . Q q + l =[A0 . . " A i _ ~ D o . . " D t ] : t , w h e r e A 0 = I , D o = A ~ . , t ~ 1 . . ' . D t _ ~ - - - > A i D t .. '. A i ~ x a A ~ D t ~ , . .' . A i ~ o ~ f C ~ o ~ A i x ~ D t ~ , a n d A~. i s s e l f - e m b e d d e d ,

c o n t r a r y t o a s s u m p t i o n .S u p p o s e t h a t i ( in Q q ) i s 1. B y ( A ) , t h e r e i s a n m 0 < m ~ s . t . Qm 0 =

[A ~ . - . A , ] ~ . . ' . t h e r e i s a u < m 0 s . t . e i t h e r Q~ = [A 1 - . . A ~]~ [ w h i c hc o n t r a d i c t s a s s u m p t i o n ( b ) ] o r Q , = [A ~ . . - A , ]~ ,

Q~+I = [A0 . - . A j _ 1 B o . . . B i n ] l ,w h e r e ~ + 1 i s f o r m e d b y ( i r a ) , A o = I , B o = A ~ , m >= 1 , s < j . A s -s u m i n g t h e l a t t e r , w e c o n c l u d e t h a t A j ~ ~ I A jw 2 B m ~ , a s a b o v e .

F r o m a s s u m p t i o n ( c ) a n d a s s u m p t i o n o f r e g u l a r i t y o f G , i t f o ll ow st h a t Cq c a n o n l y h a v e b e e n f o r m e d b y ( i ii b ) . C o n t r a d i c t i o n f o ll o w s a sa b o v e .

( I I ) S u p p o s e n = 1. P r o o f i s a n a l o g o u s .T h i s c o m p l e te s th e p r o of . F r o m L e m m a 1 2 i t fo ll ow s r ea d i ly b y t h es a m e k i n d o f r e a s o n i n g a s a b o v e t h a tC OR OLLA RY . U n d e r t h e a s s u m p t i o n s o f L e m m a 1 2,( A ) i f n = 2 , ~m 1+1 i s f o r m e d b y ( i r a ) o f t h e c o n s t r u c t i o n( B ) i f n = 1 , ~m 2 i s f o r m e d b y ( i i ib ) o f t h e c o n s t r u c t i o n( C ) Q ~ i s of t h e f o r m [ A1 . . - A k B o . . . B ~ ] t ( s >-_ O , B o = - I ) , f o r u s u c h

t h a t : ( a ) w h e r e n = 2 a n d m 0 i s a s i n ( A ) , L e m m a 12, t h e n m0 < u < m 2 ;( b ) w h e r e n = 1 a n d m ~ i s a s in ( B ) , L e m m a 12 , t h e n m l < u < m 3 .

F u r t h e r m o r e , fo r m l < u < m 2 , s > 0 i f t # n .D E F IN I T IO N 1 2. L e t D = ( ~ i , ' , ~ ) b e a d e r i v a t i o n i n G ' f o r m e d


27/31


b y t h e c o n s t r u c t i o n f r o m G . T h e n D ~ c o r r e s p o n d s t o D i f D ' i s a d e r i v a -t i o n o f z~22 i n G a n d f o r e a c h i , j , k ( i < j ) s u c h t h a t

( a ) ~ i is t h e e a r l ie s t l in e c o n t a i n i n g [ A I - . . A k ]l( b ) ~ j i s t h e l a t e s t l i n e c o n t a i n i n g [ A I . . . A k]2( c ) t h e r e is n o p , q s . t . i < p < j , q < h , a n d Q p = [A 1 . - . A q ] ~ ,

t h e r e is a s u b s e q u e n c e ( z i A ~ b , . . . , zj~ b) i n D ' .L E M M A 1 3 . L e t D = ( ~ 1 , ", ~ r ) b e a d e r i v a t i o n i n G t f o r m e d b y t h e

c o n s t r u c t i o n f r o m a r e g u l a r , n o n - s . e . G . S u p p o s e t h a t Q 1 = [A ~ . . A ~ ] I ,Q r = [A ~ - . . A ~ ] 2 , a n d t h e r e i s n o p , q s u c h t h a t 1 < p < r , q < s ,Q p = [ A 1 . . . A q ] ~ .

T h e n t h e r e is a d e r i v a t i o n D r = (~1 ~ , - , ~ , ) c o r r e s p o n d i n g t o D .P nO O F . P r o o f is b y i n d u c t i o n o n th e n u m b e r o f r e c u r r e n c e s o f s y m b o l sQ i i n D ( i.e ., t h e n u m b e r o f c y c l e s i n t h e d e r i v a t i o n ) .S u p p o s e t h a t t h e r e a r e n o r e c u r r e n c e s o f a n y Q~ i n D . I t f o l l o w s t h a t

t h e r e c a n h a v e b e e n n o a p p l i c a t i o n s o f ( i v a ) i n t h e c o n s t r u c t i o n of D ,i .e . , n o p a i r s Q i - - [A 1 - - . A j ] 2 , Q i + ~ = [A 1 . . . A ~ ]I w h e r e j < h . F o rs u p p o s e t h e r e w e r e s u c h a p a i r . . ' . A k _~ ~ A ~ A k . A lso , j > s , o r Q~ i sr e p e a t e d a s Q ~ . C l e a r l y t h e r e i s a n m > i ~- 1 s . t. Q ~ = [A ~ . . - A ~+ ~]~( n > 0 ) . . ' . t h e r e i s a t > m s .t . e i t h e r Q t = [A 1 . . . A j ]2 ( c o n t r a r y t oa s s u m p t i o n o f n o re p e t i t i o n s ) o r

Q t = [ A 1 . . . A s + ~ ] 2 , Q t + l = [ A 1 . . . A j - ,, ] I ( u , v > 1 ) ,w h e r e ~ + 1 is f o r m e d b y ( i i i b ) . . ' .

c o n t r a r y t o t h e a s s u m p t i o n t h a t G is n o n - s .e . S i m i la r ly , t h e r e c a n b e n oa p p l i c a t io n s o f ( i ii b ) i n th e c o n s t r u c t i o n o f D . B u t n o w t h e p r o o f f o rt h i s c a se fo l lo w s im m e d i a t e l y b y i n d u c t i o n o n t h e l e n g t h o f D .

S u p p o s e n o w t h a t t h e l e m m a i s t r u e f o r e v e r y d e r i v a t i o n c o n t a i n i n g< n o c c u r r e n c e s o f r e p e a t i n g Q ' s. S u p p o s e t h a t D c o n t a i n s n s u c h o c -c u r r e n c e s .

I .1 . S u p p o s e t h a t t h e s h o r t e s t r e c u r r i n g Q i n D i s [A ~ A k i n .2 . S e l e c t m l , m ~ s . t . m ~ < m ~ ; Q ~ = [A ~ . . . A ~ ] ~ = Q ~ : ; t h e r e i s

n o i , m~ < i < m ~ , s .~ . Q i = [ A 1 . . . A ~ ] ~ ; t h e r e i s n o j > m 2 s . t .Q j = [A 1 . . . A ~ ] ~ .

22 C o m p a r e C o n v e n t i o n 2 .


28/31

164 CHOMSKY3 . B y L e m m a 12 ( A ) , w e k n o w t h a t t h e r e i s a n m0 < m l s . t . Qm0 =

[A 1 - . . A ~ ] I . S e l e c t m0 a s t h e e a r l i e s t s u c h ( t h e r e i s i n f a c t o n l y o n e ) .B y t h e C o r o l l a r y t o L e m m a 1 2 , ( C ) , a n d t h e i n d u c t i v e h y p o t h e s i s , t h e r ei s a d e r i v a t i o n D ~ = ( z m o A k , . . . , z m ) 5 8 c o r r e s p o n d i n g t o ( ~ o , "" " , ~ 1 ) .

4 . B y C o r o l l a r y ( A ) , w e k n o w t h a t~ ml+ ~ = z , ~l [A o " ' " A k - i B o " ' " B m ] l ,

w h e r e A o = I , B o = A k , m >= 1 , B m - ~ - -~ A k B ~ . O b v i o u s l y , t h e r e i s av ( m ~ < v < m 2 ) s . t . e i t h e r Q ~ = [ A 0 - . - A ~ _ I B o . . . B m]2 o r

Qv = [ A0 . . . A ~ - I B o . . . B ~ + t ] 2 , Qv + ~ = [Ao . . - A k - l B o " ' " B m - u ] l ,w h e r e u , t > 1 a n d ~ + 1 i s f o r m e d b y ( i i i b ) [n o t e C o r o l l a r y (C ) ] . F r o mt h e l a t t e r a s s u m p t i o n w e c a n d e d u c e s e l f - e m b e d d i n g , a s a b o v e . . ' , w ec a n s e l e c t v a s t h e l a r g e s t i n t e g e r ( m 2 s . t . Q , = [A 0 A k _ ~ B o B ~ ] ~ .

5 . L e t t b e t h e l a r g e s t i n t e g e r ( m l ~ - 1 ~ t ~ v ) s . t .Q t = [ A 0 . . . A k _ ~ B o . . . B m _ u ] i , u > O .

S u p p o s e t h a t i = 1. B u t ~ t+1 m u s t b e f o r m e d b y ( i i a ) o r ( i l i a ) o f t h ec o n s t r u c t i o n . . ' , u = 1 a n d B m _~ - ~ B m C , c o n t r a r y t o a s s u m p t i o n o fr e g u l a r i t y , s i n c e B ~ _ ~ ~ A k B m .

. '. i = 2 , a n d Q t + l = [ A 0 . . . A k _ l B o . . . B ~ + . ] 1 ( n _-> 0 ) , w h e r e~ t + l i s f o r m e d b y ( i r a ) o f t h e c o n s t r u c t i o n . . ' .

B m + , - I - ~ B , ,_ ~ ,B ~ + ,~ ~ ~ B m - I ~ 2 B ~ , + ~ - -~ ~ I A k B m ~ 2 B m + , ~ .S u p p o s e n = 0 . T h e n B , , _~ --+ B m - ~ B m , s o t h a t , b y r e g u l a r i t y , B ~ _ ~ ----

A k . . ' . Q t = [A ~ A ~-]2, c o n t r a r y t o a s s u m p t i o n i n s t e p 2 ..'. n ~ O . .'. B m ~ ~ B , ~ + ~ _ ~ 4 . .'. B m ~ ~ 3 ~ A ~ B , ~ : B ~ + , ~ x 4 , c o n t r a .

( s . e . ) .6 . . ' . t h e r e i s n o t s u c h a s t h a t p o s t u l a t e d i n s t e p 5 . C o n s e q u e n t l y

( ~ + i , . , ~ ) m e e t s t h e a s s u m p t i o n o f t h e i n d u c t i v e h y p o t h e s i s e ~ a n dt h e r e i s a d e r i v a t i o n D ~ - - ( z ~ + ~ B m , . " ' , Z m ~ ) ~ c o r r e s p o n d i n g t o( ~ m 1 + 1 , " " " , ~ t~ v )"

7 . S i n c e v w a s s e l e c t e d i n s t e p 4 t o b e m a x i m a l , i t f o l lo w s t h a t ~ ,+ ~c a n n o t b e f o r m e d b y ( i v a ) , b y r e a s o n i n g s i m i l a r t o t h a t i n v o l v e d i n

~ Re c a l l th a t z ~ ,~0+~.m0z~ , i .e . , th e re is a de r iv a t io n (A ~, , z,~o+h~.~a F r o m n o n e x i s t e n c e o f s u c h a t i t f o l l o w s a t o n c e t h a t f o r u s u c h t h a t m tu < v, Q~ = [A0 .- - A~_~Bo . . . B,~Co . . . C~]i (~ >= O, Co ~ I ) .~ T h a t i s , t h e r e i s a d e r i v a t i o n (B ,~ , . - . , z ~ + ~ ).


29/31

ON CERTAIN FORMAL PROPERTIES OF GRAMMARS 1 6 5s t e p 4 . B y r e g u l a r i ty a s s u m p t i o n , i t c a n n o t h a v e b e e n f o r m e d b y ( l i b )o r ( i i i b ) o f t h e c o n s t r u c t i o n , s i n c e B , , - I ---+ A k B ~ . . ' .

Q ~ + I - - [ A 0 . . - A k - I B o . . . B , ~ - 1 1 2 .8 . S u p p o s e m = 1 , s o t h a t Q ~+ I = [ A I . . - A ~ ] 2 . . ' . v - t - 1 = m 2 , b y

a s s u m p t i o n o f s t e p 2 , a n d A Io ~ A k B ~ . L e t D 2 ' b e th e d e r i v a t i o n f o r m e df r o m D ~ ( c f . s t e p 6 ) b y d e l e t i n g i n i t i a l z~ 1+ ~ f r o m e a c h l i n e . L e t

( ~ 1 , " " , ~ ) = D I *D 2 '( c f . D e f i n i t i o n 1 1 ; D I a s i n s t e p 3 ) . C l e a r l y D 3 = (z ,~ oA l~ , ~ , . . . , ~ )i s a d e r i v a t i o n c o r r e s p o n d i n g t o ( ~ 0 , " " " , ~ ) .

9 . S u p p o s e m _>- 2 . B y a s s u m p t i o n t h a t G i s n o n - s . e . , a n d t h a t v i sm a x i m a l ( in s t e p 4 ) w e c a n s h o w t h a t e ~+ ~ m u s t b e f o r m e d b y ( i i b ) o ft h e c o n s t r u c t i o n ( a ll o t h e r c a s e s l e a d t o c o n t r a d i c t i o n ) . . ' .

Q~+2 = [A0 . . A k - l B o . . . B m - 2 C ] l , B ~ _ 2 --> B m _ l C .A s a b o v e , w e c a n f i n d a v l w h i c h i s t h e l a r g e s t i n t e g e r < m 2 s . t. Q ~ =[ A 0 . . . A k - ~ B o . . . B , ~ - 2 C ] 2 a n d s . t . ( , + ~ , . . . , ~ 1 ) m e e t s t h e i n d u c t i v eh y p o t h e s i s . . ' , t h e r e i s a d e r i v a t i o n D 4 = (z~+ 2C , . . . , z ~ ) c o r r e s p o n d i n gt o ( ~ + ~ , . , ~ ) .

1 0. S u p p o s e m = 2 . . ' .B,~_2---+ B1 C , v l + 1 = m 2 , Q ~ + I = l A x . . . A k ]~

( a s a b o v e ) . L e t D 4 ' b e t h e d e r i v a t i o n f o r m e d f r o m D 4 b y d e l e t i n g i n i t i a lz~ +2 f r o m e a c h l in e . L e t ( ~b ,, . . . , ~ p ) b e a s i n s t e p 8 . L e t ( x l , ", x q ) =( z ~ o B 1 , ~ 1 , " " " , ~ b ~ ) * D 4 ' . C l e a r l y D 5 = ( z m o A k , X l , " " , X q) i s a d e r i v a -t i o n c o r r e s p o n d i n g t o ( ~ 0 , - , ~ = ) .

1 1. S i m i l a r l y , w h a t e v e r m i s, w e c a n f in d a d e r i v a t i o nA = (Z ,~oAk , . . - , z ~ )

c o r r e s p o n d in g t o ( ~ 0 , . . . , e ~ ) .1 2. C o n s i d e r n o w t h e d e r i v a t i o n D 6 f o r m e d b y d e l e t in g f r o m t h e

o r i g in a l D t h e l in e s ~ + ~ , . , ~ a n d t h e m e d i a l s e g m e n t z ~ +~ f r o me a c h l a t e r l i n e . T h a t is , D 6 = ( ~ 1 , " " ", ~ t ) ( t = r - ( m ~ - m ~ ) ) , w h e r ef o r i < m ~ , ~b~ = ~ , a n d f o r i > m ~ , ~b~ -= z ~ z m _ ~ + ~ 2 _ ~ 1 + ~ . B yi n d u c t i v e h y p o t h e s i s , t h e r e i s a d e r i v a t i o n D ~ c o r r e s p o n d i n g t o D ~ .

I n s t e p s 2 a n d 3 , m 0 , m ~ , m~ w e r e c h o s e n s o t h a t ~ 0 c o n t a i n s t h ee a r l i e s t o c c u r r e n c e o f Q ~0 = [A ~ . . - A ~ ]~ , a n d ~ t h e l a t e s t o c c u r r e n c eo f Q ~ = Q m~ = [A ~ . - - A ~ .]2 , a n d s o t h a t n o o c c u r r e n c e s o f Q ~ o c c u r


30/31

166 CHOMSKYbetween ~1 and ~ . . ' . in Ds, Cmo contains the earliest occurrence ofQ,~o and /ml the latest occurrence of Q~I. Furthermore, by Corollary(C) , there is no Q shorter than Qm0 between ~m0 and ~1 . . ' . by induc-tive hypothesis and the definition of correspondence, it follows that D7contains a subsequenee D 7 = ( z~ oA k~ b, . . . , z m ~ ) . But step 11 guaranteesus a derivation A = (z ,~ oA k , . . . , z ~ ) corresponding to ( ~ 0 , "" ", ~ ) .We now construct Ds by replacing 1)7 in D7 by A = ( zm o A k ~ , . . . , z ~ ) ,formed by suffixing ~ to each line of 4, and inserting z~+~ after z~ inall lines of D7 following the subsequenee/)7.

Clearly/)8 corresponds to D, which is the required result in case theshortest recurring Q is of the form [...]~.II .An analogous proof can be given for the case in which the shortestrecurring Q is of the form [ .. .] ~.We have shown that the lemma holds for derivations with no recur-

sions, and that it holds of a derivation with n occurrences of recurringQ's on the assumption that it holds for all derivations with 2) such that Q1 -~ aQ2 and Q~ --~ Q3 --~ " . --+Q~, where Q~ -- [S]~, Qi is of the form [-. "]2 for 1 < i =< n, and Q1is of the form [...]1.

But in the grammar thus formed all rules are of the form A --~ a B(wh

Documents

Chomsky Article 2