LING 388: Language and Computers

Sandiway Fong

Lecture 21: 11/7

2

Administrivia

• homework 6– grammar programming – due Thursday

• lab class Thursday– Social Sciences 224

3

Review

• exploring questions of formal equivalence

Regular Grammars

FSA Regular ExpressionsNDFSA

FSA with-transitions

x --> y, [t].x --> [t]. (left recursive)orx --> [t], y.x --> [t]. (right recursive)

4

Review

• exploring questions of formal equivalence

Regular Grammars

(ND)FSA

equivalent (right-recursive) regular grammars --> [b], x.x --> [a], y.y --> [!].y --> [a], y.

s x

z

b

!

ya

a

>

Prolog FSAs([b|L]) :- x(L).

x([a|L]) :- y(L).

y([a|L]) :- y(L).

y([‘!’|L]) :- z(L).

z([]).

queries?- s(L).?- s(L,[]).equivalent all theway down to whenthey outputanswers and loop

5

Review

• exploring questions of formal equivalence

Regular Grammars

s

b x

a y

a y

!

s

!x

ay

ay

b

•right recursive regular grammars(s(b,X)) --> [b], x(X).

x(x(a,Y)) --> [a], y(Y).

y(y(!)) --> [!].

y(y(a,Y)) --> [a], y(Y).

•left recursive regular grammars(s(X,!)) --> x(X),[!].

x(x(Y,a)) --> y(Y),[a].

y(y(b)) --> [b].

y(y(Y,a)) --> y(Y),[a].

•same query?- s(P,[b,a,a,!],[]).

X = s(b,x(a,y(a,y(!))))

X = s(x(y(y(b),a),a),!)

then semicolon ;(more answers)

terminates

infinite loop

6

Last Time

• exploring questions of formal equivalence

Regular Grammars

the determiner-noun system–the man/men–a man/*a men

•DCGnp(np(Y)) --> pronoun(Y).np(np(D,N)) --> det(D,Number), common_noun(N,Number).det(det(the),sg) --> [the].det(det(the),pl) --> [the].det(det(a),sg) --> [a].common_noun(n(ball),sg) --> [ball].common_noun(n(man),sg) --> [man].common_noun(n(men),pl) --> [men].pronoun(i) --> [i].pronoun(we) --> [we].

DCG

•regular grammar equivalentnp(np(i)) --> [i].np(np(we)) --> [we]. np(np(the,N)) --> [the], common_nounsg(N).np(np(the,N)) --> [the], common_nounpl(N).np(np(a,N)) --> [a], common_nounsg(N).common_nounsg(n(ball)) --> [ball].common_nounsg(n(man)) --> [man].common_nounpl(n(men)) --> [men].

7

Determiner-Noun Agreement

• by extension– this determiner-noun

system also has a FSA encoding

• determiner-noun regular grammar • np(np(i)) --> [i].• np(np(we)) --> [we]. • np(np(the,N)) --> [the],

common_nounsg(N).• np(np(the,N)) --> [the],

common_nounpl(N).• np(np(a,N)) --> [a], common_nounsg(N).• common_nounsg(n(ball)) --> [ball].• common_nounsg(n(man)) --> [man].• common_nounpl(n(men)) --> [men].

np

icommon_nounsg

>

we

the

a

ball man

common_nounpl

the

men

note: machine is aNDFSA.can be transformed into a deterministic machine

8

Determiner-Noun Agreement

• NDFSA FSA➟– set-of-states

construction

• determiner-noun regular grammar • np(np(i)) --> [i].• np(np(we)) --> [we]. • np(np(the,N)) --> [the],

common_nounsg(N).• np(np(the,N)) --> [the],

common_nounpl(N).• np(np(a,N)) --> [a], common_nounsg(N).• common_nounsg(n(ball)) --> [ball].• common_nounsg(n(man)) --> [man].• common_nounpl(n(men)) --> [men].

np

icommon_nounsg

>

we

the

a

ball man

common_nounpl

the

men

np

icommon_nounsg

>

we

a

ball man

{common_nounpl,common_nounsg}

the

menman

ball

9

Determiner-Noun Agreement

• equivalence instantiated

Regular Grammars

FSA Regular ExpressionsNDFSA

FSA with-transitions

np

icommon_nounsg

>

we

a

ball man

{common_nounpl,common_nounsg}

the

menman ball

np

icommon_nounsg

>

we

the

a

ball man

common_nounpl

the

men

np(np(i)) --> [i].np(np(we)) --> [we]. np(np(the,N)) --> [the], common_nounsg(N).np(np(the,N)) --> [the], common_nounpl(N).np(np(a,N)) --> [a], common_nounsg(N).common_nounsg(n(ball)) --> [ball].common_nounsg(n(man)) --> [man].common_nounpl(n(men)) --> [men].

we|i|(a (ball|man))|(the (ball|man|men))

10

Limits of Finite State Technology

• In general, it’s not true that grammars can always be rewritten into regular grammar form (and therefore simulatable by FSA)

11

Limits of Finite State Technology

• language = set of strings• case 1

– suppose set is finite– e.g. L = {ba, abc, ccb, dd}

• easy to encode as a FSA

(by closure under union)

• case 2– set is infinite– ...

s1 s2 s3ab

s1 s2 s3ba s4

c

s1 s2 s3cc s4

b

s1 s2 s3dd

s0

ε

ε

ε

ε

12

Limits of Finite State Technology

• Language = set of strings• case 2

– set is infinite– e.g. L = a+b+ – = { ab, aab, abb, aabb, aaab, abbb, … }

• “one or more a’s followed by one or more b’s”• we know this set is regular

– however, consider – L = {anbn | n ≥ 1} – = { ab, aabb, aaabbb, aaaabbbb, …}

• “the same number of b’s as a’s…”• this set is not regular. • Why? answer lies in the nature of the recursion

s x

y

aa

b

b

13

The Limits of Finite State Technology

• [Aside: we can use the Pumping Lemma to prove this particular case.]

• informally, – we can build FSA for…– ab– aabb– aaabbb– …

a b

a a b b

a a a b b b

= end state

14

The Limits of Finite State Technology

• we can merge the individual FSA for…– ab– aabb– aaabbb

a a a b b bb

b

b

• such direct encoding would require an infinite number of states– and we’re using Finite State Automata

• quite different from the infinity obtained by looping– freely iterate (no counting)

15

The Limits of Finite State Technology

• example– L = a+b+ = { ab, abb, aab,

aabb, aaab, abbb, … }– “one or more a’s followed

by one or more b’s”

• Note:– can be divided into two

independent halves– each half can be replaced

by iteration

s1 s2 s3ba

s1 s2 s3aa s4

b

s1 s2 s3ba s4

b

s1 s2 s3aa s4

bs5

b

s1 s2 s3aa s4

as5

b

s1 s2 s3ba s4

bs5

b

16

The Limits of Finite State Technology

• example– L = a+b+ = { ab, abb, aab,

aabb, aaab, abbb, … }– “one or more a’s followed

by one or more b’s”

• Note:– can be divided into two

independent halves– each half can be replaced

by iteration

s1 s2 s3ba

s1 s2 s3aa s4

b

s1 s2 s3ba s4

b

s1 s2 s3aa s4

bs5

b

s1 s2 s3aa s4

as5

b

s1 s2 s3ba s4

bs5

b

s1 s2 s3ba s4

b

s1 s2 s3aa s4

bs5

b

s0

εε

s1 s2 s3aa s4

as5

b s6b

s0

εε

s1 s2 s3aa s4

as5

b s6b b s7

s1 s2 s3aa s4

as5

b bs3 s4a

s5b ba

17

Distance and Finite State Limits

• Does linear distance between dependent elements create machines with impractically large number of states?

• Subject-Verb number agreement– The men were ...

– The man was ...

– The men forced from their homes were ...

– The man forced from his home was ...

• (Miller and Chomsky 1963)– The people who called and wanted to rent your house when you go

away next year are from California

– need “memory”

– 14 words separating people from are

18

Distance and Finite State Limits

• (Miller and Chomsky 1963)– The people/person who called and wanted to rent your house when you go

away next year are/is from California– need “memory”– 14 words separating people from are

from (Coleman 2005)

19

Next Time

• Back to writing grammars...– building sentence meaning– next step towards our translator