NLP (Fall 2013): Recursive & Augmented Transition Networks

Natural Language Processing

Pushdown Automata

&

Recursive Transition Networks

Vladimir Kulyukin

vkedco.blogspot.com

Outline

Pushdown Automata (aka Stack Machines)


Pushdown Automata (PDA)

(aka Stack Machines)

Finite

Control

Input Tape

Stack

PDA

read pop push

a c abc

If the current input symbol is a and the symbol on top of the stack is

c, pop c off the stack, push abc onto the stack, and advance to the

next symbol (move right). Reading a symbol means seeing it in the

input and advancing to the next symbol.

Stack Machine Move

read pop push

a c b

a c

d

Input Tape Stack

Stack Machine Move Example 01

a c

d

Read a & Move

one cell right

Input Tape Stack

read pop push

a c b

Stack Machine Move Example 01: READ

read pop push

a c b

a

d

POP c OFF Sack

Input Tape Stack

Stack Machine Move Example 01: POP

a b

d

PUSH b ONTO Stack

Input Tape Stack

read pop push

a c b

Stack Machine Move Example 01: PUSH

Pushdown Automata

&

Non-Determinism

A stack machine is like NFA, because it

may have multiple sequences of moves on

the same input

A stack machine accepts iff the input has

been read and the stack is empty

Stack Machine Non-Deterministic Acceptance

read pop push

ε S ab

a S ef

a S ε

Stack Machine Example 02

Suppose the input is a

The initial stack contains S

We have three possible move sequences:

Sequence 1: no input is read, pop S, push ab

Sequence 2: read a, pop S, push ef

Sequence 3: read a, pop S, push ε

Stack Machine Example 02

Stack Machine Construction

Construct a stack machine for {anbn}

Problem

If a is read, pop off S, push 1, and push S

In the middle of the string, pop off S

If b is read, pop 1 off the stack

If the stack is empty, that means we have read

some number of a’s (pushing) followed by the same

number of b’s (popping)

Basic Insight

read pop push

a S S1

ε S ε

b 1 ε

Stack Machine Table

a S a a b b b

Read a; Pop S; Push 1; Push S; Move Right

Processing Sample Input

a S

1 a a b b b



a

S

1

1 a a b b b



a

S

1

1

1 a a b b b

1. PDA has reached the break point so

it needs to start popping

2. Read ε; Pop S; Push ε; Stay on b


a

1

1

1 a a b b b

Read b; Pop 1; Push ε; Move Right


a

1

1 a a b b b



a

1 a a b b b



a a a b b b

End of Input Tape is Reached;

Stack is Empty;

Input is Accepted by Empty Stack


(x, y) where x ϵ Σ* is the unread part of the input

and y ϵ Γ* is the contents of the stack, where the leftmost character of y is on top of the stack.

Instantaneous Description

→ is a relation on instantaneous descriptions

such that

1. (ax, Bz) → (x, yz) iff y is in δ(a, B)

2. (x, Bz) → (x, yz) iff y is in δ(ε, B)

One-Step Derivation

→* is a relation on instantaneous descriptions

such that if A & B are two instantaneous

descriptions, then A →* B iff there is sequence

of zero or more → transitions that start at A

and end at B

Multi-Step Derivation

L(M) = {x is in Σ* | (x, S) →* (ε, ε)}

Acceptance by Empty Stack

Conversion of FA to PDA

We can construct an FA for a regular language

We can construct a regular grammar from an NFA

Regular grammars are context-free grammars

It is no surprise that we can construct stack machines

from finite automata

FAs to SMs

q0 q1

q2 q3

0

1

0

1

1

1

0

0

FAs to SMs

3

2

33

23

12

02

31

21

10

00

,

,

,1

,0

,1

,0

,1

,0

,1

,0

q

q

qq

qq

qq

qq

qq

qq

qq

qq read pop push

0 q0 q0

1 q0 q1

0 q1 q2

1 q1 q3

0 q2 q0

1 q2 q1

0 q3 q2

1 q3 q3

ε q2 ε

ε q3 ε

FAs to SMs

Conversion of CFGs to PDA

., ,a allfor .2

;|v, , allfor 1.

:follows as defined

is where,,, :Sketch Proof

.such that machinestack

a is thereCFG, a is ,,, If :Theorem

aa

PxvxVv

S,VM

MLGLM

PSV G

Conversion Theorem

Let us convert this CFG to SM:

S → aSa

S → bSb

S → ε

Example

read pop push

ε S aSa

ε S bSb

ε S ε

a a ε

b b ε

Example



A Recursive Transition Network (RTN) is a

directed graph with labeled states and arcs

Arc labels are state names (variables) and

terminal symbols

There is a unique state called start state and

a set of states called final states

RTN Arc Execution

The arc execution is as follows: suppose there is an arc

from A to B labeled L

If the arc is followed, then B is pushed onto the

pushdown stack and control jumps to state L without

advancing the input tape pointer if L is non-terminal or

with advancing the input tape pointer if L is terminal

When a final state of some network is reached, the

pushdown stack containing the name of that network may

be popped

When the stack is empty and the end of the input is

reached, the RTN accepts the input

RTN Example

Recursive Transition Network: Subnetwork S

S

Q1

Q2

Q3 Q4/1 Q5/1

NP

AUX

V

AUX

V NP

PP

1. S is start state

2. Q4/1 and Q5/1 are final states

NP

Categories & Terminals

DET a | the

N robot | bot | room

V work | clean | run

ADJ small | large

AUX can | does

PREP in | at

NPR RG | R2D2 | BOT_103


DET a | the



ADJ small | large

AUX can | does

PREP in | at



DET a | the



ADJ small | large

AUX can | does

PREP in | at


Recursive Transition Network: Subnetwork NP

NP Q7/1

DET N (Noun)

PP

Q6

Q8/1

NPR (Proper Name)

ADJ

1. NP is start state

2. Q7/1 and Q8/1 are

final states

Recursive Transition Network: Subnetwork PP

PP Q10/1

Prep NP

Q9

1. PP is start state

2. Q10/1 is final state


DET a | the



ADJ small | large

AUX can | does

PREP in | at


RTN Input Processing

Suppose input is “|can the small bot run” (|

is input pointer)

RTN starts in state S

RTN non-deterministically decides to execute

the AUX arc, because the category of “can” is

AUX

So, “can” is scanned, input pointer is

advanced to “the” and RTN goes to state Q2


Current input is “can |the small bot run”

In state Q2, RTN can execute only the NP arc

So, Q3 is pushed onto the stack (to remember

which state it needs to come back to), input

pointer is not advanced, and RTN goes to state

NP


Current input is “can |the small bot run”

In state NP, RTN non-deterministically

chooses to execute the DET arc, because the

category of “the” is DET

So, “the” is scanned, input pointer is

advanced to “small” and RTN goes to state Q6


Current input is “can the |small bot run”

In state Q6, RTN non-deterministically

chooses to execute the ADJ arc, because the

category of “small” is ADJ

So, “small” is scanned, input pointer is

advanced to “bot” and RTN goes back to state

Q6


Current input is “can the small |bot run”

In state Q6, RTN non-deterministically

chooses to execute the N arc, because the

category of “bot” is N

So, “bot” is scanned, input pointer is

advanced to “run” and RTN reaches a final

state Q7/1

RTN goes back to the stack, pops Q3 off the

stack, and goes to Q3


Current input is “can the small bot |run”

In state Q3, RTN executes the V arc, because

the category of “run” is V

So, “run” is scanned, input pointer is

advanced to the end of the input, and RTN

reaches a final state Q4/1


Current input is “can the small bot run|”

In state Q4/1, RTN accepts the input,

because

1. the stack is empty (cannot be popped);

2. the end of the input is reached;

3. Q4/1 is a final state

References & Reading Suggestions

Woods, W. A. (1970). Transition Network Grammars

for Natural Language Analysis. Communication of the

ACM, Vol 13, Number 10, October 1970

Moll, Arbib, and Kfoury. An Introduction to Formal

Language Theory

Technology

NLP (Fall 2013): Recursive & Augmented Transition Networks