Semantic Role Chunking Combining Complementary Syntactic Views

Sameer Pradhan, Kadri Hacioglu, Wayne Ward, James H. Martin, Daniel Jurafsky

Center for Spoken Language Research

Department of Computer ScienceUniversity of Colorado at Boulder

Department of LinguisticsStanford University

Different Syntactic Views

Hypothesis: Different views make different errors

Two views: Phrase structure based (Charniak, Collins) Chunk based

Constituents from Charniak parse tree

Charniak Parse Tree

Constituent Views

John kicked the ball .

Collins Parse Tree

Constituents from Collins parse tree

Chunk View

Salomonwillbuysufficientsharesto coveritsentireposition

OOOB-A2I-A2OB-VB-A1I-A1I-A1

Chunk using an IOB representation [Ramshaw & Marcus, 1995]

Yamcha [Kudo & Matsumoto, 2001]

Bottom up as opposed to top down

Flat representation Uses flat syntactic chunks

[Hacioglu & Ward 2003]

Algorithm

Generate Charniak and Collins parse based features Add few features from one to the other Generate semantic IOB tags using these views Use them as features Generate the final semantic role label set using a phrase-

based chunking paradigm

Architecture

Chunker

Charniak Collins Words

Features

IOB

Semantic Role Labels

IOBIOB IOB

Phrases

Illustration

Train Model

1 2 RB

B

B B B

B B

I

I

I

I

I

I

I

I

IIII

I

I

II

I

O

OO

O

O

O

O

O

O

O

O

O

O

O

B B

O O

Classifier

Model

1 2 HB

B B B

B

B

B

B

B

I

I

I

I

I

I

I

I

I

I

IIII

I

II

I

II

I

O

O

OOO

O

OOO

OO

OOO

OO

OO

Features

Semantic IOB tags for Charniak and Collins based semantic role labels [Pradhan et al., 2005]

Phrase level chunk features [Hacioglu et al., 2004]

Active Learning

Randomly selelected 10k examples and trained a NULL vs ARGUMENT classifier

Classified remaining examples using this classifier Added misclassified examples to the seed set Iterated Final data amounted to about a third of the total

Combination Results

Test : Section 24 of PropBankTrain : Sections 02-21 of PropBank

ID + Class

ASSERTCharniak

System P R F1

80 75 77ASSERTCollins 79 74 76ASSERTCombined

81 76 78

Results

Section 24

Submitted System

System P R F1

80.9 75.4 78.0

Section 23

P R F1

81.9 73.3 77.4

Brown

P R F1

73.7 61.5 67.1Bug fixed System 81.9 75.1

78.382.9 74.7

78.674.5 63.3

68.4

ID + Class

Thank You

Arda AQUAINT program contract OCG4423B

NSF grant IS-9978025

Software

ASSERT (Automatic Statistical SEmantic Role Tagger) Publicly downloadable at http://oak.colorado.edu/assert Downloaded by more than 50 research groups

Null Filtering

Removed constituents with P(NULL) > 0.9 Removed phrases with P(NULL) > 0.8 after incorporating

context

Analysis

Active learning using confidence threshold Constituent level instead of Sentence level N-Best Charniak parses

Features (Constituent)

Features (Constituent)

Features (Phrase)

Features (Phrase)

Representation

Features

Features

But analysts reckon underlying support for sterling has been eroded by the chancellor 's failure to announce any new policy measures in his Mansion House speech last Thursday

Minipar-based Semantic Labeling

Rule-based dependency parser