Upload
lindsay-mitchell
View
216
Download
0
Embed Size (px)
DESCRIPTION
SALSA-WS 09/05 Frame Semantics (Fillmore 1976, Fillmore et. al. 2003) Lexical semantic classification of predicates and their argument structure A frame represents a prototypical situation (e.g. Commercial_transaction, Theft, Awareness) A set of roles identifies the participants or propositions involved Frames are organized in a hierarchy Berkeley FrameNet Project db: 600 frames, lexical units, annotated sentences
Citation preview
SALSA-WS 09/05
Approximating Textual Entailment with LFG and
FrameNet FramesAljoscha Burchardt, Anette Frank
Computational Linguistics DepartmentSaarland University, Saarbrücken
Second Pascal Challenge WorkshopVenice, April 2006
SALSA-WS 09/05
Outline of this Talk• Frame Semantics• A baseline system for approximating
Textual Entailment– LFG syntactical analyses with– Frame semantics– Statistical decision: entailed?
• Walk-through example from RTE 2006• RTE 2006 results / brief conclusions
SALSA-WS 09/05
Frame Semantics (Fillmore 1976, Fillmore et. al. 2003)
• Lexical semantic classification of predicates and their argument structure
• A frame represents a prototypical situation (e.g. Commercial_transaction, Theft, Awareness)
• A set of roles identifies the participants or propositions involved
• Frames are organized in a hierarchy• Berkeley FrameNet Project db: 600 frames,
9.000 lexical units, 135.000 annotated sentences
SALSA-WS 09/05
Seller BMW bought Rover from British Aerospace.
Buyer Rover was bought by BMW, which financed [...] the new Range Rover.
Goods BMW, which acquired Rover in 1994, is now dismantling the company.
Money BMW‘s purchase of Rover for $1.2 billion was a good move.
Linguistic Normalizations(Frame: Commerce_buy)
Voice: active / passive
POS: verb / noun
Lexicalization
SALSA-WS 09/05
Frame Semantics for RTEFocusing on lexical semantic classes and
role-based argument structure– Built-in normalizations help to determine
semantic similarity at a high level of abstraction
– Disregarding aspects of “deep“ semantics: negation, modality, quantification, ...
– Open for deeper modeling on demand (e.g. our treatment of modality)
SALSA-WS 09/05
A Baseline System for Approximating Textual Entailment
• Fine-grained LFG-based syntactic analysis – English LFG grammar (Riezler et al. 2002)– Wide-coverage with high-quality probabilistic
disambiguation• Frame Semantics
– Shallow lexical-semantic classification of predicate-argument structure
– Extensions: WordNet senses, SUMO concepts• Computing structural and semantic overlap of t
and h– Hypothesis: large overlap ≈ entailment
text hypothesis
Statistical Decision: Entailment?
ComputingSemanticOverlap
Linguistic AnalyseshypothesisLFG f-structure graph w/ frames & concepts
text LFG f-structure graph w/ frames & concepts
text-hypothesis match graphdifferent types of matches (aspects of similarity)
Feature extractionlexical, syntactic, semantic structure & overlap measures
Model training & classification
A Baseline System for Approximating Textual Entailment
SALSA-WS 09/05
Rule-based: extend & refine sem. proj.• NEs, Locations• Co-reference • Modality, etc.
Linguistic ComponentsXLE parsing: LFG f-structure
F-structure w/ semantics projection
WordNet-based WSD: WordNet &
SUMO
Fred / Detour / Rosy:
frames & roles
Using XLE term rewriting system (Crouch 2005)
SALSA-WS 09/05
Example from RTE 2006Pair 716
Text In 1983, Aki Kaurismäki directed his first full-time feature.
Hypothesis Aki Kaurismäki directed a film.
LFG F-Structures
SALSA-WS 09/05
Automatic Frame Annotation for Text (SALTO Viewer)
Fred & Rosyframes & roles
(statistical)
Detour Systemframes
(via WordNet)
Collins Parse
SALSA-WS 09/05
Automatic Frame Annotation for Hypothesis
716_h: Aki Karusmäki directed a film.
SALSA-WS 09/05
LFG + Frames for Hypothesis(FEFViewer)
Aki Kaurismäki directed a film.
Rule-based(LFG-NER)
SALSA-WS 09/05
Hypothesis-Text-Match Graphs Computing Structural and Semantic overlap
Match graph bundles overlapping partial graphs marked by match types
• Aspects of similarity– Syntax-based (i.e. lexical and structural): Identical
predicates (attributes) trigger node (edge) matches.– Semantics-based: Identical frames/concepts (roles)
trigger node (edge) matches.• Degrees of similarity
– Strict matching– Weak matching conditions for non-identical predicates:
• “Structurally related” e.g. via coreference (relative clauses, appositives, pronominals)
• “Semantically related” via WordNet, Frame-Relations
h: Aki Kaurismäki directed a film.
WordNetrelated
t: In 1983, Aki Kaurismäki directed his first full-time feature.
Grammaticallyrelated
Statistical Modeling• Feature extraction on the basis of
– Syntactic, Semantic matches (of different types)– Matching clusters’ sizes– Ratio (matched vs. hypothesis)– (Non-)matching modality– RTE-task, fragmentary (parse),…
• Training/classification with WEKA tool– Feature selection
1. Predicate Matches2. Frame overlap3. Matching cluster size
– Model 1: Conjunctive rule (Feat. 1,2)– Model 2: LogitBoost (Feat. 1,2,3)
RTE 2006 Resultsall tasks IE IR QA SUM
Model 1 59.0 49.5 59.5 54.5 72.5Model 2 57.8 48.5 58.5 57.0 67.0
• SUM (and IR) are natural tasks for Frame Semantics, IE and QA need more deeper modeling (aboutness vs. factivity)
• Error analysis– True positives: high semantic overlap– True negatives: 27% involve modality mismatches– False examples: poor modeling of dissimalrity
• Many high-frequency features measuring similarity• Few low-frequency features measuring dissimilarity
SALSA-WS 09/05
Brief Conclusions• Good approximation of semantic similarity
– Deep LFG syntactical analyses integrated with– Shallow lexical Frame Semantics (plus other lex.
resources)– Match graph measuring overlap
• Need better model for semantic dissimilarity– Too few rejections (false positives >> false negatives)
• Towards deeper modeling– Treatment of modal contexts– Integration of lexical inferences– Open for collaborations
stmt_type(f(0),declarative).tense(f(0),past).pred(f(0),direct).mood(f(0),indicative).dsubj(f(0),f(7)).dobj(f(0),f(2)).pred(f(2),film).num(f(2),sg).det_type(f(2),indef).proper(f(7),name).pred(f(7),'Kaurismaki').num(f(7),sg).mod(f(7),f(10)).proper(f(10),name).pred(f(10),'Aki').num(f(10),sg).sslink(f(0),s(41)).sslink(f(2),s(42)).sslink(f(7),s(45)).sslink(f(10),s(59)).
frame(s(41),'Behind_the_scenes').artist(s(41),s(45)).production(s(41),s(42)).frame(s(42),'Behind_the_scenes').frame(s(45),'People').person(s(45),s(59)).person(s(45),s(45)).
ont(s(41),s(48)).ont(s(42),s(49)).ont(s(45),s(56)).wn_syn(s(48),'direct#v#11').sumo_sub(s(48),'Steering').milo_sub(s(48),'Steering').wn_syn(s(49),'film#n#1').sumo_sub(s(49),'MotionPicture').milo_sub(s(49),'MotionPicture').sumo_syn(s(56),'Human').sumo_syn(s(58),'Human').
LFG + Frames for Hypothesis (FEF)