Upload
chad-fitzgerald
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Overview of the merger prototype
Overview
• Backgrounds: The MUMIS project• Cross document annotation merging
• Alignment of parallel fragments• Unification of aligned fragments• Clean up unified fragments
• Reasoning• Evaluation• Future work & Conclusions
The MUMIS project
Semantic access to a multimedia database.
The MUMIS project
Semantic access to a multimedia database.
Subject: Soccer
The MUMIS project
Semantic access to a multimedia database.
Corpus: Video recordings of matches, formal texts, ‘ticker’ texts.
The MUMIS project
Semantic access to a multimedia database.
Approach: • Extract knowledge from textual sources• Align this (time based) knowledge with
video• Do retrieval on annotation, returning
corresponding video fragments to user
The MUMIS project
Semantic access to a multimedia database.
Main subject of this presentation: Merging the annotations resulting from separate texts into one cross-document annotation.
Merging
Intention of merging:- start with various texts- annotate each text individually- combine annotations
Example match:Netherlands – Yugoslavia(European Championship 2000)
Two types of text in merger:
• Formal texts• Ticker texts
Example formal text
Netherlands-YugoslaviaFinal score: 6-1Referee: Garcia Aranda
Goals: 24' Patrick Kluivert 90' Marc Overmars 91' Savo Milosevic
Substitutions: 53' out : Nisa Saveljic in : Jovan Stankovic 58' out : Patrick Kluivert in : Roy Makaay
Yellow Cards: Paul Bosvelt
Example ticker text (BBC)
19 mins: Bergkamp scuffs his left-foot shot but still forces Kralj into a diving save low down to his left.
20 mins: Edgar Davids wastes the best chance of the game so far when he blazes over with just the goalkeeper to beat after being put through by Bergkamp.
24 mins: Kluivert puts Holland in front after latching onto a wonderful chip from Bergkamp and then planting a right-foot shot past Kralj from eight yards.
25 mins: Boudewijn Zenden comes close to doubling Holland's lead when he fires in low, right-foot shot which Kralj just about hangs onto.
Example of parallel fragments
BBC - 15: Van der Sar pulls of great save to block Mijatovic's shot after
Savo Milosevic has cut through the Dutch defence like a knife.
Guardian - 17: Mijatovic, played in with a quick square ball from Milosevic,
finds himself one-on-one with van der Sar 10 yards out. He picks his spot, but unfortunately for Mijatovic, it's the spot occupied by van der Sar. A great save and Yugoslavia should be one-nil up.
Kickers 15: Milosevic auf Mijatovic, doch der Stuermer vom AC Florenz
scheitert aus 12 Metern freistehend an van der Sar.
WEBTEC 15: Milosevic filtreert door de Nederlandse defensie door één
beweging en legt af voor Mijatovic. Deze laatste trapt op van der Sar.
Example of parallel fragments
BBC - 15: Van der Sar pulls of great save to block Mijatovic's shot after
Savo Milosevic has cut through the Dutch defence like a knife.
Guardian - 17: Mijatovic, played in with a quick square ball from Milosevic,
finds himself one-on-one with van der Sar 10 yards out. He picks his spot, but unfortunately for Mijatovic, it's the spot occupied by van der Sar. A great save and Yugoslavia should be one-nil up.
Kickers 15: Milosevic auf Mijatovic, doch der Stuermer vom AC Florenz
scheitert aus 12 Metern freistehend an van der Sar.
WEBTEC 15: Milosevic filtreert door de Nederlandse defensie door één
beweging en legt af voor Mijatovic. Deze laatste trapt op van der Sar.
Merging process: overview
• 2 document alignment• N-document alignment• Unification of events from separate
sources• Special situations
Merging process:2-document alignment
Step 1 of the merging process: merge annotations of 2 texts
Merging process:2-document alignment
Source A Source B
Merging process:2-document alignment
The strongest binding is selected, ruling out certain other bindings.
Merging process:2-document alignment
The strongest binding is selected, ruling out certain other bindings.
Merging process:2-document alignment
The strongest binding is selected, ruling out certain other bindings.
Merging process:2-document alignment
The strongest binding is selected, ruling out certain other bindings.
Merging process:N-document
Given the 2-document alignments for each pair of sources, find the n-document alignment where all fragments describing same scene in all separate sources are aligned.
Merging process:N-document
Merging process:N-document
Merging process:N-document
Merging process:Unification of scenes
Merging and reasoning: types of rules
• Within events or scenes: Player1 and Player2 will not be the same person, a player performing a save will not score a goal in the same scene, etc.
• Role of teams and events: offensive vs. defensive
• Combinations of events that probably have the same player: ShotOnGoal+Goal, Penalty+HitThePost
• Terminology of authors may vary:Cross—Pass, Save—Clearance
Merging and reasoning:example rules
Merging and reasoning:example rules
Reasoning:mistakes in IE
Sometimes the information extraction component makes mistakes. Example rules have been applied to solve some of these.
Reasoning:mistakes in IE
Fix: The goal made by Kralj (Yugoslavian keeper) is removed
Evaluation:What do we want to know?
Quality of the merger in itself
The advantages and disadvantages of merging
Evaluation:Quality of the merger
• Quality of alignments• Quality of unification• The effect of the quality of the original
information extraction on both
Evaluation:Approach
• Create gold standard annotations for single sources
• Create gold standard merged annotation of all sources
• Run merger in different conditions• Compare everything with everything
Evaluation:Results
Alignments based on machine IE
Version 1
Version 2
Version 3
Manual 210 210 210
Automatic
104 187 189
Overlap 82 172 172
Precision 78.8 92.0 91.0
Recall 39.0 81.9 81.9
Evaluation:Results
Alignments based on manual IE
Version 1
Manual 210
Automatic
188
Overlap 174
Precision 92.6
Recall 82.9
Evaluation:Conclusions
• Quality of alignments is pretty good.
• Better IE improves alignments.• Low quality IE does not degrade
alignments too much.
MORE TO COME….
----- Extra Sheets -----
Extra example – 15th min.
Extra example – graph
Extra example – unification
Extra example – the source
BBC - 15: Van der Sar pulls of … Milosevic has cut …
Guardian - 17: Mijatovic, played in with a quick square ball from Milosevic,
finds himself one-on-one with van der Sar 10 yards out. He picks his spot, but unfortunately for Mijatovic, it's the spot occupied by van der Sar. A great save and Yugoslavia should be one-nil up.
Kickers 15: Milosevic auf Mijatovic, doch der Stuermer vom AC Florenz
scheitert aus 12 Metern freistehend an van der Sar.
WEBTEC 15: Milosevic filtreert door de Nederlandse defensie door één
beweging en legt af voor Mijatovic. Deze laatste trapt op van der Sar.
Pass MilosevicShotOnGoal MijatovicSave Van der Sar
Reasoning: incomplete graphs
Reasoning: incomplete graphs
Reasoning: incomplete graphs
Reasoning: incomplete graphs
Reasoning: incomplete graphs
Reasoning: incomplete graphs
Reasoning: incomplete graphs
Reasoning: incomplete graphs
Reasoning:own goal
Reordering
Observation from corpus:• Scenes in correct order• Events within scenes often in wrong
order
Reordering
Manual annotation of several matches
Pass, Shot-on-goal, GoalPass, Shot-on-goal, SaveShot-on-goal, Hitting-the-postFoul, Free-kick, Shot-on-goal, Corner
Reordering
Reordering
Not fully implemented yet in the merger.