Upload
emory-day
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Informatics and Telematics InstituteCentre for Research and Technology Hellas
ITI-CERTH
Amsterdam, Multimedia Semantics XG, 10-11 July 2006
Vasileios Papastathis, Research Fellow
Centre for Research and Technology Hellas (CERTH)
Informatics and Telematics Institute (ITI) (http://www.iti.gr)
Director Prof. Michael Strintzis
Multimedia Knowledge Group (http://mkg.iti.gr)Head Dr. Yiannis Kompatsiaris
Thessaloniki, GREECE
Informatics and Telematics InstituteCentre for Research and Technology Hellas
2
Presentation Overview
• ITI Multimedia Knowledge Group• Towards a Common Multimedia
Ontology Framework• IST BOEMIE Project
Informatics and Telematics InstituteCentre for Research and Technology Hellas
3
ITI Multimedia Knowledge Group
• Semantic Multimedia Analysis• Multimedia Indexing and Retrieval• Multimedia and the Semantic Web• Knowledge Structures, Languages and Tools for
Multimedia• Reasoning and Personalization for Multimedia Applications
• Ontology structures in OWL for knowledge-assisted multimedia analysis and reasoning
• Rules in SWRL including multimedia features for analysis and reasoning
• MPEG-7 Visual part representation in RDF• Graphical tool for ontology-based annotation of images
(M-Ontomat)
Informatics and Telematics InstituteCentre for Research and Technology Hellas
4
R&D Projects
• “aceMedia: Integrating knowledge, semantics and content for user centred intelligent media services”, IP 2004-2007
• “KnowledgeWeb: Realizing the Semantic Web”, funded by the DG XIII, NoE 2004-2007
• “MESH: Multimedia Semantic Syndication for Enhanced News Services”, IST – IP, 2006-2008
• “K-Space: Knowledge Space of Semantic Inference for Automatic Annotation and Retrieval of Multimedia Content, IST - NoE, 2006-2009
• “X-Media: Knowledge Sharing and Reuse Across Media”, IST – IP, 2006-2009
• “BOEMIE: Bootstrapping Ontology Evolution with Multimedia Information Extraction”, IST – STREP, 2006-2008
• “PATExpert: Advanced Patent Document Processing Techniques”, IST-STREP, 2006-2008 (collaboration with AUTH)
Informatics and Telematics InstituteCentre for Research and Technology Hellas
Towards a common Multimedia Ontology Framework
Requirements and Harmonization approaches
Informatics and Telematics InstituteCentre for Research and Technology Hellas
6
Multimedia Ontology Framework
• Framework for collecting existing multimedia knowledge representation
• Define approach for harmonization• Initiated by aceMedia in EWIMT 2005
but not directly connected to aceMedia work
• http://www.acemedia.org/aceMedia/reference/multimedia_ontology/index.html
Informatics and Telematics InstituteCentre for Research and Technology Hellas
7
List of contributors
1. National Technical University of Athens2. Istituto per la Matematica Applicata e le Tecnologie
Informatiche, Consiglio Nazionale delle Ricerche, AIM@SHAPE
3. University Rennes4. Centre for Digital Music, Queen Mary, University of London5. Lehrgebiet Multimedia und Internetanwendungen,
Informatikzentrum6. University of Southampton, UK7. Institute of Informatics & Telecommunications, N.C.S.R.
"Demokritos"8. Institute of Information Systems & Information
Management, JOANNEUM RESEARCH Forschungsgesellschaft mbH
Informatics and Telematics InstituteCentre for Research and Technology Hellas
8
List of contributors (2)
9. ISWeb, University of Koblenz
10. Freie Universitaet,Netzbasierte Informationssysteme, Institut fuer Informatik
11. 'Signals and Images' Laboratory, Institute of Information Science and Technologies "A. Faedo" (ISTI–CNR), Italian National Research Council (CNR)
12. Image Bioinformatics Research Group, Department of Zoology, University of Oxford
13. Technical University of Crete,
14. CWI Amsterdam, Maryland Information and Network Dynamics Lab, National Technical University of Athens
15. University of Twente
Informatics and Telematics InstituteCentre for Research and Technology Hellas
9
Requirements for MM ontologies
• Scope and Usage• Media Description-Description of the
information object• Representation of Content Structure• Representation of Content Description• Knowledge Representation Formalism• Reasoning Support • Following an established methodology
Informatics and Telematics InstituteCentre for Research and Technology Hellas
10
Requirements (2)
• Scope and Usage Annotation Analysis Retrieval Reasoning Personalized
filtering Meta-Modeling
• Media description Image Video 3D Graphics Audio AudioVisual Multimedia
presentation
Informatics and Telematics InstituteCentre for Research and Technology Hellas
11
Requirements (3)
• Representation of content structure Multimedia
structure representation
Spatiotemporal relation representation
Localization
• Representation of content description Low-level description Multiple audiovisual
descriptors Alignment to existing
metadata standards Basic and complex
data types
Informatics and Telematics InstituteCentre for Research and Technology Hellas
12
Requirements (4)
• Knowledge representation formalism High complexity of
multimedia modeling Compliance to W3C
standards Inferencing support OWL
• Reasoning support DL-based
reasoning Rules Uncertainty Fuzziness
• Established methodology OntoKnowledge OntoClean
Informatics and Telematics InstituteCentre for Research and Technology Hellas
13
Harmonization approaches (1)
• Modularization Clear, independent ontology modules Examples: Visual Descriptors Ontology (VDO),
Multimedia Structure Ontology (MSO), Spatio-Temporal Ontology (STO)
• Ontology cross-domain linking Alignment techniques for multimedia ontologies linking
with domain-specific ontologies Merging techniques for multimedia ontology
coordination
• Compliance to standards Semantic Web, W3C, MPEG-7, etc
Informatics and Telematics InstituteCentre for Research and Technology Hellas
14
Harmonization approaches (2)
• Harmonization of core ontologies Core ontologies cannot be simply mapped but
need to be “harmonized” Need to apply ontological modeling principles
during harmonization
• Development of a common upper ontology Starting point for new ontology construction Bridge between existing ontologies Act as the ground basis for the establishment of
an integrated ontology framework Back up the common upper ontology
Informatics and Telematics InstituteCentre for Research and Technology Hellas
BOEMIE
Multimedia annotation tool and
A proposed knowledge framework
Informatics and Telematics InstituteCentre for Research and Technology Hellas
16
M-OntoMat Annotizer
Mid-Level
Concepts
Selection of regions for mergingSemantic
labeling
Automatic segmentation
Region merging
functionality
Descriptor extractionDescriptor extraction
Informatics and Telematics InstituteCentre for Research and Technology Hellas
18
Knowledge framework
Core Ontology-Holistic Representation
Domain Ontology
Spatiotemporal Constraints
Modality Elements
Multimedia Content Ontology
Classifiers/ Algorithms
Features/ Multimedia Descriptor Ontology
Informatics and Telematics InstituteCentre for Research and Technology Hellas
19
Athletics_Event
High_ Jump_Event
Pole_Vault_EventLong_Jump_E
vent
Jumping_Event
Horizontal_Bar
Pole
Sandpit
Foam_Mats
Foam Mats
Mats
Still region
Multimedia object
VideoImage
Video segment
Element
Textual Element
Visual Element
Horizontal_Bar_ve
Sandpit_ve
Foam_Mats_ve
is_Right
is_Below
is_On
HasVis
ualEle
ment
Still_region_Inst1Foam_Mats_ve_Inst1 Still_region_Inst2
Horizontal_Bar_ve_Inst1depicts
Horizontal_Bar_Inst_1 Foam_Mats_Inst_1HasVisualElem
ent
depicts
Image_Inst1
Athletics_Event_Inst_1
Pole_Vaulter_Failed_Landing
High_Jumper_Failed_Landing
High_Jumper_Successful _Landing
Pole_Vaulter_Successful_Landing
Pole_Vaulter_Jump_Off
High_Jumper_ApproachHigh_Jumper_Jump_Off
Pole_Vaulter_Swing_and_Row
An indicative example