Embed Size (px)
DESCRIPTION
Semantic annotation of media resources has been a focus in research since many years, the closing of the "semantic gap" being seen as key to signi cant improvements in media retrieval and browsing and enabling new media applications and services. However, current tools and services exhibit varied approaches which do not easily integrate and act as a barrier to wider uptake of semantic annotation of online multimedia. In this paper, we outline the Linked Media principles which can help form a consensus on media annotation approaches, survey current media annotation tools against these principles and present two emerging toolsets which can support Linked Media conformant annotation, closing with a call to future semantic media annotation tools and services to follow the same principles and ensure the growth of a Linked Media layer of semantic descriptions of online media which can be an enabler to richer future online media services.
Citation preview
Television Linked To The Web
www.linkedtv.eu
Lyndon Nixon, MODUL University Vienna GmbH Raphael Troncy, EURECOM
Survey of Semantic Media Annotation Tools for the Web: towards New Media
Applications with Linked Media
25 May 2014 LiME Workshop ESWC2014
2
www.linkedtv.eu
Overview
Why Linked Media? What is Linked Media? Requirements on online media annotation Survey of existing tools LinkedTV services and applications for Linked Media Summary and Outlook
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
3
www.linkedtv.eu
Why Linked Media?
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
„It is growing at more than 20% per annum, fuelled by increased demands for new programming and the huge saving it represents compared with shooting new footage. Interactive technology and the Internet will further contribute to the growth of the market as it makes stock footage cheaper and easier to locate and license.“ - http://moneyam.uk-wire.com/cgi-bin/articles/200201020827103514P.html
4
www.linkedtv.eu
Why Linked Media?
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
Dublin Core Metadata Initiative (DCMI)
Metadata Encoding and Transmission Standard (METS)
5
www.linkedtv.eu
Why Linked Media?
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
6
www.linkedtv.eu
What is Linked Media?
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
• What is being described? • The media object – what is it?
Video uploaded to YouTube 2011 ZDF documentary broadcast 1980 Rembrandt self-portrait from 1639 Date?
Is Rembrandt a subject of the whole video? Or only a specific temporal segment of the video? Is the whole frame important or only the part with Rembrandt‘s face?
7
www.linkedtv.eu
What is Linked Media?
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
Media fragment players: http://ninsuna.elis.ugent.be/MFPlayer/html5 http://linkedtv.eurecom.fr/mediafragmentplayer Emerging support in Web browsers: Firefox, Safari, Chrome
8
www.linkedtv.eu
What is Linked Media?
The object of annotation • What is the description of the media?
"Rembrandt"
"Rembrandt self-portrait" from 1639
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
9
www.linkedtv.eu
What is Linked Media?
Digital identifiers for non-addressable resources • “URIs for concepts” since the Semantic Web
• Linked Data has popularized the notion
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
10
www.linkedtv.eu
What is Linked Media?
1. Web media descriptions need a common representation of media structure
2. Web media descriptions need a
common representation of media content
3. Web media descriptions need to use a media ontology which supports description of both the structure and content of media
4. The descriptions of media in terms of common representations of structure and content are the basis for deriving links across media on the Web (Linked Media)
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
11
www.linkedtv.eu
Requirements on online media annotation
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
“Just as Web browsers could never have been successful without a core consensus on the Web page mark up (HTML), Web media can not fulfil its potential without a core consensus on the available descriptive metadata.”
(1) tool or service is online and allows to annotate media of different types via URL (2) media can be annotated in terms of its parts and the media fragmentation follows a non-proprietary approach supported by other tools (3) media can be annotated by concepts where a shared understanding is possible across systems (4) media ontology used for the generated descriptions is interoperable across systems
12
www.linkedtv.eu
Survey: overview of annotation tools
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
EuropeanaConnect Media Annotation Suite (http://dme.arcs.ac.at/annotation/)
"Developed in the EuropeanaConnect project, it supports image,audio and video. Both DBPedia and Geonames resources can be annotation targets, and are suggested from free text or location references respectively. Annotations can be exported as RDF using a tool-specific vocabulary."
13
www.linkedtv.eu
Survey: overview of annotation tools
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
ImageSnippets (http://www.imagesnippets.com/)
Open Video Annotation (http://www.openvideoannotation.org/)
14
www.linkedtv.eu
Survey: overview of annotation tools
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
Annotorius (http://annotorious.github.io/)
Pundit and SemTube (http://thepund.it and http://www.semedia.dibet.univpm.it/semtube/)
15
www.linkedtv.eu
Survey: overview of annotation tools
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
Web based; open videos from online or the repository; term suggestion; export annotations as file or to repository.
ConnectME annotation tool https://github.com/lyndonnixon/ annotationtool Contact [email protected] GNU GPL 3.0 license
16
www.linkedtv.eu
LinkedTV: background
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
40% of TV viewers are using a companion device alongside the TV program.*
* J. Abreu, P. Almeida, B. Teles, and M. Reis. Viewer behaviors and practices in the (new) television environment. In Proceedings of the 11th European Conference on Interactive TV and Video, EuroITV '13.
http://www.linkedtv.eu
Ever saw something on TV and wanted to know more about it, but didn‘t even know how to search for it?
17
www.linkedtv.eu
LinkedTV technology
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
“...schilderij van Jan Sluijters....”
Video object and word detection
dbpedia.org/resource/Jan_Sluyters
Connection to concepts
Paintings by Jan Sluijters
Selection of related concepts
Selection of related content
Presentation engine
18
www.linkedtv.eu
Annotation task: automatic approaches
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
http://nerd.eurecom.fr
19
www.linkedtv.eu
Media Fragment + Open Annotation + NERD
08.11.2012
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
20
www.linkedtv.eu
Annotation task: automatic approaches
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
21
www.linkedtv.eu
Annotation task: automatic approaches
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
http://linkedtv.eurecom.fr/tv2rdf/
22
www.linkedtv.eu
Annotation task: manual approaches
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
The LinkedTV Editor Tool will have a public code release in late 2014 (planned GNU GPL 3.0), but a test installation can be used at http://editortool.linkedtv.eu (Enquiries: [email protected])
23
www.linkedtv.eu
Enrichment task: associating media
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
Edward Snowden asks for asylum in Russia (04 / 07 / 2013)
24
www.linkedtv.eu
Enrichment task: associating media
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
In which Russian airport is he exactly?
LSCOM:Face LSCOM:Building
?
25
www.linkedtv.eu
Enrichment task: associating media
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
List of Relevant Named Entities (1) Named Entity
(2) Filtering and Ranking
b) Expanded Entities
b) Re-ranked Entities
a) Entities from Video
26
www.linkedtv.eu
Enrichment task: associating media
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
http://linkedtv.project.cwi.nl/news/mainscreen.html
http://linkedtv.project.cwi.nl/news/ Second Screen @
Main Screen @
27
www.linkedtv.eu
Outlook
Survey of Semantic Media Annotation Tools for the Web: Towards new Media Applications with Linked Media
Tool Input Fragment support Conceptual annotation
Ontology
Europeana Connect
Audio, image or video. By URL
Spatial & temporal (SVG)
DBPedia and GeoNames
Own vocabulary
ImageSnippets Image only. By URL Spatial. Free to import from LOD or create own
LIO ontology
OpenVideo Annotation Video. Unclear if support URL
Only temporal in the demo, unknown syntax
Free text or tags Open Annotation Model
Annotorius Image only. By URL. Spatial, proprietary syntax
DBPedia No ontology (JS data model)
Pundit and SemTube Image and video. By URL
Spatial, proprietary syntax
Freebase. Terms can be imported.
Own RDF vocabulary
ConnectME Video. By URL Spatial & temporal using MF URIs
DBPedia and Geonames
ConnectME Ontology
LinkedTV TV2RDF and Editor Tool
Video. By URL (Editor Tool uses resources on the LinkedTV platform)
Spatial & temporal using MF URIs
DBPedia. Editor tool suggests DBPedia but allows any URI.
LinkedTV Ontology
