Audio Culture Researcher Requirements, Metadata Design and Potentials of Automated Feature Extraction
Birger Larsen & the RSLIS LARM/CoSound teamInformation Systems and Interaction DesignRoyal School of Library and Information ScienceUniversity of CopenhagenDenmark
www.iva.dk/blar - [email protected]
Marianne Lykke AAU
Mette SkovAAU
Toine BogersRSLIS
Haakon LundRSLIS
Slide 2
CoSound, June 21, 2013
This talk
• Audio culture researcher requirements for an audio research infrastructure
• User-driven development of a metadata scheme for radio broadcast archives
• Reflections on the potential of automated feature extraction
Slide 3
CoSound, June 21, 2013
Audio culture researcher requirements for an audio research infrastructure
LARM Audio Research Archive• Research infrastructure for radio and
auditive cultural heritage, 2010-1013• Funded by the Danish Ministry of Science, Technology
and Innovation• Consortium of 13 Danish institutions, universities +
Danish Radio + State and University Library
Objective: To further radio and audio research by maturing research source material
• IT-infrastructure (main architecture and interface)• Solutions to copyright issues• National bibliography of radio• Toolbox: Tools for metadating, annotating, search and
disseminationOrganisation: technical solutions, national bibliography,
research cases
Slide 4
CoSound, June 21, 2013
Two main challenges
• Large heterogeneous collection• 1 million hours of broadcast radio• 1937-present (24/7 coverage from 1989)• Extremely sparse metadata, no funding to index• Streaming access only because of copyright issues
• Heterogeneous user group• All audio culture researchers... • ...but very diverse research interests• No public access due to copyright > large scale
crowdsourcing not possible
Slide 5
CoSound, June 21, 2013
Audio culture researcher requirements
Initial wish: a common philology / metadata / classification scheme to be used across researchers
How to design a metadata scheme for audio culture research?
Analysis of Humanities information needs• Questionnaire to Danish and international audio culture
researchersAnalysis of Humanities research tasks and needs
• Wiki for describing and analysing used concepts and coding wishes and practises
Slide 6
CoSound, June 21, 2013
Questionnaire results
Brain storming + tagging sessions
Respondents17 Danish + 51 international
Preferred search entries (how do you/would you like to search for audio for your research?)
• On average title, genre and topic are important, media specific metadata less so
• But large individual differences
Slide 7
CoSound, June 21, 2013
48%
45%48%
60%
Questionnaire results
Indexing level: Which unit of material is important??
• Several levels of indexing is necessary
Needed more detail:• Wiki for describing
and analysing used concepts and coding wishes and practises
Slide 8
CoSound, June 21, 2013
Sample research cases
WP5.8 – The auditive syntax of radio• Jacob Kreutzfeldt: Urban Soundscapes
• Identifying ‘real sound’ from Copenhagen• Analysing how the ‘experience of space’ is
constructed • Investigating the correlation between technology
and the portrait of ‘the sound of the city’• Need for search and annotation
• Sound category (e.g. traffic, shop, car, bell, goat)• Free tagging• Mix (sound level (fore/background), fade in/out)• Authenticity (real/scenographic /effect)• Anchorage (space, language, rhythm, theme)• Other
• Torben Sanglid: DR Jingles and Theme Songs• Investigating changes of focus over time• Theme songs (intro, separators, outro)• Station IDs, Spots
CoSound, June 21, 2013
Developing a metadata scheme
Need for • effective retrieval of radio broadcasts• adding research-specific annotations• at both the broadcast level as well as at segments of
broadcasts. Needs of humanities researchers are so diverse that it is
unlikely that a single unified subject list will suit allMain metadata requirements
• easy to work with• easily extensible• would provide for flexible data exchange
Different sources• Original archive data• User generated
• Of a general nature• Project specific
Slide 10
CoSound, June 21, 2013
Conceptual metadata scheme
Slide 11
CoSound, June 21, 2013
Channel, program title, start/end time, narrative, creators and roles
Program title (missing or wrong), persons (participants/subject), genre, related objects, subject, tags, annotation
[project specific]
Slide 12
CoSound, June 21, 2013
Archivemetadata
Administrative metadata
LARM metadata
Project metadata
Administrative metadata
Administrative metadata
Slide 13
CoSound, June 21, 2013
Slide 14
CoSound, June 21, 2013
Conceptual metadata scheme
Consistency across project metadata?• project metadata scheme encoded in streaming
interface• Coding manual saved in repository
• Documentation• Reuse
Current state• 3-level schema implemented in larm.fm• In use by some research groups• Individual project schemas needs to hand coded
Slide 15
CoSound, June 21, 2013
Reflections on the potential of automated feature extraction
Flexible annotation made possible• But still relatively few programs can be annotated
Automatic feature extraction has the potential to enrich the audio files automatically
• First pass to e.g. segment the files• More manual passes to refine and correct automatic
output• Data driven humanities
For search in radio programs, automatic speech recognition holds great potentials
• But not easy to do well for Danish• On-going tests on DR radio news and the Queen’s New
Year addresses in collaboration with SpeechOp.com
Slide 16
CoSound, June 21, 2013
References and links
LARM – http://www.larm-archive.orgCoSound – http://cosound.dk
Unlocking radio broadcasts : User needs in sound retrieval / Lykke, Marianne; Skov, Mette. Proceedings of the 4st international conference on Information interaction in context. Association for Computing Machinery, 2012. s. 298-301.
CHAOS: User-driven Development of a Metadata Scheme for Radio Broadcast Archives / Lykke, Marianne; Bogers, Toine; Larsen, Birger; Lund, Haakon. iConference 2013 Proceedings. 2013. (Poster)
Slide 17
CoSound, June 21, 2013
Search and Interaction in Media Archives
Open seminar at DR, 9:00-13:00 Thursday June 27, 2013
Designing the Search ExperienceTony Russell-Rose, Director of UXlabs, UK
larm.fm master classAndreas Røll Larsen, DR Archive, Research & Rights
Good Practices for Finding InformationKristian Norling, Enterprise Search Evangelist at Findwise, Sweden
Interaction with Sound Art InstallationsMarianne Lykke, Professor at University of Aalborg, Denmark
The Future of Music Interaction Daniel Boland, PhD student at University of Glasgow, UK
CoSound, June 21, 2013
http
://b
it.ly
/DRse
min
ar