Upload
roelandordelmannl
View
161
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Presentation at The International Conference on Theory and Practice of Digital Libraries (TPDL)
Citation preview
User Requirements in Audiovisual SearchA Quantitative Approach
Danish NadeemRoeland Ordelman
Robin AlyErwin Verbruggen
AUDIOVISUAL ARCHIVE
BUSINESS ARCHIVE DUTCH PUBLIC BROADCASTERS
DIGITIZATION
CLARIAH PRESENTATIE 11 September 2013
5
800.000 hrs
?METADATA
Document levelvs
Segment level
CLARIAH PRESENTATIE 11 September 2013
8
TOOLS
9
USEFUL?
Developer/ICT researcher
User
CLARIAH PRESENTATIE 11 September 2013
10
FEEDBACK
Access toAudiovisualArchives
Cont
ent
Technology
Users
ARCHIVAL METADATASUBTITLES
AUDIO ANALYSIS:SPEECH/SPEAKER RECOGNITION
pose
beach
Luther King
Obama
Visual analysis
AUDIOVISUAL SEARCH
VIDEO HYPERLINKING
Broadcast professionals
Broadcast Professionals
Re-use
Media Archivists Annotate
Research & Education
Broadcast Professionals
Hergebruik
Media Archivists (documentalisten)
Beschrijven
Journalists Research
Academic researchers Investigate
Education Illustrate
Home user
Public EntertainmentEdutainment
21
Use Scenarios&
System Requirements
Interview & elicitation sessions
Mock-up creation & evaluation
Prototype evaluation
System evaluation
Surveys & log analysis
Qualitative
Qualitative
QualitativeQuantitative
Quantitative/Qualitative
Quantitative approach• Categorization of user requirements
• According to different user groups
• Prioritize implementation of system features
1. 2. 3.
Concept Mapping
• Data collection
• Multidimensional Scaling
• Hierarchical Cluster Analysis
Method• Phase 1: “Idea generation”
– Caveat: unfamiliarity with A/V search– Interviews, requirements elicitation
experiments using system mock-ups– professionals, researchers, home user– 71 “statements”
Method (2)• Phase 2: “Sorting & Rating”
– 47 participants, evenly spread over 3 groups– Categorize according to perceived similarity
• Create categories and label with description
– Rate according to desirability• 1-5: undesirable – highly desirable
Categories vizualisation
Categories
User-defined search functions
Translates into:1. having options for search manipulation and
result representation:– change sort order of results (date, relevance)– Use filters and activate them at any point during
search– Use boolean operators
2. the system should be clear and guide users in a rather complex search process
Advanced search
• Users are eager to make use of advanced search options:– Search using example image– Events, persons, speakers, visual objects– Using multiple metadata sources, e.g.,
• Archival metadata• Speech transcripts• Visual labels
• Less interest in “technical” metadata– Shot types, camera movement, b/w-color
Help functions
• Slight miss-alignment with what developers understand as “help functions”
• Next to “access to help manuals”:– System should be fast – Clear, user-friendly, intuitive UI – Access to video – Clear message when no results are found – Consistent results
USER GROUP COMPARISON
DEVELOPMENT IMPLICATIONS
PRO
RESEARCH
HOME
Conclusions
• Quantitative approach using concept mapping– Useful approach to get some more grip on user
requirements and differences between user groups
– Helps to steer development process of advanced user type specific A/V search tools
– User defined labels may not always be in-line with formal definitions used by developers which may cause confusions, e.g.,
• Help functions vs. clear, smooth and consistent system
Conclusions (2)
• Clear interest of users in 1. advanced search options 2. segment-level access points to AV by means of
rich multi-modal annotations (speech, speakers, events, objects)
3. Clear, transparent, smoothly working system
Upcoming
STUDY HOME USERS
&VIDEO HYPERLINKING
@BBC
http://www.axes-project.eu
Information Retrieval for Information Serviceshttp://www.project-infiniti.nl/