User Requirements in Audiovisual Search: a Quantitative Approach

User Requirements in Audiovisual SearchA Quantitative Approach

Danish NadeemRoeland Ordelman

Robin AlyErwin Verbruggen

AUDIOVISUAL ARCHIVE

BUSINESS ARCHIVE DUTCH PUBLIC BROADCASTERS

DIGITIZATION

CLARIAH PRESENTATIE 11 September 2013

5

800.000 hrs

?METADATA

Document levelvs

Segment level

CLARIAH PRESENTATIE 11 September 2013

8

TOOLS

9

USEFUL?

Developer/ICT researcher

User

CLARIAH PRESENTATIE 11 September 2013

10

FEEDBACK

Access toAudiovisualArchives

Cont

ent

Technology

Users

ARCHIVAL METADATASUBTITLES

AUDIO ANALYSIS:SPEECH/SPEAKER RECOGNITION

pose

beach

Luther King

Obama

Visual analysis

AUDIOVISUAL SEARCH

VIDEO HYPERLINKING

Broadcast professionals

Broadcast Professionals

Re-use

Media Archivists Annotate

Research & Education

Broadcast Professionals

Hergebruik

Media Archivists (documentalisten)

Beschrijven

Journalists Research

Academic researchers Investigate

Education Illustrate

Home user

Public EntertainmentEdutainment

21

Use Scenarios&

System Requirements

Interview & elicitation sessions

Mock-up creation & evaluation

Prototype evaluation

System evaluation

Surveys & log analysis

Qualitative

Qualitative

QualitativeQuantitative

Quantitative/Qualitative

Quantitative approach• Categorization of user requirements

• According to different user groups

• Prioritize implementation of system features

1. 2. 3.

Concept Mapping

• Data collection

• Multidimensional Scaling

• Hierarchical Cluster Analysis

Method• Phase 1: “Idea generation”

– Caveat: unfamiliarity with A/V search– Interviews, requirements elicitation

experiments using system mock-ups– professionals, researchers, home user– 71 “statements”

Method (2)• Phase 2: “Sorting & Rating”

– 47 participants, evenly spread over 3 groups– Categorize according to perceived similarity

• Create categories and label with description

– Rate according to desirability• 1-5: undesirable – highly desirable

Categories vizualisation

Categories

User-defined search functions

Translates into:1. having options for search manipulation and

result representation:– change sort order of results (date, relevance)– Use filters and activate them at any point during

search– Use boolean operators

2. the system should be clear and guide users in a rather complex search process

Advanced search

• Users are eager to make use of advanced search options:– Search using example image– Events, persons, speakers, visual objects– Using multiple metadata sources, e.g.,

• Archival metadata• Speech transcripts• Visual labels

• Less interest in “technical” metadata– Shot types, camera movement, b/w-color

Help functions

• Slight miss-alignment with what developers understand as “help functions”

• Next to “access to help manuals”:– System should be fast – Clear, user-friendly, intuitive UI – Access to video – Clear message when no results are found – Consistent results

USER GROUP COMPARISON

DEVELOPMENT IMPLICATIONS

PRO

RESEARCH

HOME

Conclusions

• Quantitative approach using concept mapping– Useful approach to get some more grip on user

requirements and differences between user groups

– Helps to steer development process of advanced user type specific A/V search tools

– User defined labels may not always be in-line with formal definitions used by developers which may cause confusions, e.g.,

• Help functions vs. clear, smooth and consistent system

Conclusions (2)

• Clear interest of users in 1. advanced search options 2. segment-level access points to AV by means of

rich multi-modal annotations (speech, speakers, events, objects)

3. Clear, transparent, smoothly working system

Upcoming

STUDY HOME USERS

&VIDEO HYPERLINKING

@BBC

http://www.axes-project.eu

Information Retrieval for Information Serviceshttp://www.project-infiniti.nl/