Content Visualization in a Content Visualization in a Digital Music LibraryDigital Music Library

Eric J. IsaacsonEric J. Isaacson

Assoc. Prof. of Music TheoryAssoc. Prof. of Music Theory

Indiana University School of MusicIndiana University School of Music

This material is based upon work supported by the National Science Foundation under Grant No. 9909068. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the presenters and do not necessarily reflect the views of the National Science Foundation.

3rd International Workshop on Information Visualization Interfaces for Retrieval and Analysis (IVIRA), 31 May 2003 (Houston)

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

22

Importance of Vis. in MusicImportance of Vis. in Music

““Freezes” otherwise ephemeral content by Freezes” otherwise ephemeral content by mapping temporal onto a spatial domainmapping temporal onto a spatial domain

Supports recall, reproductionSupports recall, reproduction Can depict structures/featuresCan depict structures/features

Hidden and surfaceHidden and surface Large and small scaleLarge and small scale HierarchiesHierarchies

Enables communicationEnables communication

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

33

Quick History of Western Quick History of Western NotationNotation

Origins are >1000 years old (heightened Origins are >1000 years old (heightened neumes)neumes)

Began as memory aidBegan as memory aid Rhythmic notation develops ca. 1300Rhythmic notation develops ca. 1300 ““Modern” music notation reaches present Modern” music notation reaches present

form ca. 1600form ca. 1600

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

44

Notation as VisualizationNotation as Visualization

Useful for random accessUseful for random access Provides concise guide for how music Provides concise guide for how music

soundssounds Trained musicians can read “structural” Trained musicians can read “structural”

features in notation (but only indirectly)features in notation (but only indirectly)

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

55

Variations2: Indiana Univ. Variations2: Indiana Univ. Digital Music LibraryDigital Music Library

ContentContent Digital Audio/VideoDigital Audio/Video Scanned score imagesScanned score images Symbolically encoded score filesSymbolically encoded score files

Software forSoftware for Enhanced search for musicEnhanced search for music Player/viewers for (synchronized) playbackPlayer/viewers for (synchronized) playback Notation display, editingNotation display, editing Library catalogingLibrary cataloging Learning, study, teachingLearning, study, teaching

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

66

Potential Visual SearchPotential Visual Search

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

77

Content Vis. in Variations2Content Vis. in Variations2

Primary focus is on user-created, user-Primary focus is on user-created, user-manipulated techniquesmanipulated techniques

Driven by curriculum, disciplineDriven by curriculum, discipline

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

88

Visualization in Variation2Visualization in Variation2

Stage 1: Score imagesStage 1: Score images Bookmarks add minimal visualizationBookmarks add minimal visualization

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

99

Visualization in Variations2Visualization in Variations2

Stage 2: TimelinerStage 2: Timeliner Diagrams large-scale structureDiagrams large-scale structure Initially tied to recording onlyInitially tied to recording only

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1010

Visualization in Variations2Visualization in Variations2

Stage3: Annotation toolsStage3: Annotation tools Drawing, text, smart-labelsDrawing, text, smart-labels Designed for showing Designed for showing

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1111

Visualization in Variations2Visualization in Variations2

Stage 4: Encoded scoresStage 4: Encoded scores Display of selected partsDisplay of selected parts ColorColor AnnotationAnnotation Analytic notationAnalytic notation

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1212

Other Views (Audio)Other Views (Audio)

Spectrograms (auto)Spectrograms (auto) Self-similarity graphs (auto, with Self-similarity graphs (auto, with

parameters)parameters)

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1313

Other Views (Symbolic)Other Views (Symbolic)

Lerdahl & Jackendoff tree diagrams (user)Lerdahl & Jackendoff tree diagrams (user)

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1414

Other ViewsOther Views

Line graphsLine graphs Pitches in Bartók, Pitches in Bartók, Music for Strings, Perc. and Music for Strings, Perc. and

CelesteCeleste, I (Brinkman & Mesiti 1991), I (Brinkman & Mesiti 1991)

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1515

Other ViewsOther Views

Harmonic regions (“keyscape”) in Bach, C Harmonic regions (“keyscape”) in Bach, C Major Prelude (Craig Sapp)Major Prelude (Craig Sapp)

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1616

Other ViewsOther Views

Timelines, “pattern repetition”Timelines, “pattern repetition” http://turbulence.org/Works/song/index.htmlhttp://turbulence.org/Works/song/index.html

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

IVIRA 2003IVIRA 2003 Isaacson, Content Visualization in a Digital MusiIsaacson, Content Visualization in a Digital Music Libraryc Library

1717