Upload
montgomery-webster
View
1.404
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Literature review and exploration of music visualization as an Information Visualization research area. Presentation examines all forms of visual music, from 2d images and music videos to media player plug-ins and Computer Science publications.
Citation preview
A Preliminary Evaluation of
Music VisualizationDanny Rado
The “clavilux” [above] was invented by Thomas Wilfred in 1924
¿Is Music Visualization an area of research? PlanMotivationEveryday music visualizationsResearchThe very best of music visualizationsDiscussionIdentify some key issues for the fieldDetermine why the field is underdeveloped
Motivation :: Karaoke
Karaoke sounds bad because it is difficult:• The text gives little music information • User must rely on memory
Motivation :: PersonalAI project:• LongSong: Automatic Song Extension
Songs can be extended at theme repetitionsIdentifying theme structure by sound is difficult
The identification is simple with a 2D visualization:
Martin Wattenburg, http://www.bewitched.com/match/music.html
Motivation :: Music InformaticsMusic information that requires visualization:• Interaction of instruments• Tonality, movement of tonal center• Rhythm• Relationship between notes
o Consonance, dissonance, triadic relationships• Harmonic Analysis• Phrase Structure• Note spelling
Bach, Minuet G Major
Itunes Visualization
Effects are generated on sound eventsThese visualizations give less information than the standard music video
Windows Media Player Visualization
Frequency bars are the most informative visualizationsStill not portraying any depth of music visualization
Winamp Visualization
Stick Figures plug-in allows users to create drawings and animate them to a few frequenciesGives semantic meaning to music information making it easier to identify and follow
Media Player Visualization Review• Equivalent to music video screen savers• Simply hypnotic, psychedelic imagery
Despite the useful framework for plug-ins, the creative effort is effectively wasted• Frequencies simply have little inherent information
The Standard Music Video
Not a visualization• Limited information is obscured by performance, creativity,
and celebrity
Shows the instruments being played Serves as a baseline to measure the success of music visualizations
The Important Music Video
Hints at the structure of the music Graphics are repeated on musical theme repetition
Music Format• Standard music formats do not have enough information
MIDI format has far more information Instead of audio information, MIDI stores composition information:• Instruments• Note times• Note intensities• Changes in pitch• Changes in rhythm
MIDI creates the opportunity to visualize music
MIDI Text [Midi Disassembler]1. MThd | Format=1 | # of Tracks=8 | Division=384
Track #0 ****************************************** Time Event 1: 1: 0 |Tempo | BPM=140 | micros\quarter=428570 |Time Sig | 4/4 | MIDI-clocks\click=24 | 32nds\quarter=8 106: 1: 0 |End of track|
Track #1 ****************************************** Time Event 1: 1: 0 |Track Name | len=9 | 0x31 0x73 0x74 0x20 0x56 0x6F 0x69 0x63 0x65 <1st Voice> 9: 1: 0 |Program | chan=14 | pgm #= 54 Choir Oohs |Controller | chan=14 | contr=Volume H | value=100 96 |Controller | chan=14 | contr=Effects | value= 40 3: 0 |On Note | chan=14 | pitch=F 3 | vol=127 336 |Off Note | chan=14 | pitch=f 3 | vol=127 4: 0 |On Note | chan=14 | pitch=G 3 | vol=127 336 |Off Note | chan=14 | pitch=g 3 | vol=127 10: 1: 0 |On Note | chan=14 | pitch=A 3 | vol=127 336 |Off Note | chan=14 | pitch=a 3 | vol=127 2: 0 |On Note | chan=14 | pitch=G 3 | vol=127 4:332 |Off Note | chan=14 | pitch=g 3 | vol=127
2. [Skipping ahead]3. Track #5 ******************************************
Time Event 1: 1: 0 |Track Name | len=7 | 0x54 0x72 0x75 0x6D 0x70 0x65 0x74 <Trumpet> 2: 1: 0 |Controller | chan= 4 | contr=Effects | value= 40 |Program | chan= 4 | pgm #= 57 Trumpet |Controller | chan= 4 | contr=Volume H | value=100 1 |On Note | chan= 4 | pitch=F 2 | vol=62
UIUC Research: Isochords Isochords: Visualizing Structure in Music,• Helps classify music structure with information about:
o chord qualityo interval qualityo chord progression
• http://social.cs.uiuc.edu/projects/isochords-movies/Here%20Comes%20the%20Sun.mov
University of Illinois,Urbana Champaign
UL + CUL Research: i-Maestro
3D Augmented Mirror: A Multimodal Interface for StringInstrument Learning and Teaching with Gesture Support• Prevent or repair bad performance posture
University of Leeds + City University of London
USC Research: MuSA.RT
MuSA.RT: Music on the Spiral Array . Real-Time• a multimodal interactive system for music analysis and visualization• 3D model, shows tonal structures such as pitches, chords and keys.
Applications: Animusic
Manually Animated Music Visualizations• Invent virtual instruments that often resemble real ones• Visualizations present some limited music information
Applications: MidiJam
Automatically Animated Music Visualizations• Shows interaction between real instruments and animates
their performance
Applications: Music Animation MachineProgram has many display types:Piano Roll [right], Pitch Class Wheel, Interval Wheel, Interval + Pitch, Shapes,Tonality Staff, Tonality Compass, Triads Lattice, Parts Sequence:• Lines• Balls• Wedges
Applications: Music Animation MachineProgram has many display types:Piano Roll, Pitch Class Wheel, Interval Wheel, Interval + Pitch, Shapes,Tonality Staff, Tonality Compass, Triads Lattice, Parts Sequence:• Lines• Balls [right]• Wedges
Conclusion:Despite the common perception that Music Visualization is not an area of research, significant work has been done.
Beethoven, Fur Elise
Discussion:• Does the Music Visualization research seem substantial?• Why are there so few publications in the field?
o ACM Portal has less than 35• Can we identify some key issues for the field?