Content Visualization in a Digital Music Library Eric J. Isaacson Assoc. Prof. of Music Theory Indiana University School of Music This material is based upon work supported by the National Science Foundation under Grant No Any opinions, findings, and conclusions or recommendations expressed in this material are those of the presenters and do not necessarily reflect the views of the National Science Foundation. 3rd International Workshop on Information Visualization Interfaces for Retrieval and Analysis (IVIRA), 31 May 2003 (Houston)

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 2 Importance of Vis. in Music  “Freezes” otherwise ephemeral content by mapping temporal onto a spatial domain  Supports recall, reproduction  Can depict structures/features  Hidden and surface  Large and small scale  Hierarchies  Enables communication

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 3 Quick History of Western Notation  Origins are >1000 years old (heightened neumes)  Began as memory aid  Rhythmic notation develops ca  “Modern” music notation reaches present form ca. 1600

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 4 Notation as Visualization  Useful for random access  Provides concise guide for how music sounds  Trained musicians can read “structural” features in notation (but only indirectly)

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 5 Variations2: Indiana Univ. Digital Music Library  Content  Digital Audio/Video  Scanned score images  Symbolically encoded score files  Software for  Enhanced search for music  Player/viewers for (synchronized) playback  Notation display, editing  Library cataloging  Learning, study, teaching

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 6 Potential Visual Search

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 7 Content Vis. in Variations2  Primary focus is on user-created, user- manipulated techniques  Driven by curriculum, discipline

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 8 Visualization in Variation2  Stage 1: Score images  Bookmarks add minimal visualization

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 9 Visualization in Variations2  Stage 2: Timeliner  Diagrams large-scale structure  Initially tied to recording only

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 10 Visualization in Variations2  Stage3: Annotation tools  Drawing, text, smart-labels  Designed for showing

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 11 Visualization in Variations2  Stage 4: Encoded scores  Display of selected parts  Color  Annotation  Analytic notation

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 12 Other Views (Audio)  Spectrograms (auto)  Self-similarity graphs (auto, with parameters)

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 13 Other Views (Symbolic)  Lerdahl & Jackendoff tree diagrams (user)

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 14 Other Views  Line graphs  Pitches in Bartók, Music for Strings, Perc. and Celeste, I (Brinkman & Mesiti 1991)

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 15 Other Views  Harmonic regions (“keyscape”) in Bach, C Major Prelude (Craig Sapp)

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 16 Other Views  Timelines, “pattern repetition” 

IVIRA 2003Isaacson, Content Visualization in a Digital Music Library 17