Designing Software for Personal Music Management and Access Frank Shipman & Konstantinos Meintanis Department of Computer Science Texas A&M University

Why Music Management? Lots of people have large personal collections. Common software supports access via context- independent metadata –Artist, title, album, year, … But selection happens based on personal interpretation –Memories, emotional effect, … Room for improvement

Supporting Personal Interpretation How to do this? –Could create metadata fields … but experience shows that those will not be used. –Could use a Web 2.0 strategy (few contributors, many consumers) but that only works if there is agreement. Instead, we draw from our experience in supporting personal interpretation during information analysis

Spatial Hypertext Provide a visual workspace for arranging and annotating content that facilitates rapid expression. Prior uses: –Intelligence analysis –Meeting management –Authoring papers or presentations

“Float the Idea” Study To explore the potential, we decided to see how people would use an existing spatial hypertext to work with music. Visual Knowledge Builder (VKB) –Users organize information objects in a hierarchy of collections –Users modify visual attributes of information objects to indicate characteristics of the content

The Study Participants –12 graduate students age –Most had substantial personal music collections –Only 1 was satisfied with playlist creation features in existing software Music collection –participants chose between existing collections of 100 songs in rock or classical music Task –organize the collection for later use in 60 minutes –create three playlists for events in 30 minutes –Time is an issue for this activity

Example Organizations Single-level organization using collections, color and border width Organization using categories, subcategories and labels

What worked and what did not The Good –Expressed lots of personal opinions, memories, activities, and effects. –Preview feature good for reminding The Bad –Wanted to search by traditional metadata –Needed to hear more for unknown songs

An Augmented Design Design goals: –Combine access via context-free metadata with personal expression –Support rapid access and playlist creation through suggestions –Provide audio summaries of songs The result: MusicWiz

MusicWiz Spatial Hypertext Workspace File System Tree View Related Songs Tree View & Search Results Playback Controls Playlist Pane Spatial Hypertext Workspace

MusicWiz

Generating Music Suggestions Suggestions are meant –to provide more rapid access to music users want –to identify music that the user might like but not know they like –to be appropriate based on the user’s current mood and activity MusicWiz uses a multifaceted approach to generating suggestions

Facets of Similarity Metadata –How many metadata fields are the same or similar Lyrics –How similar are the topics in the songs Audio characteristics –How similar is the beat, brightness, pitch, and the key User expression –How similar are the user’s placement and visualization Web-based information –How similar do others think of the songs

Generating Music Summaries Most previews are either the introduction or the refrain of the song. Research focus has been on detecting the refrain. We focus on creating multiphrase music summaries – summaries that piece together selections from the song. How to select the sections?

Our Approach(es) Three phrase summaries with –the most salient phrase (8 seconds) –two supplemental phrases (6 seconds each) –1 second of silence between the phrases Developed three algorithms –All share the approach to selecting most salient phrase –Differ in selection of supplemental phrases based on whether to place greater value on the sonic uniqueness of the phrases or the frequency of the phrases in the song.

Study of Summaries Purpose –Evaluate and compare algorithms’ power in the tasks of remembering known songs and getting familiar with new ones Participants –Fifteen, over 18 years old, mainly students, including 12 men and 3 women. –Majority (67%) had some kind of music education and more than half of them (67%) a personal music collection of at least 50 songs (8 participants had more than 200 songs)

Study Design Participants were asked to choose and rate the summary that best represents each of twenty popular rock and pop songs Four 22-second summaries per song, three generated with our algorithms and one representing the first 22 seconds of the song No time limit for the completion of the task

Self-Reported Results Parts / features of songs fundamental for becoming familiar with and recalling music –Introduction most important for remembering –Refrain best for becoming acquainted with the music

Best Summary Results Distribution of participants’ selections for their favored summary: –The introduction was chosen in only 13% of the cases while the REA in 35% of the cases –The difference between the introduction-based algorithm and the REA and IA was statistically significant (Tukey HSD test, P=0.001 and P=0.041 respectively, α=0.05)

Discussion Study showed that –the multi-phrase summaries represent music better than just its introduction –the selection of the best summary was independent of its effectiveness and the music knowledge and recency of access –there was a likely preference for algorithms that emphasize the selection of repeated phrases

Summary Examining personal music management Explored use of spatial hypertext to organize music collections Designed MusicWiz to combine context- free metadata and personal expression Developed and compared algorithms for generating multi-phrase music summaries.