A Time Based Approach to Musical Pattern Discovery in Polyphonic Music Tamar Berman Graduate School of Library and Information Science University of Illinois at Urbana-Champaign ICMPC 9, Bologna 2006

Musical Pattern Retrieval Method and system for musical pattern discovery and retrieval Designed as a tool for music researchers, scholars and students. Not designed as a model of human perception Yet, analysis of the system’s outputs through evaluation by humans yields interesting data for music theory/perception/cognition research

Musical Pattern Retrieval Question: Can we create a search engine that receives a sung or played melody as input, and searches for matches in a music database? Answer: Yes –String matching: McNab et al. (1996) –N-grams: Downie and Nelson (2000) –Markov models: Birmingham et al. (2001)

Musical Pattern Retrieval Question: Can we create a search engine that receives a description of a musical structure as input, and searches for matches in a music database?

Musical Schemas / Style Structures Leonard Meyer describes archetypical patterns and traditional schemata that are the “classes” in terms of which particular musical events are perceived and comprehended. “No melody, however original and inventive, is an exception to this principle”‎ (Meyer 1973)

Musical Schemas / Style Structures Eugene Narmour (1977) discusses style forms and style structures, upon which a “stylistic language” is constructed. Style forms are “parametric entities” which achieve enough closure so we can understand their functional coherence without reference to the specific contexts from which they come. Style structures can be created from style forms by arranging them in various contexts “according to their statistically most common occurrences”

Example: The 1-7…4-3 schema Prevalent in 18 th century music First noted by Meyer (1973) and studied further by Gjerdingen (1988) Consists of two event pairs (*): –1-7: The melody descends from the 1st degree to the 7th. The harmony shifts from I to V –4-3: The melody descends from the 4th degree to the 3rd. The harmony shifts from V7 to I Examples: –KV543.sibKV543.sib –KV200.sibKV200.sib (*) Simplified definition

System for Musical Pattern Retrieval Distinguishing features: –Support for the description and retrieval of complex, polyphonic patterns –Noise resilience: instances will be retrieved even if embedded within other patterns or interspersed with other events –User-friendly interface for pattern specification –Retrieved instances can be ranked according to their likelihood of fit to the desired pattern

Retrieving the 1-7…4-3

Sequence Retrieval Example kv268-1.sib Mozart, Violin Concerto No. 6 in Eb K268, Allegro moderato, measures (51.5’)

Test Data 505 Midi files of music by W.A. Mozart, taken from Includes symphonies, piano sonatas, piano concertos, other concertos and piano trios Truncated to first 50 measures Normalized Converted into note objects and then into time series

Time Series Representation A time series is a set of observations on the value of one or more variables, taken at successive points in time In the musical time series: –Variables: 12 pitch classes –Values: role played by pitch class at the time of observation (top/bass/middle/absent) Result: a series of “harmonic windows” representing each musical piece

Musical Time Series Parameters Window length: size (in seconds) of the time interval described by each harmonic window Onset interval: time (in seconds) between window onsets (“sampling rate”)

Use of Absolute Time Units Motivation: –Readily and reliably available in midi data –Potential application to audio data Justification: –For events that are close to each other in time, seconds – rather than beats – are likely more relevant –For fast music, schema events could be further apart (in beats/measures) than for slow music

System Evaluation A selection of 115 retrieved candidate instances were evaluated by 3 human judges and by 12 queries The queries differed from each other in parameters such as window length, onset interval and role specifications for pitch classes within each event Instances that were rated as correct by a majority of the human judges were rated as correct by a majority of the queries => 100% precision is attainable!

System Performance Window LengthOnset IntervalQuery TypePrecision TV CB TV CB TV CB TV CB TV CB TV CB0.600 N/A Majority vote1.000 Optimal at 0.5 second windows - Observed by Wundt (1874)

Question Do these excerpts sound similar? –Mozart Clarinet Concerto in A, K622, beginning of AllegroMozart Clarinet Concerto in A, K622, beginning of Allegro –Mozart Piano Concerto No. 6 in Bb, K238, beginning of RondoMozart Piano Concerto No. 6 in Bb, K238, beginning of Rondo

Similarity They both contain sequences which satisfy the following constrains: 1. The first event includes pitches C, E, G with G on top 2. The second event includes pitches C, E with E on top 3. The third event includes pitches F, A, C 4. The fourth event includes pitches C, E 5. The fifth event includes pitches D, F, A 6. The sixth event includes pitches D, F, A with F on top 7. The seventh event includes pitches C, G 8. The eighth event includes pitches G, B, D, F 9. The maximum duration of the sequence is 15 seconds

Conclusions Applying simple pitch constraints at multiple time resolutions yields successful retrieval The “top voice” requirement for melody is effective –Observed by Meek and Birmingham (2001) Creating a search tool for musical structures is feasible! The technology could be used for similarity retrieval or theme variations retrieval

Future Work Support for constraints on rhythm, contour and metric placement Enabling multiple roles per pitch class Describing distance in beats and measures Integration with alternative representations Application to audio data

Thank You!