Download presentation
Presentation is loading. Please wait.
1
1 Robust Temporal and Spectral Modeling for Query By Melody Shai Shalev, Hebrew University Yoram Singer, Hebrew University Nir Friedman, Hebrew University Shlomo Dubnov, Ben-Gurion University
2
2 Prelude
3
3 Problem Setting Database of real recordings Query: a melody Find: performances of the queried melody
4
4 Challenge Find performances of the queried melody independent of: –Tempo –Performing instrument –Dynamics –Expression –Accompaniment
5
5 Related Work A. Ghias, et al. “Query by humming” A. S. Durey and M. A. Clements. “Melody spotting using hidden markov models” C. Raphael. “Automatic segmentation of acoustic musical signals using HMMs” B. Doval and X. Rodet. “Fundamental frequency estimation using a new harmonic matching method”
6
6 Overview of Solution Employ a statistical framework Align a melody to a performance using an explicit tempo modeling Employ a maximum likelihood model for the spectrum of a note given the note’s pitch value Find the best alignment of a melody to a performance using dynamic programming
7
7 Statistical Framework Query Engine For each recording find: A database of real recordings A melody query Ranked list of According to
8
8 Melody Modeling Hidden Variable Observed Variable Legend: MelodyTempo Aligned Melody Sound
9
9 Tempo Modeling Sequence of scaling factors (one per note) Model tempo as a first order Markov model Use log-normal distribution to model conditional probability of tempo
10
10 Spectral Modeling
11
11 Spectral Modeling
12
12 Spectral Modeling (cont.)
13
13 Spectral Modeling (cont.) Estimate the amplitude at each harmony and global variance of the noise using the maximum likelihood principle Resulting signal-to-noise likelihood function:
14
14 Finding the best melody-performance alignment Recurse over tempo and end-time of the previous note Dynamic Programming procedure Complexity: #notes Length of Signal #Possible Tempo values
15
15 Queries: 50 melodies from opera arias (from Midi files) Database: over 800 performances of opera arias performed by over 50 tenors with full orchestral accompaniment Compared our variable-tempo (VT) model vs. fixed-tempo (FT) and locally-fixed-tempo (LFT) models Compared our Harmonic with Scaled Noise (HSN) spectral model vs. Harmonic with Independent Noise (HIN) model Experimental Results
16
16 Evaluation Measures Oerr = 0 Cov = 3 - 2 + - + - - - - - Likelihood Value Index of Performance in the ranked list 12345
17
17 Summary of Results One Error of VT+HSN: 8% Average Precision of VT+HSN: 95% Coverage of VT+HSN: 0.21
18
18 Results 0.7521.670.350.6922.960.38 FT 0.7517.940.370.6917.330.43 LFT 0.6911.830.460.6510.670.51 VT 5 Sec. 0.7319.080.360.7119.830.38 FT 0.428.150.660.448.100.66 LFT 0.193.020.830.191.750.86 VT 15 Sec. 0.7922.460.330.7720.690.34 FT 0.485.980.630.465.900.66 LFT 0.100.400.920.080.210.95 VT 25 Sec. OerrCovAvgPOerrCovAvgP HINHSN Spectral Distribution Model
19
19 Precision-Recall
20
20 Illustration of Segmentation
21
21 Future Work More data Other genre of music Alternative spectral distribution models using supervised learning methods. Use alignment results for separating a soloist from the accompaniment
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.