1. Music retrieval Conventional music retrieval systems Exact queries: ”Give me all songs from J.Lo’s latest album” What about ”Give me the music that I like”?  New methods are needed: sophisticated similarity measures Increasing importance: MP3 players (10 3 songs) Personal music collections (10 4 songs) Music on demand many songs, huge market value…

2. Proposal Try a classifier method –Similarity measure  enables matching of fuzzy data  always returns results Implement relevance feedback –User feedback Improves retrieval performance

3. Classifier systems Genetic programming Neural networks Curve fitting algorithms Vector quantizers

4. Tree structured Vector Quantization Audio parameterization Feature space: MFCC coefficients Quantization tree A supervised learning algorithm, TreeQ: Attempts to partition feature space for maximum class separation

5. Features: MFCC coefficients waveform DFTLogMelIDFT MFCCs: A 13-dimensional vector per window 5 minutes song  30  10 3 windows 100 Hamming windows/second

6. Classifying feature space

7. Nearest neighbor Discrimination line in feature space Problems: –Curse of dimensionality –Distribution assumptions –Complicated distributions

8. Vector Quantization: Adding decision surfaces Each surface is added such that It cuts only one dimension (speed) the mutual information is maximized:

12. Until further splits are not worthwile – according to certain stop conditions

13. Decision tree Tree partitions features space –L regions (cells/leaves) –Based on class belonging of training data

14. Template generation Generate templates for –Training data –Test data Each MFCC vector is routed through the tree

15. Template generation With a series of feature vectors, each vector will end up in one of the leaves. This results in a histogram, or template, for each series of feature vectors.

16. Template comparison Corpus templates – one per training class ABn X Query template Compute similarity sim(X,A), sim(X,B), sim(X,C), …sim(X,n) Augmented similarity measure, e.g. DiffSim(X) = sim(X,A) – sim(X,C)

17. Template comparison Corpus templates – one per training class ABn Query templates Compute similarity DiffSim(X) Sort Result list

18. Preliminary experiments Test subjects listened to 107 songs Rated them: good, fair, poor (class belonging C g, C f, C p ) Training process: –For each user Select randomly a subset (N songs) from each class Construct a tree based on class belonging Generate histogram templates for C g, C f, C p For each song X –Generate histogram template –Compute DiffSim(X) = sim(X,C g ) – sim(X,C p ) Sort the list of songs according to DiffSim

19. Results N13579 random,236,234,246,240,234 cos,305,364,370,388,389

20. Relevance feedback Result list user classifier

21. Implementation Adjust histogram profiles based on user feedback For each user –Select the top M songs from the result list –Add the contents of the songs to the histogram profile based on the user rating (class belonging C g, C f, C p ) –For each song X Generate histogram template Compute DiffSim(X) = sim(X,C g ) – sim(X,C p ) –Sort the list of songs according to DiffSim

22. Improvement Amount of training data N M13579 127,685,8810,222,524,74 340,9419,7023,8017,2027,50 552,1532,1434,0827,9940,59 762,8943,4543,7636,4552,89