Music retrieval Conventional music retrieval systems Exact queries: ”Give me all songs from J.Lo’s latest album” What about ”Give me the music that I like”?  New methods are needed: sophisticated similarity measures Increasing importance: MP3 players (10 3 songs) Personal music collections (10 4 songs) Music on demand many songs, huge market value…

Proposal Try a classifier method –Similarity measure  enables matching of fuzzy data  always returns results Implement relevance feedback –User feedback Improves retrieval performance

Classifier systems Genetic programming Neural networks Curve fitting algorithms Vector quantizers

Tree structured Vector Quantization Audio parameterization Feature space: MFCC coefficients Quantization tree A supervised learning algorithm, TreeQ: Attempts to partition feature space for maximum class separation

Features: MFCC coefficients waveform DFTLogMelIDFT MFCCs: A 13-dimensional vector per window 5 minutes song  30  10 3 windows 100 Hamming windows/second

Classifying feature space

Nearest neighbor Discrimination line in feature space Problems: –Curse of dimensionality –Distribution assumptions –Complicated distributions

Vector Quantization: Adding decision surfaces Each surface is added such that It cuts only one dimension (speed) the mutual information is maximized:

Until further splits are not worthwile – according to certain stop conditions

Decision tree Tree partitions features space –L regions (cells/leaves) –Based on class belonging of training data

Template generation Generate templates for –Training data –Test data Each MFCC vector is routed through the tree

Template generation With a series of feature vectors, each vector will end up in one of the leaves. This results in a histogram, or template, for each series of feature vectors.

Template comparison Corpus templates – one per training class ABn X Query template Compute similarity sim(X,A), sim(X,B), sim(X,C), …sim(X,n) Augmented similarity measure, e.g. DiffSim(X) = sim(X,A) – sim(X,C)

Template comparison Corpus templates – one per training class ABn Query templates Compute similarity DiffSim(X) Sort Result list

Preliminary experiments Test subjects listened to 107 songs Rated them: good, fair, poor (class belonging C g, C f, C p ) Training process: –For each user Select randomly a subset (N songs) from each class Construct a tree based on class belonging Generate histogram templates for C g, C f, C p For each song X –Generate histogram template –Compute DiffSim(X) = sim(X,C g ) – sim(X,C p ) Sort the list of songs according to DiffSim

Results N13579 random,236,234,246,240,234 cos,305,364,370,388,389

Relevance feedback Result list user classifier

Implementation Adjust histogram profiles based on user feedback For each user –Select the top M songs from the result list –Add the contents of the songs to the histogram profile based on the user rating (class belonging C g, C f, C p ) –For each song X Generate histogram template Compute DiffSim(X) = sim(X,C g ) – sim(X,C p ) –Sort the list of songs according to DiffSim

Improvement Amount of training data N M ,685,8810,222,524,74 340,9419,7023,8017,2027,50 552,1532,1434,0827,9940,59 762,8943,4543,7636,4552,89