Berenzweig - Music Recommendation1 Music Recommendation Systems: A Progress Report Adam Berenzweig April 19, 2002

Berenzweig - Music Recommendation2 Music Recommendation Is: Music IR for the masses –Kids in candy stores –Querying is hard; people can’t describe music –Recommendation can be integrated into players, streaming services, music stores, etc. –Break major label/retail monopoly on choice!!

Berenzweig - Music Recommendation3 Music Recommendation Is: Set-based IR –“Find me items similar to this set, in the way that the set is similar to itself” –Set = collection, or playlist extension. –Be sensitive to themes or aspects of the user’s collection. All about similarity

Berenzweig - Music Recommendation4 Background I: IR/Statistics Collaborative Filtering Latent Semantic Analysis (Deerwester & al., ‘90) –SVD to find hidden meaning Probabilistic LSA (Hofmann, ‘99) –EM to find hidden meaning Latent Class Models (Hofmann, ‘99)

Berenzweig - Music Recommendation5 Background II: Audio IR Artist classification –Whitman & Lawrence; Berenzweig, Ellis & Lawrence Genre classification –Tzanetakis Fingerprinting, query-by-example What features??? –What is it I like about the music that I like?

Berenzweig - Music Recommendation6 Artist Classification Using Vocals Anchor Models Similarity Metrics

Berenzweig - Music Recommendation7 Artist Classification Using Vocals Are vocal segments more easy to identify than instrumental segments? –“Using Voice Segments to Improve Artist Classification of Music”, Berenzweig, Ellis & Lawrence, to appear AES 22.

Berenzweig - Music Recommendation8 Segmented Posteriograms

Berenzweig - Music Recommendation9 Segmented Posteriograms

Berenzweig - Music Recommendation10 Experiment at-a-glance Audio input Cepstra (MFCC, PLP) Artist Classifier Vox/Music Classifier Frame labels Song labels Confidence Weighting

Berenzweig - Music Recommendation11 Results

Berenzweig - Music Recommendation12 The Album Effect Testing on different album than trained hurts performance by 30-40% relative. Is it production effects or style?

Berenzweig - Music Recommendation13 Future Work Album Effect: production or style? Better segmentation Further analysis of posteriograms –song structure: change detection, clustering –another level of classification? leads to...

Berenzweig - Music Recommendation14 Artist Classification Using Vocals Anchor Models Similarity Metrics

Berenzweig - Music Recommendation15 Anchor Models Dual Motivation: –Scalable artist classification –Induced artist similarity metric Technique from speaker identification literature (Reynolds, Sturim & al.)

Berenzweig - Music Recommendation16 Anchor Models

Berenzweig - Music Recommendation17 Anchor Space n-dimensional Euclidean space Distance metric is simple Dimensions have meaning

Berenzweig - Music Recommendation18 Anchor Models Basically doing dimensionality reduction or feature extraction, where –nonlinear mapping to low-D feature space is learned –mapping is musically relevant –but no theoretical justification like PCA

Berenzweig - Music Recommendation19 Anchor Space Artists are distributions, not points. –Model with GMMs –Each frame of audio (32 milliseconds) is a point. Each song is a cloud, too. Distance is KL-divergence –estimate with total likelihood under GMM.

Berenzweig - Music Recommendation20 Artist Classification Using Vocals Anchor Models Similarity Metrics

Berenzweig - Music Recommendation21 Searching for Ground Truth Does a single “correct” similarity metric exist? –Subjective, relative, mood-dependent. Aspects of similarity - Tversky ‘77 –(Psychological) similarity is not a metric. –A “dynamic interplay between classification & similarity”

Berenzweig - Music Recommendation22 Similarity is not a metric? No Triangle Inequality Asymmetry An ellipse is like a circle. A circle is like an ellipse.

Berenzweig - Music Recommendation23 Salient Aspects Distance in big Euclidean space may not have any meaning! Goal: find big Euclidean space, then analyze salient dimensions of collections. Directly answers the question: what is it I like about the music that I like?

Berenzweig - Music Recommendation24 Searching for Ground Truth Sources of Opinion –Ask directly? –Preference Data: Spidering opennap lists. –Expert Opinion: Allmusic Guide “Similar Artist” sections. –Semantic Similarity: Whitman & Lawrence

Berenzweig - Music Recommendation25 Semantic Similarity “Community Metadata”. (Whitman and Lawrence) Web spider collects terms. Treats artists like documents

Expert Opinion

Berenzweig - Music Recommendation27 Completing the Erdos Numbers Incomplete Graph Complete Erdos Distance

Berenzweig - Music Recommendation28 Human Evaluation Want many judgements, but full matrix not likely Problem of relativity, drift –Ask for relative judgements –Game and Survey mode Problem of unknown artists –Use total history

Musicseer

Berenzweig - Music Recommendation30 Evaluation: Ranking

Berenzweig - Music Recommendation31 Thanks!