1. Berenzweig - Music Recommendation1 Music Recommendation Systems: A Progress Report Adam Berenzweig April 19, 2002

2. Berenzweig - Music Recommendation2 Music Recommendation Is: Music IR for the masses –Kids in candy stores –Querying is hard; people can’t describe music –Recommendation can be integrated into players, streaming services, music stores, etc. –Break major label/retail monopoly on choice!!

3. Berenzweig - Music Recommendation3 Music Recommendation Is: Set-based IR –“Find me items similar to this set, in the way that the set is similar to itself” –Set = collection, or playlist extension. –Be sensitive to themes or aspects of the user’s collection. All about similarity

4. Berenzweig - Music Recommendation4 Background I: IR/Statistics Collaborative Filtering Latent Semantic Analysis (Deerwester & al., ‘90) –SVD to find hidden meaning Probabilistic LSA (Hofmann, ‘99) –EM to find hidden meaning Latent Class Models (Hofmann, ‘99)

5. Berenzweig - Music Recommendation5 Background II: Audio IR Artist classification –Whitman & Lawrence; Berenzweig, Ellis & Lawrence Genre classification –Tzanetakis Fingerprinting, query-by-example What features??? –What is it I like about the music that I like?

6. Berenzweig - Music Recommendation6 Artist Classification Using Vocals Anchor Models Similarity Metrics

7. Berenzweig - Music Recommendation7 Artist Classification Using Vocals Are vocal segments more easy to identify than instrumental segments? –“Using Voice Segments to Improve Artist Classification of Music”, Berenzweig, Ellis & Lawrence, to appear AES 22.

8. Berenzweig - Music Recommendation8 Segmented Posteriograms

9. Berenzweig - Music Recommendation9 Segmented Posteriograms

10. Berenzweig - Music Recommendation10 Experiment at-a-glance Audio input Cepstra (MFCC, PLP) Artist Classifier Vox/Music Classifier Frame labels Song labels Confidence Weighting

11. Berenzweig - Music Recommendation11 Results

12. Berenzweig - Music Recommendation12 The Album Effect Testing on different album than trained hurts performance by 30-40% relative. Is it production effects or style?

13. Berenzweig - Music Recommendation13 Future Work Album Effect: production or style? Better segmentation Further analysis of posteriograms –song structure: change detection, clustering –another level of classification? leads to...

14. Berenzweig - Music Recommendation14 Artist Classification Using Vocals Anchor Models Similarity Metrics

15. Berenzweig - Music Recommendation15 Anchor Models Dual Motivation: –Scalable artist classification –Induced artist similarity metric Technique from speaker identification literature (Reynolds, Sturim & al.)

16. Berenzweig - Music Recommendation16 Anchor Models

17. Berenzweig - Music Recommendation17 Anchor Space n-dimensional Euclidean space Distance metric is simple Dimensions have meaning

18. Berenzweig - Music Recommendation18 Anchor Models Basically doing dimensionality reduction or feature extraction, where –nonlinear mapping to low-D feature space is learned –mapping is musically relevant –but no theoretical justification like PCA

19. Berenzweig - Music Recommendation19 Anchor Space Artists are distributions, not points. –Model with GMMs –Each frame of audio (32 milliseconds) is a point. Each song is a cloud, too. Distance is KL-divergence –estimate with total likelihood under GMM.

20. Berenzweig - Music Recommendation20 Artist Classification Using Vocals Anchor Models Similarity Metrics

21. Berenzweig - Music Recommendation21 Searching for Ground Truth Does a single “correct” similarity metric exist? –Subjective, relative, mood-dependent. Aspects of similarity - Tversky ‘77 –(Psychological) similarity is not a metric. –A “dynamic interplay between classification & similarity”

22. Berenzweig - Music Recommendation22 Similarity is not a metric? No Triangle Inequality Asymmetry An ellipse is like a circle. A circle is like an ellipse.

23. Berenzweig - Music Recommendation23 Salient Aspects Distance in big Euclidean space may not have any meaning! Goal: find big Euclidean space, then analyze salient dimensions of collections. Directly answers the question: what is it I like about the music that I like?

24. Berenzweig - Music Recommendation24 Searching for Ground Truth Sources of Opinion –Ask directly? –Preference Data: Spidering opennap lists. –Expert Opinion: Allmusic Guide “Similar Artist” sections. –Semantic Similarity: Whitman & Lawrence

25. Berenzweig - Music Recommendation25 Semantic Similarity “Community Metadata”. (Whitman and Lawrence) Web spider collects terms. Treats artists like documents

26. Expert Opinion

27. Berenzweig - Music Recommendation27 Completing the Erdos Numbers Incomplete Graph Complete Erdos Distance

28. Berenzweig - Music Recommendation28 Human Evaluation Want many judgements, but full matrix not likely Problem of relativity, drift –Ask for relative judgements –Game and Survey mode Problem of unknown artists –Use total history

29. Musicseer

30. Berenzweig - Music Recommendation30 Evaluation: Ranking

31. Berenzweig - Music Recommendation31 Thanks!