Brian Whitman Paris Smaragdis MIT Media Lab Combining Musical and Cultural Features for Intelligent Style Detection Brian Whitman Paris Smaragdis MIT Media Lab

Background Music classification by style A “human” concept; hard to model. Defines subclasses of genres. Can be utilized by recommendation engine for high-confidence results. 11/6/2018 ISE599 - by Frances Kao

Approach An automatic style detection system that operate on both of acoustic content of the audio community metadata: a vector space of descriptive textual terms crawled from the web Dataset: 5 styles, each with 5 different artists 11/6/2018 ISE599 - by Frances Kao

Audio-based Classification Form each song into some presentation Train a neural network to classify a song Representation: randomly choose 12 songs of each artist -> downsampling -> extract Power Spectral Density (PSD) -> use Principal Components Analysis (PCA) to reduce dimension -> representation of each artist. Feedforward time-delay neural network 11/6/2018 ISE599 - by Frances Kao

Audio-based Classification – Result Heavy Metal Contemporary Country Hardcore Rap Intelligent Dance Music R&B Fail to overcome intra-style auditory inconsistency. Particularly not good for IDM. Since this style is with huge auditory variance. 11/6/2018 ISE599 - by Frances Kao

Community Metadata-based Classification (1) Cultural feature Each artist is associated with terms which appear on the same web document as the artists’ name. Each term has a score calculated in terms of position and frequency of occurrence. 11/6/2018 ISE599 - by Frances Kao

Community Metadata-based Classification (2) Similarity For every 2 artists, calculate an overlap weight, which is the summation of every shared term. Form a similarity matrix to predict the style of each artist 11/6/2018 ISE599 - by Frances Kao

Community Metadata-based Classification - Result Heavy Metal Contemporary Country Hardcore Rap Intelligent Dance Music R&B Performed somewhat not perfectly for 2 styles, Rap and R&B. 11/6/2018 ISE599 - by Frances Kao

Combined Classification Heavy Metal Contemporary Country Hardcore Rap Intelligent Dance Music R&B Posterior probability, and average value 11/6/2018 ISE599 - by Frances Kao

Conclusion & Future Work Combined classification can overcome all the problems Future development can use a “culture ratio” to alert recommendation engines to use which classification method. 11/6/2018 ISE599 - by Frances Kao