MSc Project Musical Instrument Identification System MIIS Xiang LI ee05m216 Supervisor: Mark Plumbley

Motivation of MIIS Musical instrument identification plays an important role in musical signal indexing and database retrieval. People can search music by the musical instruments instead of the type or the author For instance, user is able to query ‘find piano solo parts of a musical database’.

Introduction Bass Drum Piano Saxophone Identification results Musical Mixtures Musical instruments

Structure of MIIS Functional Components DUET algorithm: Separate the input musical mixture into sources Feature Extraction: Extract features of each source Classification: Implement classifier on testing source and find out the class it belongs to Input Mixture X(n) DUET algorithm Separation Estimated Sources Feature Extraction Classification Results

DUET algorithm Time-Frequency representation: and are representations in time-frequency domain, i.e. Short-time Fourier Transform, Modified Cosine Discrete Transform. Mixing parameters computation: Time-frequency points are labeled with Mask construction: Mask equals deciding set,which could be achieved by grouping the time-frequency point with the same label Source estimation is the time-frequency representation of one source. Time-domain conversion Convert each to in time domain

Feature Extraction Mel-Frequency Cepstral Coefficient (MFCC) Relationship between Mel and Hertz Spectral Rolloff It is calculated by summing up the power spectrum samples until the desired percentage (threshold) of the total energy is reached. Bandwidth Defined as the width of the range of frequencies that the signal occupies. Root Mean Square RMS features are used to detect boundaries among musical instruments Spectral Centroid Correlates strongly with the subjective qualities of “brightness” or “sharpness”. Zero Crossing Rate A simple measure of the frequency content of a signal

Classification K-Nearest Neighbor Nonparametric classifier Large storage required X Class a Class b Class c y x

Experiments Musical Instruments Database Database : Downloaded from University of Iowa website. Mixtures are composed by isolated notes. Training set: Includes 18 classes musical instruments Testing set: Choose 3 to 5 instruments to generate mixtures The instruments to be tested: Alto Saxophone Bassoon Double Bass Flute Viola

Experiments of three groups Group 1Group 2Group 3 No. of Sources 345 Percentage correct 80%60%48% For each group, five mixtures are tested and the result of each group is listed as follows:

Example SourceSDROriginal Source Estimated Source Result AltoSaxophone.C4B correct Bassoon.C3B correct Double Bass.D2B correct Estimated Sources Original Sources

Results discussion Without MISS, the recognisation percentage of each source in 18 classes is 1/18 which is about 5.5%. The worst case in our experiments is group 3 where each mixture consists five sources. The percentage is 48%. The less sources mixtures have, the higher percentage system performs. More sources introduce more interferences among each other.

Conclusion MISS is a system able to identify each musical instrument in a musical mixture. Three functional components are introduced:  DUET algorithm  Feature Extraction  Classification Experiments of three groups, which is fifteen mixtures in total have been tested. Correct percentages are 80%,60%and 48% respectively. More features could be extracted such as features of MPEG7 A more adaptive mask could help overcoming interferences among sources.