1. BASS TRACK SELECTION IN MIDI FILES AND MULTIMODAL IMPLICATIONS TO MELODY gPRAI Pattern Recognition and Artificial Intelligence Group Computer Music Laboratory Spain Octavio Vicente & Jose M. Iñesta Description and Retrieval of Music and Sound Information Descripción y Recuperación de Información Musical y Sonora PROJECT

2. introduction CONTEXT DOMAIN: Multimedia content management Content-based music information retrieval DATA: Symbolic music container files (digital scores) Multi-track MIDI files Organized by instruments or parts in tracks Some tracks have particular useful information

3. WHAT KIND OF USEFUL INFORMATIONS? MELODY TRACK – Melody is what we use to remember of a song – Music repository indexing (music thumbnails) – Fingerprinting – Music similarity and retrieval BASS TRACK – Bass is an important feature in music structure – Harmonic analysis – Rhythm analysisintroduction

4. THE PROBLEM: Bass track selection in multi-track MIDI files using our background in melody track selectionintroduction (D. Rizo et al. “A Pattern Recognition Approach for Melody Track Selection in MIDI Files”. ISMIR 2006)

5. WHAT CAN WE A PRIORI EXPECT: MELODY TRACK – The concept of melody is somehow elusive: Something singable Something catchy in a song A monophonic part easy to remember BASS TRACK – Seems easier at first: Low pitches involved Monophonic, melodic, etc. No a priori assumtions about instrumentationintroduction

6. using max and min value for all the tracks in the file About its content About the track track description Statistical features are extracted from every track A feature vector represents each track Most descriptors include normalized versions

7. Average polyphony No Yes Is it a bass track? track description No Yes Lowest pitchNormalized no. notes Some descriptors prove to be useful: The combination of these hints will permit us to assign each track a probability of being a bass track. No Yes

8. track description The tool for giving the probability is a Random Forest Classifier (RFC) due to their ability for making their own feature selection –Using K trees, each T j gives its decision d j on t –then –where (Breiman, L. (2001). “Random forests”. Machine learning, 45(1): 5–32 ) (“purity”) ratio between the number of samples of the winning class for the decision leaf

9. experimental setup Three data sets used (200 files each): –CL200: classical music –JZ200: jazz –KR200: pop-rock (karaoke) Number of bass tracks per file: Number of bass and non-bass tracks in the MIDI datasets: The system should say NO-TRACK The system should select it The system should select any of them

10. MIDI files MIDI files Bass tags dictionary Bass tags dictionary Tag compilation and selection Tag compilation and selection Track labels Bass labels Bass tags Bass tracks Bass tracks Genre tags Genre tags Classifier experimental setup Jazz Classical Pop-rock RFCs Classical Jazz Pop-rock

11. Experiment 1 Bass versus non-bass classification: given a particular track, is it a bass one? ?

12. Experiment 1 Bass versus non-bass classification: given a particular track, is it a bass one?

13. Experiment 2 Bass track selection: given a file, which track contains the bass part? ? None 1 2 3 4 … N Notation: For solving the no bass track situation: 0

14. Experiment 2 Bass track selection: given a file, which track contains the bass part? In addition to accuracy, other evaluations are computed: FP : the classifier selects a non-bass track TP : the selected track contains the correct bass line FN : no track selected but the MIDI file indeed contains at least one bass track

15. Experiment 2 Bass track selection: given a file, which track contains the bass part?

16. Experiment 2’ Bass track selection across styles: style specificities of the bass part The test style files were not used for training Piano left hand issue (82.8%)

17. Experiment 3 A question of multimodal nature arises: Can we use the bass track information for improving melody track detection? Melody tracks classification is based on the corresponding estimated from the provided by the random forest using the melody tagged data, using Constraint: A first naïve approach could be 1st estimate and remove that track for selection. PRO: it simplifies the problem (less tracks) CON: no new information is provided

18. Experiment 3 The proposed approach: instead of looking at, let’s consider the probabilities of being a melody conditioned also by the knowledge of how a bass looks like and the different-track constraint If we also assume that bass and melody tracks are not mutually conditioned, we reach to and

19. Experiment 3 Multimodal bass track selection: Multimodal melody track selection: Results:

20. Conclusions Global statistical features and RFC have proven to be useful for other kind of tracks other than melody. In fact, it works better (+24.4 %) for bass than for melody (seems to be easier). Bass track characterization depends on the music genre. Using bass information improved significantly the melody track selection. The improvement was lower when melody was used to select bass tracks.

21. and future works Generalization studies are needed – conditioned by the long and tedious work of tagging and checking the ground truth in hundreds of MIDIs Natural extension to other tracks: – instrument-based: piano, for example, but any; – role-based: solos, intros, etc. – Study *multi*modal interactions among them

22. BASS TRACK SELECTION IN MIDI FILES AND MULTIMODAL IMPLICATIONS TO MELODY Octavio Vicente & Jose M. Iñesta gPRAI Pattern Recognition and Artificial Intelligence Group Computer Music Laboratory Spain Description and Retrieval of Music and Sound Information Descripción y Recuperación de Información Musical y Sonora PROJECT