2. State.of.the.art June 2008 Pedro J. Ponce de León

3. Index Melody Characterization by a Genetic Fuzzy System Doing it right Music genre dynamic classification Pre-filtering test data Classifier combination: (SSPR-2006): A Java implementation

4. Melody Characterization by a Genetic Fuzzy System Motivation To describe with readable linguistic labels what a melody is And learn it automatically from a dataset

5. Related previous works Ponce de León et. al. [ISMIR’07]: melody characterization by automatically induced rules AvgPitch>=65 and TrackOccupationRate>=0.51 and ….

6. Track description Set of statistics on properties of note streams: CategoryDescriptors Track info Normalized duration, Number of notes, Occupation rate, Polyphony rate Pitch Highest, Lowest, Mean, Standard deviation Pitch intervals Number of distinct intervals, Largest, Smallest, Mean, Mode, Standard deviation Note durations Longest, Shortest, Mean, Standard deviation Syncopation Number of syncopated notes Class Melody, accompaniment

7. From crisp rules to fuzzy rules Data representation (descriptors) Crisp descriptor (e.g. avgPitch)  fuzzy variable Fuzzy variable membership function parameters are optimized using a Genetic Fuzzy System Fuzzy Inference System (FIS) + Genetic Algorithm (GA) Rules Fuzzify antecedents: (x > a)  (x IS term 1 ) OR (x IS term 2 ) …

8. 3.2 Fuzzy set parameter optimization: encoding 1 0... r1r1 r2r2 r3r3 rnrn xyz p0p0 Gene = {p 0,r 1,r 2,r 3,…r n } Only fuzzy set support points considered for optimization the relative values of the vertices are encoded into genes

9. 3.2 Fuzzy set parameter optimization: encoding Fuzzy variables Chromosome

10. 3.2 GA fitness function Chromosome Build fuzzy sets from chromosome Fitness measure: test quality of fuzzy rules system with training corpus: Two possible quality measurements: a) Number of hits: num. of tracks correctly classified by the FIS b) F-measure for class ‘IsMelody=true’

11. 3.3 Crisp Rule fuzzification Fuzzify antecedents: keep the sets whose support holds the inequality CrispFuzzy A B CDA B CD A BCD (x < v)  (x IS C) OR (x IS B) OR (x IS A) (x > v)  (x IS C) OR (x IS D) v x SMC08: ad-hoc rule fuzzification Now: Rule fuzzification based on evolved fuzzy sets (for fitness computation)

12. Experiments and results

13. FIS optimization parameters Experiment parameterValues GA population size100,500,1000 GA no. of generations100,500,1000 GA mutation rationone, 0.05, 0.1 GA selection strategyBest one, Best 10%, Best 20% GA fitness metricHit count, F-measure Defuzzification threshold0.5,0.6, 0.7

14. Fitness evolution (F-measure fitness)

15. Fitness evolution (F-measure fitness : 1000 generations)

16. Fitness evolution Similar results for Hit-based fitness… So reduce number of generations: max. 300

17. FIS optimization parameters Experiment parameterValues GA population size100,200,300 (updated) GA no. of generations100,200,300 (updated) GA mutation rationone, 0.05, 0.1 GA selection strategyBest one, Best 10%, Best 20% GA fitness metricHit count, F-measure Defuzzification threshold0.5,0.6, 0.7 3 5 *2 = 486 experiments!

18. Ortogonal arrays for experiment design OA(rows,levels,strength) OA(18,3,2): 000000 111111 222222 001212 112020 220101 010221 121002 202110 022011 100122 211200 012102 120210 201021 021120 102201 210012 population=100 200 300 evolutions=100 200 300 fitness=Hits Fmeasure Fmeasure mutation=0 10 20 selector=Best Best0.1 Best0.2 threshold=0.5 0.6 0.7 + Distributed GA computation es.ua.dlsi.im.experiments.generation.ExperimentDesignGenerator es.ua.dlsi.im.experiments.generation.OrthogonalArrayCombinator

19. Ortogonal arrays for experiment design 000000 111111 222222 001212 112020 220101 010221 121002 202110 022011 100122 211200 012102 120210 201021 021120 102201 210012 population=100 200 300 evolutions=100 200 300 fitness=Hits Fmeasure Fmeasure mutation=0 10 20 selector=Best Best0.1 Best0.2 threshold=0.5 0.6 0.7 es.ua.dlsi.im.experiments.generation.ExperimentDesignGenerator usage: ExperimentDesignGenerator -commandPrefix Prefix for command line in generated script -file Experiment definition file -help Help -latex Output LaTeX file name -latex_with_header Output LaTeX table header -outputDir Output directory -outputFilePrefix Prefix for configuration files -outputScript Output script file name

20. Results DatasetPrecisionRecallFError rate SMALL (crisp)0.890.870.880.05 SMALL (best fuzzy) 0.630.910.750.11 SMALL (avg. fuzzy) 0.390.94 (!) 0.530.34 SMALL (avg. fuzzy fixed rules) 0.800.770.780.08 System parameters learnt with SMALL dataset Large recall -> A lot of FP: Many non-melody tracks fire a rule TODO: Investigate why

21. Some hints for large recall Unrealistic rules: FUZZIFY AvgAbsInterval TERM high := (90.6, 0.0) (108.6, 1.0) (121.7, 1.0) ; TERM fourth := (65.4, 0.0) (80.3, 1.0) (107.1, 1.0) (111.7, 0.0) ; TERM third := (26.6, 0.0) (32.0, 1.0) (66.2, 1.0) (88.0, 0.0) ; TERM second := (3.2, 0.0) (3.8, 1.0) (29.0, 1.0) (41.6, 0.0) ; TERM unison := (0.0, 0.0) (0.4, 1.0) (4.5, 1.0) (4.5, 0.0) ; END_FUZZIFY Some fuzzy variables have 5 terms, other 3 Range: [0..127] not realistic

22. Best evolved Fuzzy rule set so far: 1.(((AvgPitch is high or AvgPitch is veryHigh) and ((TrackOccupationRate is average or TrackOccupationRate is high) or TrackOccupationRate is full)) and (AvgAbsInterval is unison)) and (TrackNumNotes is average or TrackNumNotes is high) 2.(((AvgPitch is high or AvgPitch is veryHigh) and ((TrackOccupationRate is average or TrackOccupationRate is high) or TrackOccupationRate is full)) and (TrackPolyphonyRate is low or TrackPolyphonyRate is none)) and (NormalizedDistinctIntervals is highest) 3.((((AvgPitch is high or AvgPitch is veryHigh) and (TrackNumNotes is high)) and (LowestNormalizedDuration is shortest)) and ((ShortestDuration is low or ShortestDuration is average) or ShortestDuration is high)) and (NormalizedDistinctIntervals is highest) 4.…

23. Melody description example (I) Air In F, Watermusic, Handel (Baroque) Melody (AvgPitch IS NOT veryLow) AND (AvgAbsInterval IS NOT high) AND (AvgAbsInterval IS NOT third) AND (AvgAbsInterval IS NOT fourth) AND (TrackSyncopation IS alot) AND (TrackOccupationRate IS NOT low) AND (LowestNormalizedPitch IS NOT low) AND (DistinctIntervals IS alot) AND (TrackNormalizedDuration IS largest) Most fired rule:

24. Melody description example (II) There Is No Greater Love, I. Jones (pre-Bop Jazz) Melody (AvgPitch IS NOT veryLow) AND (AvgAbsInterval IS NOT high) AND (AvgAbsInterval IS NOT third) AND (AvgAbsInterval IS NOT fourth) AND (TrackSyncopation IS alot) AND (TrackOccupationRate IS NOT low) AND (LowestNormalizedPitch IS NOT low) AND (DistinctIntervals IS alot) AND (TrackNormalizedDuration IS largest) Most fired rule:

25. Music genre dynamic classification Pre-filtering test data Based on number of notes: The fewer notes in a fragment, the less robust its statistical description Proof-of-concept: jvc melodies for training jvc2 for test NN classifier

26. Music genre dynamic classification Pre-filtering test data on the number of notes per window w=4,8,16,32,64,960 <= threshold < 30

27. Music genre dynamic classification Preliminary investigation on the number of notes vs. window length suggest a ratio between 1.5 and 2 w81216326496 Non-filtered 0.2080.1260.1230.0850.0540.008 Filtered (min = 1.5*w) 0.2020.1230.1210.0840.0530.006 Filtered (min = 2*w) 0.1960.1190.1210.0840.0540.005 Accuracy Non-significant improvements

28. Classifier combination: a Java implementation es.ua.dlsi.im.pr.* Prediction prediction weight getPrediction() getWeight() isWeighted() PredictionCombiner PluralityVote() : Prediction SimpleWeightedVote() : Prediction RescaledWV() : Prediction BestWorstVW() : Prediction QuadraticBWVW() : Prediction WeightedMajority() : Prediction

29. Classifier combination: a Java implementation es.ua.dlsi.im.windows.* Instances DynamicCombiner getPredictions() : Prediction[] getPredictionByPluralityVote() : Prediction getPredictionBySimpleWeightedVote() : Prediction … Classifier