2. Content-based Music Retrieval from Acoustic Input (CBMR)

3. -2--2- Outline zWhat is CBMR? zMethods ySignal processing ySimilarity comparison zExperiment results zDemo zFuture work

4. -3--3- What is CBMR? zCBMR : yContent-based Music Retrieval zTraditional database query : yText-based or SQL-based zOur goal : yMusic retrieval by singing/humming

5. -4--4- Related Work zQuery by humming by Ghias,Loga and Chamberlin in 1995 yAutocorrelation pitch detection y183 songs in database zMELDEX system by New Zealand Digital Library Project in 1996 yGold/Rabiner Algorithm (800 songs) ySing ‘la’ or ‘ta’ when transposition zKaraoke song recognizer by J.F. Wang in 1997 yNovel pitch detection y50 songs in database

6. -5--5- Flowchart Post Signal Processing Pitch Tracking Microphone Signal Input Filtering Query Results (Ranked Song List) Similarity Comparison Off-line processing Midi message Extraction Songs Database Sampling 11KHz Mid-level Representation On-line processing

7. -6--6- Original Wave Input 小雨中的回憶 11025 Hz 8 Bits Mono

8. -7--7- Single Frame 512 points/frame 340 points overlap Zoom in Overlap Frame

9. -8--8- Pitch Tracking zRange yE2 - C6 y82 Hz - 1047 Hz ( - ) zMethod yAuto-correlation y

10. -9--9- Auto-correlation without Clipping

11. -10- Center Clipping (a)(b)(c) 000 Clipping limits are set to  % of the absolute maximum of the auto-correlation data

12. -11- Auto-correlation with Clipping

13. -12- Pitch Contour

14. -13- Signal Process zRemove violent point & short notes zDown sampling & smoothing zFrequency to semitone ySemitone : A music scale based on A440 y

15. -14- Pitch Contour (After Smoothing)

16. -15- Mid-level Representation

17. -16- Mid-level Representation without Rest

18. -17- Similarity Comparison zGoal yFind the most similar Midi file zChallenge yTempo variance xDynamic time warping (DTW) yTune variance xKey transposition

19. -18- Compare by DTW Wave File Mid File DTW

20. -19- Dynamic Time Warping (DTW) i j t(i-1) t(i) r(j) r(j-1) window

21. -20- DTW (cont.) i j dist(i,j) = |t(i)-r(j)| if ( t(i) = Rest && r(j) = Rest ) dist(i,j) = 0; elseif ( t(i) = Rest || r(j) = Rest) dist(i,j) = restWeight;

22. -21- Example of DTW

23. -22- Key Transposition zMean sift zBinary search in the searching area yO( N) --> O (log N) Mean Searching Area

24. -23- Example of Key Transposition

25. -24- Score Function z ym : length of match string yn : length of input string ye : DTW distance yA = 0.8 yB = 0.6

26. -25- Experiment Environment z290 wave files yWave length : 5 - 8 sec yWave format : PCM, 11025Hz, 8bits, Mono zEnvironment yCeleron 450 with 128Mb RAM under Matlab 5.3 zDatabase y493 midi files

27. -26- Experiment Result (Histogram)

28. -27- Experiment Result (Pie) Total time : 4589 sec (15.8 sec/per-wave)

29. -28- Experiment Result (Pie) - With Rest Total time : 7893 sec (27.2 sec/per-wave)

30. -29- How to Accelerate? zBranch and bound yO(N) -> O(lnN) yTriangle inequality xd(a,b) + d(b,c) ≧ d(a,c) zHierarchical y2 phase x3/32 sec x2/32 sec

31. -30- Experiment Result (Pie) - 3/32 sec Total time : 2358 sec (8.9 sec/per-wave)

32. -31- Experiment Result (Pie) - 2 Phase Total time : 3006 sec (11.2 sec/per-wave)

33. -32- Error Analysis zMidi error zSinging error zLow pitch zBroken vocalism zNoise

34. -33- Future Work zTime consuming yBetter similarity comparison yDifferent comparison unit yHardware acceleration yBetter searching algorithm zSteadier pitch tracking algorithm zNoise handle