Onset Detection in Audio Music J.-S Roger Jang ( 張智星 ) MIR LabMIR Lab, CSIE Dept. National Taiwan University

-2- What Are Note Onsets?  Energy profile of a percussive instrument is modeled as ADSR stages  Note onset is the time where the slope is the highest, during the attack time.  Soft onsets via violin, etc, are much harder to define and detect.

-3- Difficulty in Onset Detection  Music types  Monophonic  Easier  Polyphonic  Harder  Instrument types  Percussive instruments  Easier  String instruments  Harder (soft onsets)

-4- Why Onset Detection is Useful?  It is a basic step in music analysis  Music transcription (from wave to midi)  Music editing (Song segmentation)  Tempo estimation  Beat tracking  Musical fingerprinting (the onset trace can serve as a robust id for fingerprinting)

-5- Onset Detection Function  ODF (onset detection function) creates a curve of onset strength, aka  Onset strength curve  Novelty curve  Most ODFs are based on time-frequency representation (spectrogram) of  Magnitude of STFT (Short-time Fourier transform)  Phase of STFT  Mel-band of STFT  Constant-Q transform

-6- ODF: Spectral Flux  Concept  sum the positive change in each frequency bin

-7- Flowchart of OSC  Steps of OSC  Spectrogram  Mel-band spectrogram  Spectral flux  Smoothed OSC via Gaussian smoothing  Trend of OSC via Gaussian smoothing  Trend-subtracted OSC  Check out wave2osc.m to see these steps.

-8- Example of OSC  Try “wave2osc.m”

-9- What Can You Do With OSC...  OSC  onsets  Pick peaks to have onsets  OSC  tempo (BPM, beats per minute)  Apply ACF (or other PDF) to find the BPM  OSC  beat tracking  Pick equal-spaced peaks to have beat positions

-10- Beat Tracking  Demos   Try “beatTracking.m” in SAP toolbox

-11- Example of Beat Tracking  beatTracking.m

-12- Performance Indices of Beat Tracking  Many performance indices of BT  Check out audio beat tracking task of MIREX  Mostly adopted ones  Precision, recall, f- measure, accuracy  Try simSequence.m in SAP toolbox Precision = tp/(tp+fp)=3/(3+3) = 0.5 Recall = tp/(tp+fn)=3/(3+2) = 0.6 F-measure = tp/(tp+(fn+fp)/2)=3/(3+(2+3)/2) = Accuracy = tp/(tp+fn+fp)=3/(3+2+3) = 0.375