1. Onset Detection, Tempo Estimation, and Beat Tracking
J.-S Roger Jang (張智星)
MIR Lab, CSIE Dept.
National Taiwan University

2. What Are Onsets?
Music instrument event is usually divided into 4 stages of ADSR (attack, decay, sustain, release) based on its energy profile.
Onset is the time when the slope is the highest, during the attack time.
Quiz!

3. Characteristics of Onset Detection
Music types
Monophonic (single sound source)  Easier
Polyphonic (multiple sound sources)  Harder
Instrument types
Percussive instruments  hard onsets which are easier to detect
String instruments  Soft onsets which are harder to detect

4. Why Onset Detection is Important?
It is a fundamental step in audio music analysis
Music transcription (from wave to midi)
Music editing (Song segmentation)
Tempo estimation and beat tracking
Musical fingerprinting (the onset trace can serve as a robust feature for fingerprinting)

5. Onset Detection Function
ODF (onset detection function) creates a curve of onset strength, aka
Onset strength curve (OSC)
Novelty curve
Most ODFs are based on time-frequency representation
Magnitude of STFT  spectrogram
Phase of STFT
Mel-band of STFT
Constant-Q transform
Short-time
Fourier transform

6. Spectrogram
wave2specGram.m
X(n, k)

7. ODF: Spectral Flux Concept
Sum the positive change in each frequency bin
Quiz!

8. Flowchart of OSC Steps of OSC Check out wave2osc.m to see these steps.
Spectrogram
Mel-band spectrogram (optional)
Spectral flux
Smoothed OSC via Gaussian smoothing
Trend of OSC via Gaussian smoothing
Trend-subtracted OSC
Check out wave2osc.m to see these steps.

9. Mel-freq Spectrogram 40 filters in Mel-freq filter bank Spectrograms
Linear freq
spec1
Mel freq
spec2
Matrix M
spec2=M*spec1
melBinPlot.m

10. Mel-freq Representation
About mel-freq spectrogram
Advantage: More correlated to human perception (just like MFCC in speech recognition)
The degree of effectiveness is yet to be verified

11. rawOsc=mean(max(0, diff(magSpec, 1, 2)));
Spectral Flux
spec2oscPlot.m
magSpec
rawOsc
rawOsc=mean(max(0, diff(magSpec, 1, 2)));
Order 1
Order 1
Dim 2

12. Gaussian Smoothing
s is small  Smoothed OSC
s is large  Trend of OSC

13. Example of OSC
wave2osc.m

14. What Can You Do With OSC... OSC  onsets
Pick peaks to have onsets
OSC  tempo (BPM, beats per minute)
Apply ACF (or other PDF) to find the BPM
OSC  beat tracking
Pick equal-spaced peaks to have beat positions
Quiz!

15. Example of Beat Tracking
beatTrack.m
Identified
beat pos.
Max BPM=250
Min BPM=50
Tempo estimation
via ACF
8 candidate sets for beat positions

16. Demo of Beat Tracking Demo link Examples
Examples
陪我看日出 分享

17. Performance Indices in Information Retrieval
Figure source:
Desired
Retrieved
Quiz!
References
F1-score
Precision & recall
Information retrieval

18. Performance Indices of Beat Tracking
Many performance indices of BT
Check out audio beat tracking task of MIREX
Mostly adopted ones
Precision, recall, f-measure, accuracy
Try simSequence.m in SAP toolbox
Precision = tp/(tp+fp)=3/(3+4) =
Recall = tp/(tp+fn)=3/(3+2) =
F-measure = tp/(tp+(fn+fp)/2)=3/(3+(2+4)/2) =
Accuracy = tp/(tp+fn+fp)=3/(3+2+4) =
Quiz!