1. Pitch Tracking (音高追蹤) Jyh-Shing Roger Jang (張智星) MIR Lab (多媒體資訊檢索實驗室)
CS, NTHU (清華大學 資訊工程系)

2. Pitch (音高） Definition of pitch Characteristics of pitch
Fundamental frequency (FF, in Hz): Reciprocal of the fundamental period in a quasi-periodic waveform
Pitch (in semitone): Obtained from the fundamental frequency through a log-based transformation (to be detailed later)
Characteristics of pitch
Noise and unvoiced sound do not have pitch.

3. Pitch Tracking (音高追蹤)
Pitch tracking: To compute the pitch vector of a give waveform (對整段音訊求取音高)
Applications
Query by singing/humming (哼唱選歌)
Tone recognition for Mandarin (華語的音調辨識)
Intonation scoring for English (英語的音調評分)
Prosody analysis for speech synthesis (語音合成中的韻律分析)
Pitch scaling and duration modification (音高調節與長度改變)

4. Pitch Tracking Algorithms
Two categories for pitch tracking algorithms
Time domain (時域)
ACF (Autocorrelation function)
AMDF (Average magnitude difference function)
SIFT (Simple inverse filtering tracking)
Frequency domain (頻域)
Harmonic product spectrum method
Cepstrum method

5. Typical Steps for Pitch Tracking
Chop signals into frames (aka frame blocking)
Compute pitch functions (ACF, AMDF, etc.)
Determine pitch for a frame
Max/min picking of the pitch function
Remove unreliable pitch
Via volume/clarity thresholding
Smooth the whole pitch vector
Via median filter, etc.

6. Frame Blocking Zoom in Frame size=256 points Overlap=84 points
Frame rate = fs/(frameSize-overlap)
= 11025/(256-84)=64 pitch/sec

7. ACF: Auto-correlation Function1
128
Frame s(i):
Shifted frame s(i+t):
t=30
acf(30) = inner product of overlap part
Pitch period 30

8. ACF Example 1 sunday.wav Fundamental frequency Sample rate = 16kHz
Frame size = 512 (starting from point 9000)
Fundamental frequency
Max of ACF occurs at index 132
FF = 16000/(132-1) = Hz

9. ACF Example 2
If the range of humans’ FF is [40, 1000], then we have the restriction for selecting pitch point:
Min FF=40Hz  acf(fs/40:end) is not considered.
Max FF=1000Hz  acf(1:fs/1000) is not considered.

10. Pitch Tracking via ACF Specs Playback Sampe rate = 11025 Hz
Frame size = 353 points = 32 ms
Overlap = 0
Frame rate = f/s
Playback
soo.wav
sooPitch.wav

11. Variations of ACF to Avoid Tapering
Normalized version
Half-frame shifting:

12. Variations of ACF to Normalize Range
To normalize ACF to the range [-1 1]:
This is based on the inequality:

13. AMDF: Average Magnitude Difference Function1
128
Frame s(i):
Shifted frame s(i+t):
t=30
amdf(30) = sum of abs. difference
Pitch period 30

14. AMDF Example sunday.wav Fundamental frequency Sample rate = 16kHz
Frame size = 512 (starting from point 9000)
Fundamental frequency
Min of AMDF occurs at index 132
FF = 16000/(132-1) = Hz

15. Variations of AMDF to Avoid Tapering
Normalized version
Half-frame shifting:

16. Combining ACF and AMDF
Frame
ACF
AMDF
ACF/AMDF

17. Example of Pitch Tracking

18. UPDUDP (1/4) UPDUDP: Unbroken Pitch Determination Using DP
Goal: To take pitch smoothness into consideration
: a given path in the AMDF matrix
: Number of frames
: Transition penalty
: Exponent of the transition difference

19. UPDUDP (2/4)
Optimum-value function D(i, j): the minimum cost starting from frame 1 to position (i, j)
Recurrent formula:
Initial conditions :
Optimum cost :

20. UPDUDP (3/4)
A typical example

21. UPDUDP (4/4)
Insensitivity in

22. Harmonic Product Spectrum
hps.m

23. Frequency to Semitone Conversion
Semitone : A music scale based on A440
Reasonable pitch range:
E2 - C6
82 Hz Hz ( )

24. Unreliable Pitch Removal
Pitch removal via volume thresholding
Plot by self demo of ptByPf.m

25. Unreliable Pitch Removal
Pitch removal via volume/clarity thresholding
Plot by self demo of ptByPf.m

26. Rest Handling
With rests
Without rests

27. Rest Handling Original pitch vectors with rests.
Rests are replaced by previous nonzero pitch. Good for LS.
Rests are removed. Good for DTW.

28. Typical Result of Pitch Tracking
Pitch tracking via autocorrelation for茉莉花 (jasmine)

29. Comparison of Pitch Vectors
Yellow line : Target pitch vector

30. Demo of Pitch Tracking Real-time display of ACF for pitch tracking
toolbox/sap/goPtByAcf.mdl
Real-time pitch tracking for real-time mic input
toolbox/sap/goPtByAcf2.mdl
Pitch scaling
pitchShiftDemo/project1.exe
pitchShift-multirate/multirate.m
Intonation assessment
ap170/matlab/goDemo.m