Effects in frequency domain Stefania Serafin Music Informatics Fall 2004

Overview Time versus frequency domain effects The phase vocoder The STFT Time stretching Pitch shifting Cross synthesis

Overview Until now we examined effects based on delays and working in time domain. What happens if we want to work in the frequency domain? In this case we talk about SPECTRAL MODELS and SPECTRAL TRANSFORMATIONS

The phase vocoder A phase vocoder represents an ensemble of techniques which take a sound in the time domain,calculate the Fourier transform, perform some manipulations in frequency domain and then reconstruct the sound in time domain.

Analysis/synthesis steps ANALYSIS TRANSFORMATIONS SYNTHESIS INPUT OUTPUT

Analysis synthesis steps

Analysis: STFT

STFT Time-domain audio signal is separated into successive short frames. Each frame is multiplied by a window function to smooth transitions. Resulting windows are passed on to FFT module. The FFT process then extracts spectral information on the signal.

Spectral transformations Pitch shifting Time stretching Morphing (Cross-synthesis)

Pitch shifting How would you perform a pitch shift in time domain? What are the drawbacks of doing it?

Solution: move to frequency domain In frequency domain 1) Calculate the spectrum 2) Shift it upwards or downwards 3) Go back to time domain

Time stretching Time-scaling is an application that allows signal length to be stretched or shortened without affecting the frequencies of its components. 1) Calculate spectrum 2)Reduce/increase time information 3) Go back to time domain

Morphing It is the combination of two signals In frequency domain: 1) Calculate spectrum of signal A 2) Calculate spectrum of signal B 3) C = A B

STFT resynthesis STFT resynthesis is the final stage of the phase vocoder process. It is based on recreating a signal in time domain from the modified spectral components of the input signal.

The third stage implements an exact inverse of the process that was used in STFT analysis of the signal. Modified spectral components of the input signal are passed through the Inverse Fourier Transform to recreate a set of time-domain frames.

Application Farinelli

Application (ctn.) Karaoke

In Max/MSP Check the examples Folder. Convolution and Phase vocoder example.