2. Sound Synthesis Part III: Distortion-based synthesis (FM & Waveshaping)

3. Plan Distortion-based synthesis –FM synthesis –Waveshaping Matlab code for synthesis Example of synthesized composition (Risset) Summary

4. Types of synthesis Sound Synthesis Additive synthesis Distortion techniques Subtractive synthesis Granular synthesis Analysis based Physical modelling

5. Distortion (aka nonlinear) Synthesis Additive synthesis is powerful & flexible BUT requires many simple oscillators. Plethoric and non-intuitive controls on the sound spectrum.  Using fewer oscillators, with more components.  Complex spectra can be obtained by distorting simple oscillators.

6. Plan Distortion-based synthesis –FM synthesis –Waveshaping Substractive synthesis Summary

7. FM Synthesis Pioneered by John Chowning (1973) “Alteration of distortion of the frequency of an oscillator in accordance with the amplitude of a modulating signal.” Similar to Vibrato modulation, but using audible modulating frequency and larger width.  Generate a large number of timbres easily controlled.

8. FM Synthesis (2) Modulated frequency in fc-d <= fv <= fc+d Peak frequency deviation d (in Hz) Vibrato: small d & fm FM Synth: larger d & fm Note: if d>fc, then fv can be < 0 (negative phase applied!)  Rich spectrum from sine waves only! fc fm A + d fv Modulating oscillator Carrier oscillator

9. FM Synthesis (3) Spectrum: fc + k.fm, for k an integer. Sidebands increase with d Modulation index I=d/fm Spectrum amplitude: –Fc+k.fm  Jk(I) –where Jk is the kth Bessel function –Sidebands significant ~ for |k|<I+1 (rule of thumb) Bessel Functions J1(X), J2(X) and J3(X) fc-fm fc+fm fc Magnitude Frequency fc+2.fm fc-2.fmfc-3.fm fc+3.fm

10. FM Synthesis (4) Bessel functions can be negative! Odd number, low- frequency components have opposite sign J -1 (I) = -J 1 (I) Negative spectrum represents out-of-phase components. Combination of in- phase and out-of-phase components of same amplitude cancels out. fc-fm fc+fm fc magnitude fc+2.fm fc-2.fmfc-3.fm fc+3.fm Bessel Functions J1(X), J2(X) and J3(X)

11. FM Synthesis: Dynamic Spectrum Dynamic spectrum can be obtained by applying envelopes to both carrier and modulating signals. I is the max value of the modulation index More limited than additive synthesis, but requires less components. I*fm fm + ASD Envelope fc A fo

12. FM Synthesis: Dynamic Spectrum TIME FREQUENCY

13. FM Synthesis: Instrument Design Envelopes functions F1 & F2 change the timbre of the sound (John Chowning, 1973) A) Bell-like timbre B) Wood drum sound C) Brass-like timbre fm + fc A fo F1 F2 1/d fm*I 1/d (John Chowning, 1973)

14. FM Synthesis: Instrument Design (2) A) Bell-like timbre –D = 15s –Fc = 200Hz –Fm = 280Hz –I = 10 fm + fc A fo F1 F2 1/d fm*I 1/d (John Chowning, 1973) 1 d F1 = F2

15. FM Synthesis: Instrument A’s Spectrogram (bell)

16. FM Synthesis: Instrument Design (3) B) Wood drum sound –D=0.2 s –Fc=80Hz –Fm=55Hz –I=25 fm + fc A fo F1 F2 1/d fm*I 1/d (John Chowning, 1973) 1 d F1 1 d F2

17. FM Synthesis: Instrument B’s Spectrogram (drum)

18. FM Synthesis: Instrument Design (4) C) Brass-like timbre –D=0.6 –fc=440Hz –Fm=440Hz –I=5 (or 3) fm + fc A fo F1 F2 1/d fm*I 1/d (John Chowning, 1973) 1 d F1 = F2

19. FM Synthesis: Instrument C’s Spectrogram (brass)

20. FM Synthesis Instrument Design (5) D) Clarinet-like timbre –d = 0.5 s. –Fc = 900 Hz –Fm = 600 Hz –I = 2 –J = 4 fm + fc A fo F1F2 1/d fm*(I-J) 1/d (John Chowning, 1973) + J*fm 1 d F1 1 d F2

21. FM Synthesis: Instrument D’s Spectrogram (clarinet)

22. FM Synthesis: Instruments Summary A) BELL-LIKEB) WOODEN DRUM C) BRASS-LIKED) CLARINET

23. Plan Distortion-based synthesis –FM synthesis –Waveshaping Matlab code for synthesis Example of synthesized composition (Risset) Summary

24. Waveshaping (1): Definition Idea: “realizes spectra that have dynamic evolution of their components.” More computationally efficient than additive (like FM) Allows generation of band-limited spectrum with a set max harmonic number (unlike FM) f0 A WS waveshaper

25. Waveshaping (2): Transfer Function Waveshaper: nonlinear processor that alters the shape of the waveform passing through. Defined by a transfer function: ys=f(y0) The shape of the output waveform (and spectrum) changes with amplitude of the input signal! INPUT OUTPUT TRANSFER FUNCTION RESULT CARRIER

26. Transfer function Low amplitude ~ linear response High amplitude Nonlinear response

27. Waveshaping (3): Transfer Function INPUT OUTPUT TRANSFER FUNCTION RESULT CARRIER

28. Waveshaping (4): Some Transfer Functions in out in out in out A) LinearB) Odd C) Even in out D) Discontinuous in out E) Sharp points in out F) Ripple

29. Waveshaping: Linear Transfer Functions Eg, f(x) = 0.5x Reduces amplitude by half. No distortion, no harmonic added. in out f(x) = 0.5*x

30. Waveshaping: Odd Transfer Functions Definition: f(-x) = -f(x) Eg, f(x) = x^3 Nearly linear for low amplitude (  low distortion) Highly non-linear for high amplitude (  high distortion) Generates only odd- numbered harmonics! in out f(x) = x^3

31. Waveshaping: Even Transfer Functions Definition: f(-x) = f(x) Eg. f(x) = 2*x^2-3*x^4 Creates only even harmonics Even transfer function doubles the fundamental frequency  raises the pitch by one octave! in out f(x) = 2*x^2-3*x^4

32. Waveshaping: Other Transfer Functions Discontinuities and sharp points will create more harmonics than a smooth function Generally causes aliasing! For this reason, it is common to use polynomial transfer functions f(x) = d0+d1*x+d2*x^2+......+dn*x^n in out D) Discontinuous in out E) Sharp points

33. Waveshaping: Polynomial Transfer Functions Divh0h1h2h3h4h5h6h7h8h9 x^00.51 x^111 x^2221 x^3431 x^48641 x^5161051 x^632201561 x^764352171 x^812870562881 See Dodge & Jerse Eg, for a pure tone of amplitude 1, f(x)=x^5 will generate? What about f(x) = 2*x^2 – 3*x^4 ?

34. Waveshaping: Polynomial Transfer Functions The transfer function: f(x) = 2*x^2 – 3*x^4 applied to a pure tone (sinusoidal) carrier of amplitude 1, will generate the following harmonics: –H0 = 2*2/2 - 3*6/8 –H2 = 2*1/2 – 3*4/8 –H4 = -3*1/8

35. Waveshaping: Spectral matching For a pure tone cosine carrier of unit amplitude (A=1) – also called distortion index, Applying a transfer function based on a Chebyshev polynomial T k (x) will generate only the kth harmonic General form for Chebyshev polynomials: T k+1 (x) = 2xT k (x)-T k-1 (x)

36. Waveshaping: Chebyshev Polynomials T 0 (x) = 1 T 1 (x) = x T 2 (x) = 2x^2-1 T 3 (x) = 4x^3-3x T 4 (x) = 8x^4-8x^2+1 T 5 (x) = 16x^5-20x^3+5x T 6 (x) = 32x^6-48x^4+18x^2-1 Etc.

37. Waveshaping: Using Chebyshev Polynomials For example, to generate only 1 st, 2 nd, 4 th & 5 th harmonics with amplitudes 5, 1, 4, and 3, respectively. We can design the transfer function: F(x) = 5*T 1 (x)+1*T 2 (x)+4*T 4 (x)+3*T 5 (x)

38. Waveshaping: Instrument Design Instrument for clarinet-like timbre From Jean-Claude Risset’s Introductory Catalogue of Computer Synthesized Sounds. fm + ASD Envelope 256 F(x) * A 255 0.64 0.085

39. Waveshaping: Instrument Transfer Function

40. Waveshaping: Instrument Spectrogram

41. Waveshaping: Amplitude Scaling Distortion index A affects sound’s amplitude, and there by loudness. Similar to acoustic instruments: louder play usually generates more overtones. Excessive loudness variation  excessive bright tones. Solution: apply a scaling function to the amplitude R(A,x) = F(Ax)S(A) f0 A F(x)S(A) *

42. Plan Distortion-based synthesis –FM synthesis –Waveshaping Matlab code for synthesis Example of synthesized composition (Risset) Summary

43. Matlab Code Examples for all sounds are available here: http://info.ee.surrey.ac.uk/Personal/N.Pugeau lt/MediacastingL3_2012/http://info.ee.surrey.ac.uk/Personal/N.Pugeau lt/MediacastingL3_2012/ Parameters can be tweaked to evaluate their influence. Matlab is convenient for plotting results > spectrogram( y, ntime, noverlap, nfft, fs ) > spectrogram( y, 256, 250, 250, fs ) Can play sounds using the command: > sound( y, fs )

44. Plan Distortion-based synthesis –FM synthesis –Waveshaping Matlab code for synthesis Example of synthesized composition (Risset) Summary

45. Plan Distortion-based synthesis –FM synthesis –Waveshaping Matlab code for synthesis Example of synthesized composition (Risset) Summary

46. Additional Reading C. Dodge, C., & Jerse, T. A. (1997). Computer Music: Synthesis, Composition, and Performance. Schrimer, UK. (see chapters 5 and 6) Matlab examples for all sounds & instruments are available here: http://info.ee.surrey.ac.uk/Personal/N.Pug eault/MediacastingL3_2012/http://info.ee.surrey.ac.uk/Personal/N.Pug eault/MediacastingL3_2012/