Copyright 2004 Ken Greenebaum Introduction to Interactive Sound Synthesis Lecture 6: Wrapped I/O Ken Greenebaum

Copyright 2004 Ken Greenebaum Assignment 1 Most people got it completely Most people got it completely Most common mistake missing the: Most common mistake missing the: scale = 0.5 * pow(2, numBits) - 1; scale = 0.5 * pow(2, numBits) - 1;

Copyright 2004 Ken Greenebaum Assignment 2 Questions? Questions?

Copyright 2004 Ken Greenebaum Assignment 2 4 new lines: 4 new lines: 1. #include "pablio.h“ 2. float buffer[1]; 3. OpenAudioStream(&outStream, 48000, paFloat32, PABLIO_WRITE|PABLIO_MONO); 4. WriteAudioStream(outStream, buffer, 1);

Copyright 2004 Ken Greenebaum Assignment 2 2 lines modified: 2 lines modified: 1. double frameTime = 1.0/48000; 2. buffer[0] = (float)sin(2*PI*frequency*theta);

Copyright 2004 Ken Greenebaum Assignment 2 Stereo? Stereo? Default mode Default mode Multiplexed L,R Multiplexed L,R Buffer[0], Buffer[1] Buffer[0], Buffer[1] 1 Frame = 2 Samples 1 Frame = 2 Samples

Copyright 2004 Ken Greenebaum Assignment 2 You did it! You did it! Sampling/Quantization/Synthesis Sampling/Quantization/Synthesis Audio Out Audio Out We can do anything now (and will) We can do anything now (and will) Next assignment: Next assignment: Collision synthesizer Collision synthesizer Using additive synthesis Using additive synthesis Wood, glass, bells Wood, glass, bells Will get us thinking about the mixer to come Will get us thinking about the mixer to come

Copyright 2004 Ken Greenebaum Ring Buffer Review Ring Buffer FIFO: Ring Buffer FIFO:

Copyright 2004 Ken Greenebaum Ring Buffer Review Ring Buffer Algebra: Ring Buffer Algebra: Samples Available Samples Available (Tail - Head - 1) mod Size (Tail - Head - 1) mod Size Space Available Space Available min((Head - Tail -1) mod Size, high water mark) min((Head - Tail -1) mod Size, high water mark) Empty Buffer when Empty Buffer when Head = TailBuffer Head = TailBuffer Full when Full when (Head - Tail) Mod Size = 1 (Head - Tail) Mod Size = 1

Copyright 2004 Ken Greenebaum Introduction to the Ring Buffer FIFO Queue Latency control via watermarks: Latency control via watermarks:

Copyright 2004 Ken Greenebaum Introduction to the Ring Buffer FIFO Queue Buffer over time: Buffer over time:

Copyright 2004 Ken Greenebaum Introduction to the Ring Buffer FIFO Queue Key Concepts: Key Concepts: Hysteresis (implications in thermostat) Hysteresis (implications in thermostat) Latency Latency Controlled by high water mark Controlled by high water mark Opportunities in latency Opportunities in latency Elasticity - Variable latency Elasticity - Variable latency Importance of the low water mark Importance of the low water mark Unblocking in time to re-fill Unblocking in time to re-fill

Copyright 2004 Ken Greenebaum Introduction to the Ring Buffer FIFO Queue Knowing the future by living in the (near) past: Knowing the future by living in the (near) past:

Copyright 2004 Ken Greenebaum Wrapped I/O: Questions from reading? Questions from reading?

Copyright 2004 Ken Greenebaum Wrapped I/O: Benefits to wrapping ring buffer in API: Benefits to wrapping ring buffer in API: Data abstraction Data abstraction Allows data structure to change Allows data structure to change Enforce uniform access Enforce uniform access Contract allows ring buffer algebra to work Contract allows ring buffer algebra to work Code re-use Code re-use Every application doesn’t have to re-create the wheel Every application doesn’t have to re-create the wheel (Potentially inconsistently!) (Potentially inconsistently!)

Copyright 2004 Ken Greenebaum Wrapped I/O: Issues with wrapping ring buffer in API: Issues with wrapping ring buffer in API: Extra copy required Extra copy required Actually Read and Write req’d anyway Actually Read and Write req’d anyway Perhaps limit possibilities: Perhaps limit possibilities: Move H or T w/o modifying samples Move H or T w/o modifying samples

Copyright 2004 Ken Greenebaum Buffer Issues D/A, A/D are realtime devices D/A, A/D are realtime devices Can’t be blocked Can’t be blocked Need/Generate Frame every 1/FrameRate seconds Need/Generate Frame every 1/FrameRate seconds Underflow Underflow D/A requires next sample but RB empty D/A requires next sample but RB empty Producer wasn’t run in time to meet deadline Producer wasn’t run in time to meet deadline Overflow Overflow A/D produces next sample but RB is full A/D produces next sample but RB is full Consumer wasn’t run in time to meet deadline Consumer wasn’t run in time to meet deadline

Copyright 2004 Ken Greenebaum Buffer Issues Underflow, Overflow result: Underflow, Overflow result: Dropped samples Dropped samples Lost forever Lost forever Annoying clicks, pops Annoying clicks, pops Very Annoying sound loop Very Annoying sound loop

Copyright 2004 Ken Greenebaum Underflow Policies Stop (silently) + wait for the missing data Stop (silently) + wait for the missing data Application: Radio talk show Application: Radio talk show Continue immediately Continue immediately Application: Playing music Application: Playing music Give up and start all over again Give up and start all over again Application: Mastering CD Application: Mastering CD

Copyright 2004 Ken Greenebaum Underflow Policy Quiescent underflow Quiescent underflow System provides zero’s (silence) on underflow System provides zero’s (silence) on underflow SGI AL, PABLIO SGI AL, PABLIO Consumer zero’s samples after they are used Consumer zero’s samples after they are used Non-Quiescent underflow Non-Quiescent underflow DirectSound DirectSound Many (most?) libraries Many (most?) libraries

Copyright 2004 Ken Greenebaum Quiescent Underflow Uses Sound injection Sound injection Hold sound stream open Hold sound stream open Only inject (brief) sounds as desired Only inject (brief) sounds as desired No need for audio process to wait and flush the samples out No need for audio process to wait and flush the samples out Sound may played at time with extreme precision Sound may played at time with extreme precision

Copyright 2004 Ken Greenebaum Stricter Underflow Policy Underflow notification required: Underflow notification required: Recording/Mastering application Recording/Mastering application Master failed, need to re-record Master failed, need to re-record Debugging/Tuning realtime application Debugging/Tuning realtime application Perhaps human tuning Perhaps human tuning Perhaps automatic, dynamic tuning Perhaps automatic, dynamic tuning Using high-water mark Using high-water mark Fallback to higher latency Fallback to higher latency Fallforward? Fallforward?

Copyright 2004 Ken Greenebaum Aliasing Underflow must be detected in audio system Underflow must be detected in audio system Did the pointer advance by one sample? Did the pointer advance by one sample? Or loop completely? Or loop completely?

Copyright 2004 Ken Greenebaum Detecting Aliasing in User-space Attempt to monitor underflow/aliasing Attempt to monitor underflow/aliasing Using large buffer: Using large buffer: (underflow?) (underflow?)

Copyright 2004 Ken Greenebaum Detecting Aliasing in User-space May also use another clock source May also use another clock source CPU’s high resolution timer CPU’s high resolution timer Issue Issue Another clock isn’t locked to D/A Another clock isn’t locked to D/A Overcome using synchronization techniques Overcome using synchronization techniques

Copyright 2004 Ken Greenebaum Does your computer drop samples? Well does it? Well does it? How do you know? How do you know? What can you do about it? What can you do about it?

Copyright 2004 Ken Greenebaum Ramptest Output a known signal Output a known signal What signal? What signal? Why? Why? Loopback outback to input Loopback outback to input Analyze the signal received Analyze the signal received Best if digital loopback Best if digital loopback What conditions to test for? What conditions to test for?

Copyright 2004 Ken Greenebaum Latency as a cure-all? Many audio systems will increase buffering in an attempt to provide more robustness Many audio systems will increase buffering in an attempt to provide more robustness How much buffering in your system? How much buffering in your system? How to determine this? How to determine this? What are the costs of latency? What are the costs of latency?

Copyright 2004 Ken Greenebaum Canonicalization/Mixing Hardware probably has native format Hardware probably has native format 16 bit 2’s complement stereo? 16 bit 2’s complement stereo? Nice if system accepts any format Nice if system accepts any format Easiest to mix like streams Easiest to mix like streams Copy opportunity to canonicalize format Copy opportunity to canonicalize format Can attenuate Can attenuate Can even re-sample at this time too Can even re-sample at this time too

Copyright 2004 Ken Greenebaum Underflow in a Mixer What should happen if one stream of a mix underflows? What should happen if one stream of a mix underflows? 1. Underflow the result of the mix? 2. Allow the mix to continue minus the one channel? AL does #2 AL does #2 What does Direct Sound do? What does Direct Sound do? Why? Why?

Copyright 2004 Ken Greenebaum Overflow in a Mixer Opposite of a mixed output Opposite of a mixed output Fanned out input Fanned out input Input signal provided to many consumers simulteneously Input signal provided to many consumers simulteneously Why would this be desirable? Why would this be desirable?

Copyright 2004 Ken Greenebaum Fan-out Simultaneous Examples: Simultaneous Examples: Audio recorder Audio recorder Audio level meter Audio level meter Voice recognizer Voice recognizer What should happen if one stream in a fanout backs up? What should happen if one stream in a fanout backs up?

Copyright 2004 Ken Greenebaum Overflow in Fan-out AL provides input to all inputs AL provides input to all inputs Only the overflowing buffer loses samples Only the overflowing buffer loses samples

Copyright 2004 Ken Greenebaum Readings: Audio Anecdotes Audio Anecdotes Perceivable Audio Latencies Perceivable Audio Latencies

Copyright 2004 Ken Greenebaum Next class: Perception of Latency Perception of Latency Additive Synthesis Additive Synthesis Next assignment: Collision Synth Next assignment: Collision Synth