Chapter 6: Modifying Sounds Using Loops

Slides:



Advertisements
Similar presentations
Georgia Institute of Technology Introduction to Processing Digital Sounds.
Advertisements

ITEC 109 Lecture 25 Sound.
Sound, Part 3. Multiple Echoes Here is a recipe to create multiple echoes: def echoes(sndfile, delay, num): s1 = makeSound(sndfile) ends1 = getLength(s1)
A Media Computation Cookbook Manipulating Images and Sounds for Use in Alice Part 1: Image Manipulations Part 2: Advanced Image Manipulations, e.g., changing.
SIMS-201 Characteristics of Audio Signals Sampling of Audio Signals Introduction to Audio Information.
IT-101 Section 001 Lecture #8 Introduction to Information Technology.
1 Live Sound Reinforcement Audio measurements. 2 Live Sound Reinforcement One of the most common terms you will come across when handling any type of.
Computer Science 101 Introduction to Programming with Sounds.
Introduction to Computing and Programming in Python: A Multimedia Approach Chapter 6: Modifying Sounds Using Loops.
Digital audio. In digital audio, the purpose of binary numbers is to express the values of samples that represent analog sound. (contrasted to MIDI binary.
Introduction to Computing and Programming in Python: A Multimedia Approach Chapter 7: Modifying Samples in a Range.
Lecture # 22 Audition, Audacity & Sound Editing Sound Representation.
Problem Solving with Data Structures using Java: A Multimedia Approach Chapter 5: Arrays: A Static Data Structure for Sounds.
CompSci Today’s topics Sound Upcoming ä Intellectual property ä Network analysis ä Security Reading.
Computing with Digital Media: A Study of Humans and Technology Mark Guzdial, School of Interactive Computing.
COMP Representing Sound in a ComputerSound Course book - pages
Lecture 5: Signal Processing II EEN 112: Introduction to Electrical and Computer Engineering Professor Eric Rozier, 2/20/13.
Art 321 Sound, Audio, Acoustics Dr. J. Parker. Sound What we hear as sound is caused by rapid changes in air pressure! It is thought of as a wave, but.
CS 101: Introduction to Computing Programming picture manipulations Developed by Mark Guzdial, Georgia Institute of Technology, 2003–2004; modified by.
CompSci Today’s topics Sound Upcoming ä Sound splicing ä Intellectual property ä Network analysis Reading Introduction to Computing & Programming.
Basics of Digital Audio Outline  Introduction  Digitization of Sound  MIDI: Musical Instrument Digital Interface.
COSC 1P02 Introduction to Computer Science 4.1 Cosc 1P02 Week 4 Lecture slides “Programs are meant to be read by humans and only incidentally for computers.
Introduction to SOUND.
More Meaningful Jargon Or, All You Need to Know to Speak Like a Geek Sound.
Chapter 7: Modifying Samples in a Range. Chapter Objectives.
Georgia Institute of Technology Introduction to Processing Digital Sounds part 1 Barb Ericson Georgia Institute of Technology Sept 2005.
Georgia Institute of Technology Processing Sound Ranges Barb Ericson Georgia Institute of Technology July 2005.
1 Introduction to Information Technology LECTURE 6 AUDIO AS INFORMATION IT 101 – Section 3 Spring, 2005.
Introduction to Computing and Programming in Python: A Multimedia Approach Chapter 7: Modifying Samples in a Range.
© Houghton Mifflin Harcourt Publishing Company Preview Objectives The Production of Sound Waves Frequency of Sound Waves The Doppler Effect Chapter 12.
Encoding and Simple Manipulation
CS 102 Computers In Context (Multimedia)‏ 03 / 30 / 2009 Instructor: Michael Eckmann.
Introduction to Computing and Programming in Python: A Multimedia Approach Chapter 6: Modifying Sounds Using Loops.
Digital Audio. Acknowledgement Some part of this lecture note has been taken from multimedia course made by Asst.Prof.Dr. William Bares and from Paul.
Intro-Sound-part1 Introduction to Processing Digital Sounds part 1 Barb Ericson Georgia Institute of Technology Oct 2009.
CS1315: Introduction to Media Computation Sound Encoding and Manipulation.
: Main Topics vibrations and waves types and effects of waves sound resonance musical sounds.
Chapter 8: Modifying Samples in a Range. Chapter Objectives.
Multimedia Sound. What is Sound? Sound, sound wave, acoustics Sound is a continuous wave that travels through a medium Sound wave: energy causes disturbance.
1 CS 177 Week 8 Recitation Slides JES Sound functions and Modifying Sounds Increasing/Decreasing Volume Maximizing (Normalizing) Splicing Reversing Mirroring.
Session 18 The physics of sound and the manipulation of digital sounds.
Digital Audio I. Acknowledgement Some part of this lecture note has been taken from multimedia course made by Asst.Prof.Dr. William Bares and from Paul.
Fundamentals of Multimedia Chapter 6 Basics of Digital Audio Ze-Nian Li and Mark S. Drew 건국대학교 인터넷미디어공학부 임 창 훈.
Chapter 12 Preview Objectives The Production of Sound Waves
COSC 1P02 Introduction to Computer Science 5.1 Cosc 1P02 Week 5 Lecture slides Psychiatrist to patient "You have nothing to worry about - anyone who can.
1 CS 177 Week 7 Recitation Slides Modifying Sounds using Loops + Discussion of some Exam Questions.
Physics Mrs. Dimler SOUND.  Every sound wave begins with a vibrating object, such as the vibrating prong of a tuning fork. Tuning fork and air molecules.
Working with Sounds Barb Ericson College of Computing Georgia Institute of Technology
Fourier Analysis Patrice Koehl Department of Biological Sciences National University of Singapore
The Physics of Sound.
Week 7 - Wednesday CS 121.
Chapter 8: Making Sounds by Combining Pieces
Multimedia Systems and Applications
Introduction to Digital Audio
Processing Sound Ranges part 3
"Digital Media Primer" Yue-Ling Wong, Copyright (c)2013 by Pearson Education, Inc. All rights reserved.
Chapter 7: Modifying Samples in a Range
Sound & Sound Waves.
Introduction to Digital Audio
How sound works: Acoustics, the physics of sound
Chapter 7: Modifying Samples in a Range
Introduction to Digital Audio
A Media Computation Cookbook
Elements of Acoustic Phonetics
Introduction to Processing Digital Sounds
Processing Sound Ranges
CS 177 Week 3 Recitation Slides
Processing Sound Ranges part 3
CS1315: Introduction to Media Computation
CS1315: Introduction to Media Computation
Presentation transcript:

Chapter 6: Modifying Sounds Using Loops Introduction to Computing and Programming in Python: A Multimedia Approach Chapter 6: Modifying Sounds Using Loops With thanks to John Sanders of Suffolk University for contributions to these slides!

Chapter Objectives

How sound works: Acoustics, the physics of sound Sounds are waves of air pressure Sound comes in cycles The frequency of a wave is the number of cycles per second (cps), or Hertz Complex sounds have more than one frequency in them. The amplitude is the maximum height of the wave

Volume and Pitch: Psychoacoustics, the psychology of sound Our perception of volume is related (logarithmically) to changes in amplitude If the amplitude doubles, it’s about a 3 decibel (dB) change Our perception of pitch is related (logarithmically) to changes in frequency Higher frequencies are perceived as higher pitches We can hear between 5 Hz and 20,000 Hz (20 kHz) A above middle C is 440 Hz

“Logarithmically?” It’s strange, but our hearing works on ratios not differences, e.g., for pitch. We hear the difference between 200 Hz and 400 Hz, as the same as 500 Hz and 1000 Hz Similarly, 200 Hz to 600 Hz, and 1000 Hz to 3000 Hz Intensity (volume) is measured as watts per meter squared A change from 0.1W/m2 to 0.01 W/m2, sounds the same to us as 0.001W/m2 to 0.0001W/m2

Decibel is a logarithmic measure A decibel is a ratio between two intensities: 10 * log10(I1/I2) As an absolute measure, it’s in comparison to threshold of audibility 0 dB can’t be heard. Normal speech is 60 dB. A shout is about 80 dB

Demonstrating Sound MediaTools Fourier transform (FFT) Click here to see viewers while recording I usually bring in a couple musical instruments (harmonica, thumb piano, ukelele, flute) to show how they have different FFT “signature” (different spikes).

Digitizing Sound: How do we get that into numbers? Remember in calculus, estimating the curve by creating rectangles? We can do the same to estimate the sound curve Analog-to-digital conversion (ADC) will give us the amplitude at an instant as a number: a sample How many samples do we need?

Nyquist Theorem We need twice as many samples as the maximum frequency in order to represent (and recreate, later) the original sound. The number of samples recorded per second is the sampling rate If we capture 8000 samples per second, the highest frequency we can capture is 4000 Hz That’s how phones work If we capture more than 44,000 samples per second, we capture everything that we can hear (max 22,000 Hz) CD quality is 44,100 samples per second

Digitizing sound in the computer Each sample is stored as a number (two bytes) What’s the range of available combinations? 16 bits, 216 = 65,536 But we want both positive and negative values To indicate compressions and rarefactions. What if we use one bit to indicate positive (0) or negative (1)? That leaves us with 15 bits 15 bits, 215 = 32,768 One of those combinations will stand for zero We’ll use a “positive” one, so that’s one less pattern for positives

Two’s Complement Numbers 011 +3 Imagine there are only 3 bits 010 +2 we get 23 = 8 possible values 001 +1 Subtracting 1 from 2 we borrow 1 000 0 111 -1 Subtracting 1 from 0 we borrow 1’s 110 -2 which turns on the high bit for all 101 -3 negative numbers 100 -4

Two’s complement numbers can be simply added Adding -9 (11110111) and 9 (00001001)

+/- 32K Each sample can be between -32,768 and 32,767 Why such a bizarre number? Because 32,768 + 32,767 + 1 = 216 < 0 i.e. 16 bits, or 2 bytes > 0 Compare this to 0...255 for light intensity (i.e. 8 bits or 1 byte)

Sounds as arrays Samples are just stored one right after the other in the computer’s memory That’s called an array It’s an especially efficient (quickly accessed) memory structure (Like pixels in a picture)

Working with sounds We’ll use pickAFile and makeSound. We want .wav files We’ll use getSamples to get all the sample objects out of a sound We can also get the value at any index with getSampleValueAt Sounds also know their length (getLength) and their sampling rate (getSamplingRate) Can save sounds with writeSoundTo(sound, "file.wav")

Demonstrating Working with Sound in JES >>> filename=pickAFile() >>> print filename /Users/guzdial/mediasources/preamble.wav >>> sound=makeSound(filename) >>> print sound Sound of length 421109 >>> samples=getSamples(sound) >>> print samples Samples, length 421109 >>> print getSampleValueAt(sound,1) 36 >>> print getSampleValueAt(sound,2) 29 >>> explore(sound)

Demonstrating working with samples >>> print getLength(sound) 220568 >>> print getSamplingRate(sound) 22050.0 >>> print getSampleValueAt(sound,220568) 68 >>> print getSampleValueAt(sound,220570) I wasn't able to do what you wanted. The error java.lang.ArrayIndexOutOfBoundsException has occurred Please check line 0 of >>> print getSampleValueAt(sound,1) 36 >>> setSampleValueAt(sound,1,12) 12

Working with Samples We can get sample objects out of a sound with getSamples(sound) or getSampleObjectAt(sound,index) A sample object remembers its sound, so if you change the sample object, the sound gets changed. Sample objects understand getSample(sample) and setSample(sample,value)

Example: Changing Samples >>> soundfile=pickAFile() >>> sound=makeSound(soundfile) >>> sample=getSampleObjectAt(sound,1) >>> print sample Sample at 1 value at 59 >>> print sound Sound of length 387573 >>> print getSound(sample) >>> print getSample(sample) 59 >>> setSample(sample,29) 29

“But there are thousands of these samples!” How do we do something to these samples to manipulate them, when there are thousands of them per second? We use a loop and get the computer to iterate in order to do something to each sample. An example loop: for sample in getSamples(sound): value = getSample(sample) setSample(sample,value)

Recipe to Increase the Volume def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 2) Using it: >>> f="/Users/guzdial/mediasources/gettysburg10.wav" >>> s=makeSound(f) >>> increaseVolume(s) >>> play(s) >>> writeSoundTo(s, "/Users/guzdial/mediasources/louder-g10.wav")

How did that work? When we evaluate increaseVolume(s), the function increaseVolume is executed The sound in variable s becomes known as sound sound is a placeholder for the sound object s. >>> f=pickAFile() >>> s=makeSound(f) >>> increaseVolume(s) def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 2)

Starting the loop getSamples(sound) returns a sequence of all the sample objects in the sound. The for loop makes sample be the first sample as the block is started. def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 2) Compare: for pixel in getPixels(picture):

Executing the block We get the value of the sample named sample. We set the value of the sample to be the current value (variable value) times 2 def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 2)

Next sample Back to the top of the loop, and sample will now be the second sample in the sequence. def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 2)

And increase that next sample We set the value of this sample to be the current value (variable value) times 2. def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 2)

And on through the sequence The loop keeps repeating until all the samples are doubled def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 2)

How are we sure that that worked? >>> print s Sound of length 220567 >>> print f /Users/guzdial/mediasources/gettysburg10.wav >>> soriginal=makeSound(f) >>> print getSampleValueAt(s,1) 118 >>> print getSampleValueAt(soriginal,1) 59 >>> print getSampleValueAt(s,2) 78 >>> print getSampleValueAt(soriginal,2) 39 >>> print getSampleValueAt(s,1000) -80 >>> print getSampleValueAt(soriginal,1000) -40 Here we’re comparing the modified sound s to a copy of the original sound soriginal

Exploring both sounds The right side does look like it’s larger.

Decreasing the volume def decreaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample,value * 0.5) This works just like increaseVolume, but we’re lowering each sample by 50% instead of doubling it. Be sure to actually do this!

We can make this generic By adding a parameter, we can create a general changeVolume that can increase or decrease volume. def changeVolume(sound , factor): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample ,value * factor)

Recognize some similarities? def increaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample, value*2) def increaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*1.2) def decreaseVolume(sound): for sample in getSamples(sound): value = getSampleValue(sample) setSampleValue(sample, value*0.5) def decreaseRed(picture): for p in getPixels(picture): value=getRed(p) setRed(p,value*0.5)

Does increasing the volume change the volume setting? No The physical volume setting indicates an upper bound, the potential loudest sound. Within that potential, sounds can be louder or softer They can fill that space, but might not. (Have you ever noticed how commercials are always louder than regular programs?) Louder content attracts your attention. It maximizes the potential sound.

Maximizing volume How, then, do we get maximal volume? (e.g. automatic recording level) It’s a three-step process: First, figure out the loudest sound (largest sample). Next, figure out how much we have to increase/decrease that sound to fill the available space We want to find the amplification factor amp, where amp * loudest = 32767 In other words: amp = 32767/loudest Finally, amplify each sample by multiplying it by amp

Maxing (normalizing) the sound This loop finds the loudest sample def normalize(sound): largest = 0 for s in getSamples(sound): largest = max(largest, getSampleValue(s)) amplification = 32767.0 / largest print "Largest sample value in original sound was", largest print ”Amplification multiplier is", amplification louder = amplification * getSampleValue(s) setSampleValue(s, louder) Q: Why 32767? A: Later… This loop actually amplifies the sound

Max() max() is a function that takes any number of inputs, and always returns the largest. There is also a function min() which works similarly but returns the minimum >>> print max(1,2,3) 3 >>> print max(4,67,98,-1,2) 98

Or: use if instead of max def normalize(sound): largest = 0 for s in getSamples(sound): if getSampleValue(s) > largest: largest = getSampleValue(s) amplification = 32767.0 / largest print "Largest sample value in original sound was", largest print ”Amplification factor is", amplification louder = amplification * getSampleValue(s) setSampleValue(s, louder) Instead of finding max of all samples, check each in turn to see if it’s the largest so far

Aside: positive and negative extremes assumed to be equal We’re making an assumption here that the maximum positive value is also the maximum negative value. That should be true for the sounds we deal with, but isn’t necessarily true Try adding a constant to every sample. That makes it non-cyclic I.e. the compressions and rarefactions in the sound wave are not equal But it’s fairly subtle what’s happening to the sound.

Why 32767.0, not 32767? Why do we divide out of 32767.0 and not just simply 32767? Because of the way Python handles numbers If you give it integers, it will only ever compute integers. >>> print 1.0/2 0.5 >>> print 1.0/2.0 >>> print 1/2

Avoiding clipping Why are we being so careful to stay within range? What if we just multiplied all the samples by some big number and let some of them go over 32,767? The result then is clipping Clipping: The awful, buzzing noise whenever the sound volume is beyond the maximum that your sound system can handle. Be sure to generate some clipping here!

All clipping, all the time def onlyMaximize(sound): for sample in getSamples(sound): value = getSampleValue(sample) if value > 0: setSampleValue(sample, 32767) if value < 0: setSampleValue(sample, -32768)

Processing only part of the sound What if we wanted to increase or decrease the volume of only part of the sound? Q: How would we do it? A: We’d have to use a range() function with our for loop Just like when we manipulated only part of a picture by using range() in conjunction with getPixels()