2. Introducing Computer Programming with Multimedia Barb Ericson Institute for Computing Education Georgia Institute of Technology ericson@cc.gatech.edu

3. Computer Science Crisis 40% drop in majors since 2001 Nationwide –Up to 60% drop in some places Percentage of women is down to about 12% –From a high of 30% in the 1980s Never attracted many African Americans, Hispanics, Native Americans

4. Why the Decline? Dot com crash in 2001 –Loss of jobs AP Exam switched from C++ to Java –Many schools quit teaching AP Computer Science Visual Basic changed –In.net to be object-oriented Perceptions of Computer Science –boring, antisocial, not creative, and just programming

5. Looming Shortage Only a few jobs have gone overseas –About 2-3% We have recovered from the dot com crash –There are now more jobs in this area than in 2001 Computing jobs are some of the fastest growing –US Dept of Labor predicts 45.93% growth

6. Reversing the Trend We need to get more students interested in computer science –Not drive them away We need to appeal to a diverse group of students –Not just geeks All students need to have some understanding of computer science –Our society relies on technology

7. Our Solution Use multimedia projects to introduce computing concepts –Pictures, sounds, movies, text Use open-ended projects to encourage creativity –Create an image collage, create a sound collage, create a movie Make computing social –Encourage students to show off their work

8. Choice of Languages Python –Used by many companies: IL&M, Google, Nextel, etc. –Easier syntax than many languages –Can use to teach functional, procedural, or object- oriented programming –Used as a scripting or extension language for many applications –Growing in popularity Java –Popular language used for AP CS

9. Digitizing Pictures We digitize pictures into lots of little dots Enough dots and it looks like a continuous whole to our eye Each picture element is referred to as a pixel –Each pixel knows its color –It also knows where it is in its picture

10. Encoding Color: RGB In RGB, each color has three component colors: –Red, Green, and Blue In most computer-based models of RGB, a single byte (8 bits) is used for each –So a complete RGB color is 24 bits, 8 bits of each –0,0,0 is black –255,255,255 is white –255,0,0 is red

11. Make and Show a Picture Use pickAFile function To select a picture file (ends in.jpg) Use makePicture(file) to create the picture Use show(picture) to display the picture

12. Use MediaTools Click on MediaTools Select the picture to explore Move the cursor while holding down the left mouse button –To see x,y,r,g,b Click on zoom –To zoom in or out

13. Creating a Python Function Define a function Load the function Try the function

14. Rules for Python Functions Use the keyword def to define a function –Followed by the function name Use parentheses after the name –Name parameters End with a colon Indent the body of the function def pickAndShow(): myFile = pickAFile() myPicture = makePicture(myFile) show(myPicture)

15. Using a Loop def decreaseRed(picture): for p in getPixels(picture): value = getRed(p) setRed(p,value * 0.5)

16. More Picture Functions def greyscale(picture): for p in getPixels(picture): redness=getRed(p) greenness=getGreen(p) blueness=getBlue(p) luminance=(redness+blueness+greenness)/3 setColor(p, makeColor(luminance,luminance,luminance)) def negate(picture): for px in getPixels(picture): red=getRed(px) green=getGreen(px) blue=getBlue(px) negColor=makeColor(255-red,255-green,255- blue) setColor(px,negColor)

17. How Sound Works: Sounds are waves of air pressure –Sound comes in cycles –The frequency of a wave is the number of cycles per second (cps), or Hertz –The amplitude is the maximum height of the wave

18. Volume and Pitch Our perception of volume is related (logarithmically) to changes in amplitude –If the amplitude doubles, it’s about a 3 decibel (dB) change. Our perception of pitch is related (logarithmically) to changes in frequency –Higher frequencies are perceived as higher pitches –We can hear between 5 Hz and 20,000 Hz (20 kHz) –A above middle C is 440 Hz

19. Digitizing Sound We can estimate the sound curve –Analog-to-digital conversion (ADC) will give us the amplitude at an instant as a number: a sample –How many samples do we need? 2 x max frequency CD quality is 44,100 samples per second

20. Digitizing sound in the computer Each sample is stored as a number –16 bits, 2 16 = 65,536 combinations –But we want both positive and negative values To indicate compressions and rarefactions. Each sample can be between -32,768 and 32,767

21. Exploring a Sound Use MediaTools to explore a Sound

22. Working with Sounds We’ll use getSamples to get all the sample objects out of a sound –Use getSample to get the value from a Sample obj We can also get the value at any index with getSampleValueAt Sounds also know their length (getLength) and their sampling rate (getSamplingRate) Save a sound to a file using writeSoundTo(sound,”file.wav”)

23. Decreasing the volume def decreaseVolume(sound): for sample in getSamples(sound): value = getSample(sample) setSample(sample,value * 0.5) This works just like decreaseRed, but decreasing the amplitude (volume) instead of the red color.

24. Multimedia Java CS1 We use DrJava –A free and easy development environment –With an interactions pane We have created Java classes –For the students to use and modify World Turtle Picture Sound FileChooser

25. Introduction to Java –Math operators, printing results, data types, casting, relational operators, Strings, variables Introduction to Programming –Creating and naming objects Using a turtle and a world –Creating new Turtle methods Draw simple shapes Using parameters Java Syllabus

26. Syllabus - Continued Modifying Pictures using Loops –One-dimensional arrays –Use for-each, while, and for loops to –Increase/decrease colors, fake a sunset, lighten and darken, create a negative, and grayscale

27. Create and Show a Picture

28. Java Negate Method /** * Method to negate the picture */ public void negate() { Pixel[] pixelArray = this.getPixels(); Pixel pixelObj = null; int redValue, blueValue, greenValue = 0; // loop through all the pixels for (int i = 0; i < pixelArray.length; i++)

29. Java Negate Method - Cont { // get the current pixel pixelObj = pixelArray[i]; // get the values redValue = pixelObj.getRed(); greenValue = pixelObj.getGreen(); blueValue = pixelObj.getBlue(); // set the pixel's color pixelObj.setColor( new Color(255 - redValue, 255 - greenValue, 255 - blueValue)); }

30. Testing Negate

31. Syllabus - Continued Modifying Pixels in a Matrix –Two-dimensional arrays –Nested loops –Copying, mirroring, blending, rotation, scaling

32. Syllabus - Continued Conditionally Modifying Pixels –Replacing a color, reducing red-eye, edge detection, sepia-toned, posterize, highlight extremes, blurring, background subtractions, chromakey

33. Syllabus - Continued Drawing on Pictures –Using existing Java classes Inheritance Interfaces –Drawing simple shapes, drawing text, general copy, general scale, shearing, gradient paint, general blending, clipping

34. Syllabus - Continued Modifying all Samples in a Sound –1D arrays –Loops –Conditional execution –Change volume, normalizing a sound (make it as loud as possible), force to extremes

35. Syllabus - Continued Modifying Samples using Ranges –Loops –Clipping, splicing, reversing, mirroring

36. Syllabus - Continued Combining and Creating Sounds –Class and private methods –Composing sounds, blending sounds, changing frequencies, and creating echoes –MP3 and MIDI

37. Syllabus - Continued Creating Classes –Defining a class –Overloading constructors –Creating and initializing an array –Creating getters and setters –Creating a main method –Javadoc comments –Using Inheritance ConfusedTurtleTurtle

38. Syllabus - Continued Creating and Modifying Text –Reading from and writing to files Handling Exceptions –Modifying programs –Getting text from networks –Creating random sentences –Using text to shift between media

39. Syllabus - Continued Making Text for the Web –Throwing exceptions, HashMap, Generics, and Iterators –Create a web page from a directory –Create a web page from other web pages –Creating a web page from a database

40. Syllabus - Continued Encoding, Manipulating, and Creating Movies –Frame-based animations with simple shapes and text –Special effects – fade out, fake sunset, and chromakey

41. Syllabus - Continued Speed –What makes programs fast? Compilers and Interpreters Searching Algorithms that can’t be written –What makes computers fast? Clock rates, Storage, Display

42. Syllabus - Continued Javascript –Syntax –User Interfaces –Multimedia

43. Multimedia CS2 in Java Driving question: “How did the wildebeests stampede in The Lion King?”

44. CS2 Ideas Teach data structures using multimedia –Stacks and queues for simulations –Linked lists of pictures Order in linked list is order in scene Order in linked list is the layer the picture is in Circular linked list for animation –Linked lists of sounds Weave and repeat musical phrases –Trees of pictures Scene graph –Trees of sounds Different traversal gives different result

45. Examples of Student Work Soup- Audio Collage Canon- LinkedList of (MIDI) Music

46. Results Python CS1 is used for non-majors at Georgia Tech –Business students went from 49% to 88% pass rate –Similar results at Gainesville and U.Ill-Chicago Java CS1 is used in Georgia High Schools –Both in an introductory course and in Advanced Placement courses –Also used at Dennison and Duke Java CS2 is used at Georgia Tech –Students are transferring into computer science

47. Why Media Computation? Works on nearly all computers –Unlike Alice which requires newer computers Low cost –Unlike LEGO Mindstorms or other robots Students can see (and hear) if their programs work Computer Science concepts are introduced to solve problems –Looping instead of changing thousands of pixels by hand Open-ended assignments allow students to be creative –Students can use their own media Students can publicly share their media artifacts with others

48. What’s on the CD CS1-Python materials –Chapters from the Media Computation book in Python –Course slides –Free Jython Environment for Students (JES) –Instructor Resources CS1-Java materials –Chapters from the Media Computation book in Java –Course Slides –Java Classes

49. What’s on the CD - Continued CS2-Java materials –Course notes –Course slides –Java Classes DrJava –Free Development environment for Java Mediasources –Royalty free pictures and sounds

50. Additional Resources Web Site for Media Computation –http://coweb.cc.gatech.edu/mediaComp-planhttp://coweb.cc.gatech.edu/mediaComp-plan CS1 Python Book –Introduction to Computing and Programming in Python, A Multimedia Approach by Mark Guzdial, ISBN: 0131176552 CS1 Java Book –Introduction to Computing and Programming with Java, A Multimedia Approach by Mark Guzdial and Barbara Ericson, ISBN: 0131496980 CS2 Java Book –Coming in 2007

51. Funding Sources National Science Foundation Georgia Tech's College of Computing Georgia’s Department of Education Al West Fund Toyota Foundation