1. Engineering Math Physics (EMP)
Digital Cameras
Engineering Math Physics (EMP)
Jennifer Rexford

2. Image Transmission Over Wireless Networks
Image capture and compression
Inner-workings of a digital camera
Manipulating & transforming a matrix of pixels
Implementing a variant of JPEG compression
Wireless networks
Wireless technology
Acoustic waves and electrical signals
Radios
Video over wireless networks
Video compression and quality
Transmitting video over wireless
Controlling a car over a radio link

3. Traditional Photography
A chemical process, little changed from 1826
Taken in France on a pewter plate
… with 8-hour exposure
Taken by Joseph Nicéphore Niépce in France. It is on a pewter plate, and used a petroleum-based compound similar to asphalt as the photographic medium. The parts of the picture exposed to light hardened, and after an eight-hour exposure (!), the areas not exposed to light were washed away with chemicals.
There is a very informative website discussing this picture at It’s from the University of Texas at Austin, where this photograph (or “heliograph”, as Niépce called it) currently resides.
The world's first photograph

4. Image Formation
Digital Camera
Film
Eye

5. Image Formation in a Pinhole Camera
Light enters a darkened chamber through pinhole opening and forms an image on the further surface

6. Aperture Hole or opening where light enters Pupil of the human eye
Or, the diameter of that hole or opening
Pupil of the human eye
Bright light: 1.5 mm diameter
Average light: 3-4 mm diameter
Dim light: 8 mm diameter
Camera
Wider aperture admits more light
Though leads to blurriness in the objects away from point of focus

7. Shutter Speed Time for light to enter camera Exposure
Longer times lead to more light
… though blurs moving subjects
Exposure
Total light entering the camera
Depends on aperture and shutter speed

8. Digital Photography Digital photography is an electronic process
Only widely available in the last ten years
Digital cameras now surpass film cameras in sales

9. Image Formation in a Digital Camera
Photon
+10V        +
A sensor converts one
kind of energy to another
Array of sensors
Light-sensitive diodes convert photons to electrons
Buckets that collect charge in proportion to light
Each bucket corresponds to a picture element (pixel)

10. CCD: Charge Coupled Device
CCD sensor
Common sensor array used in digital cameras
Each capacitor accumulates charge in response to light
Responds to about 70% of the incident light
In contrast, photographic film captures only about 2%
Also widely used in astronomy telescopes

11. Sensor Array: Image Sampling
Pixel (Picture Element): single point in a graphic image

12. Sensor Array: Reading Out the Pixels
Transfer the charge from one row to the next
Transfer charge in the serial register one cell at a time
Perform digital to analog conversion one cell at a time
Store digital representation
Digital-to-analog conversion

13. Sensor Array: Reading Out the Pixels

14. More Pixels Mean More Detail
Three pictures were taken of the previous scene. Three different resolutions were used. This sample from the center of the picture shows the differences in detail at the four resolutions x 1400 is 2.1 megapixels; 1280 x 960 is 1.1 megapixels; 640 x 480 is .3 megapixels, or old Sony Mavica resolution.
640 x 480

15. The x 1704 hand
The 320 x 240 hand

16. Representing Color Light receptors in the human eye RGB color model
Rods: sensitive in low light, mostly at periphery of eye
Cones: only at higher light levels, provide color vision
Different types of cones for red, green, and blue
RGB color model
A color is some combination of red, green, and blue
Intensity value for each color
0 for no intensity
1 for high intensity
Examples
Red: 1, 0, 0
Green: 0, 1, 0
Yellow: 1, 1, 0

17. Representing Image as a 3D Matrix
In the lab this week…
Matlab experiments with digital images
Matrix storing color intensities per pixel
Row: from top to bottom
Column: from left to right
Color: red, green, blue
Examples
M(3,2,1): third row, second column, red intensity
M(4,3,2): fourth row, third column, green intensity 1 2 3 1 2

18. Limited Granularity of Color
Three intensities, one per color
Any value between 0 and 1
Storing all possible values take a lot of bits
E.g., storing
Can a person really differentiate from ?
Limiting the number of intensity settings
Eight bits for each color
From to
With 28 = 256 values
Leading to 24 bits per pixel
Red: 255, 0, 0
Green: 0, 255, 0
Yellow: 255, 255, 0

19. Number of Bits Per Pixel
More bits can represent a wider range of colors
24 bits can capture 224 = 16,777,216 colors
Most humans can distinguish around 10 million colors
8 bits / pixel / color
4 bits / pixel / color

20. Separate Sensors Per Color
Expensive cameras
A prism to split the light into three colors
Three CCD arrays, one per RGB color

21. Practical Color Sensing: Bayer Grid
Place a small color filter over each sensor
Each cell captures intensity of a single color
More green pixels, since human eye is better at resolving green

22. Practical Color Sensing: Interpolating
Challenge: inferring what we can’t see
Estimating pixels we do not know
Solution: estimate based on neighboring pixels
E.g., red for non-red cell averaged from red neighbors
E.g., blue for non-blue cell averaged from blue neighbors
Estimate “R” and “B” at the “G” cells from neighboring values

23. Interpolation Examples of interpolation Accuracy of interpolation
Good in low-contrast areas (neighbors mostly the same)
Poor with sharp edges (e.g., text)
and
makes
and
makes
and
makes

24. Are More Pixels Always Better?
Generally more is better
Better resolution of the picture
Though at some point humans can’t tell the difference
But, other factors matter as well
Sensor size
Lens quality
Whether Bayer grid is used
Problem with too many pixels
Very small sensors catch fewer photons
Much higher signal-to-noise ratio
Plus, more pixels means more storage…

25. Digital Images Require a Lot of Storage
Three dimensional object
Width (e.g., 640 pixels)
Height (e.g., 480 pixels)
Bits per pixel (e.g., 24-bit color)
Storage is the product
Pixel width * pixel height * bits/pixel
Divided by 8 to convert from bits to bytes
Example sizes
640 x 480: 1 Megabyte
800 x 600: 1.5 Megabytes
1600 x 1200: 6 Megabytes

26. Compression Benefits of reducing the size Redundancy in the image
Consume less storage space and network bandwidth
Reduce the time to load, store, and transmit the image
Redundancy in the image
Neighboring pixels often the same, or at least similar
E.g., the blue sky
Human perception factors
Human eye is not sensitive to high frequencies

27. Joint Photographic Experts Group
Starts with an array of pixels in RGB format
With one number per pixel for each of the three colors
And outputs a smaller file with some loss in quality
Exploits both redundancy and human perception
Transforms data to identify parts that humans notice less
More about transforming the data in Wednesday’s class
Uncompressed: 167 KB
Good quality: 46 KB
Poor quality: 9 KB

28. Conclusion Conversion of information Combines many disciplines
Light (photons) and a optical lens
Charge (electrons) and electronic devices
Bits (0s and 1s) and a digital computer
Combines many disciplines
Physics: lenses and light
Electrical engineering: charge coupled device
Computer science: manipulating digital representations
Mathematics: compression algorithms
Psychology/biology: human perception
Next class: compression algorithms