DIGITAL CAMERAS IN EMBEDDED SYSTEMS BY: ANTONIO JIM É NEZGRIFFIN MILLERVARUN PATEL

BACKGROUND First cameras built in mid 1800s Pinhole Cameras (not even cameras) Silver-plated sheets of copper Film cameras Digital cameras (CCD then CMOS)  uses a semiconductor (silicon)  light charges up each pixel / sensor and is then processed Who took this? ^

Why do WE need digital cameras in embedded systems?

READABLE DATA Digital cameras and pictures are a way of transmitting information that’s easy for humans to process Enables embedded systems to “see”

CAPTURING A SPECIFIC IMAGE FOR ITS DATA QR Codes Facial Recognition Security recognition.jpg

SMART TECHNOLOGY Autonomous Robotics  Collision control  Remote control Smart Cameras Going beyond the image Pre vs post processed Human Detection

LIGHT SENSING Indoor Localization  Takes advantage of the time-lapse in digital cameras Light Fidelity  GBs per second  Cheap and easy to implement  different frequencies

Image sensors: CCD vs CMOS

Same task: photons to electrons

CCD: Charge-Coupled Device 2-D array of light-sensing “buckets” on top of a thin silicon wafer Each bucket consists of a positively doped MOS-Capacitor Incident photons hit the oxide and free electrons The greater the light intensity, the greater amount of freed electrons These photo-generated electrons are stored Each bucket in the array is capable of representing a pixel

Reading the charges (CCD) Charges are shifted in two directions: parallel and serial The serial shift occurs at the edge of the array  Charges are shifted towards an ADC to be digitized  The resulting voltage passes through an amplifier  The value for each pixel is stored in memory After the edge empties, the parallel shift occurs  Every row of capacitors passes its charge to the neighboring row  Each row gets rid old charge and receives new charge The process repeats until every charge is processed

CMOS: COMPLEMENTARY METAL OXIDE SEMICONDUCTOR Similar technique for photo-generating electrons Pixel charges are passed to a group of readers Each reader has its own charge-to-voltage conversion, amplifier, and ADC converter No need to shift charges across the pixel array Rolling shutter captures light from top left to bottom right Manufacturing process is the same for regular chips Animation / CMOS CCD

SO WHICH ONE IS THE BEST? CCD: Pros: ❏ Global shutter provides uniformity ❏ Almost every photon generates an electron ❏ Levels of noise are very reduced Cons: ❏ Special manufacturing process is expensive ❏ Relatively low speed ❏ High power consumption ❏ Blooming CMOS: Pros: ❏ Conventional manufacturing process is cheap ❏ High speed ❏ Low power consumption Cons: ❏ Rolling-shutter effect ❏ Light sensitivity is not the best

SO WHICH ONE IS THE BEST? CCD: Pros: ❏ Global shutter provides uniformity ❏ Almost every photon generates an electron ❏ Levels of noise are very reduced Cons: ❏ Special manufacturing process is expensive ❏ Relatively low speed ❏ High power consumption ❏ Blooming CMOS: Pros: ❏ Conventional manufacturing process is cheap ❏ High speed ❏ Low power consumption Cons: ❏ Rolling-shutter effect ❏ Light sensitivity is not the best

Blooming CCDnot CCD

SO WHICH ONE IS THE BEST? CCD: Pros: ❏ Global shutter provides uniformity ❏ Almost every photon generates an electron ❏ Levels of noise are very reduced Cons: ❏ Special manufacturing process is expensive ❏ Relatively low speed ❏ High power consumption ❏ Blooming CMOS: Pros: ❏ Conventional manufacturing process is cheap ❏ High speed ❏ Low power consumption Cons: ❏ Rolling-shutter effect ❏ Light sensitivity is not the best

SO WHICH ONE IS THE BEST? CCD: Pros: ❏ Global shutter provides uniformity ❏ Almost every photon generates an electron ❏ Levels of noise are very reduced Cons: ❏ Special manufacturing process is expensive ❏ Relatively low speed ❏ High power consumption ❏ Blooming CMOS: Pros: ❏ Conventional manufacturing process is cheap ❏ High speed ❏ Low power consumption Cons: ❏ Rolling-shutter effect ❏ Light sensitivity is not the best

ROLLING-SHUTTER EFFECT /

not CMOS CMOS Fast-moving objects

Summary of CCD vs CMOS CCDCMOS Light sensitivityHighLow Noise levelLowModerate ShutterGlobalRolling Power consumption HighLow Chip outputAnalogDigital

What happens after raw image is captured? Camera’s use SIMD (Single-Instruction Multiple-Data) Microprocessors Redundant tasks are often performed to process images (filters) Latency due to processing the image

Functional Block Diagram (CCD) BUS Serial/Parallel shift signals

Sampling Optimization Each pixel is a mixture of Red, Green, Blue 12 bits per color = 4096 shades of color Most cameras use 8 bits = 256 steps 256 x 256 x 256 > 16 million colors Shades 16 Shades 50 Shades of Grey

CMU CAM4 libraries: CMU CAM4 6MHz Clock TTL UART bus protocol Baud Rate : 19,200 to 250,000 Parallax P8X32A Microchip 30 FPS Servo controllers at 50Hz Refresh Rate $39.95 (robotmesh)

Few Functions in CMUCAM4 library Getpixel Andpixels cameraBrightness getServoPosition setServoPosition trackColor trackWindow trackShape And many other functions

Raspberry Pi Camera 3.3 V 5MP 1080p Video Quality $34.95 (Sparkfun.com) CMOS Camera Module (serial com port) 6 – 20 V PC1089K CMOS Chip NTSC protocol (yellow wire) $ oculus-controlled-video-camera-to-a-drone-and-take-it-for-a- spin/?utm_campaign=fb&ncid=fb Raspberry Pi Camera (OV5647) CMOS High Quality Video 5 Mega Pixel 30 FPS 68 MHz MHz Camera Serial Interface $ Uart Camera Module with Jpeg compression - C V UART: 115.2Kbps for transferring JPEG still pictures or 160x128 with 0.75fps. Built-in color conversion circuits $ html

photography/?_r=0&module=Slide&region=SlideShowTopBar&version=SlideCard- 3&action=Click&contentCollection=Blogs&slideshowTitle=The%20Future%20of%20Digital%20Photography&currentS lide=3&entrySlide=1&pgtype=imageslideshow Photography of the Future?

References d_heres_why.html operation/ccd-vs-cmos-whats-the-difference.html#b

Q & A