Digital Camera Design

Agenda Digital video formats Image sensor technology Sensor interface with CoolRunner-II LCD CoolRunner-II system design

Digital Video RGB – Digitized version of analog RGB – Most common in computer graphics YCrCb – Digitized version of analog YUV and YPbPr – Format used by DVD and digital television

Color Formats RGB – Primary additive colors: red, green, & blue – Not efficient format for image processing YCrCb – Luminance, Chroma Red, Chroma Blue – 8-bit ranges: Y (16 to 235), Cr & Cb (16 to 240) – 4:2:2 & 4:4:4 formats determine when Cr and Cb are sampled with Y

Digital Video Capture Video is series of still images (which each consist of lines of data) Control signals: – VSYNC: Vertical sync (asserted during frame scan time) – HSYNC: Horizontal sync (asserted during line scan time) – BLANK: blanking – CLK: 1x or 2x sample clock Digital output represents data of each pixel (RGB or YCrCb) Data sampled on rising edge of sample clock

Output Timing

Image Sensors CCD (charge-coupled device) vs. CMOS CMOS manufacturers: Kodak, Micron, Hynix, OmniVision, Mitsubishi, ST Microelectronics, Toshiba

MI-SOC-0343 Complete CMOS image sensor camera-on-chip solution “Active pixel” sensor architecture Image sensor core + digital image processing technology Outputs digitally processed RGB or YCrCb data Programmable control registers 640 x 480 VGA image array

Active Pixel Architecture Photodiode Active-Pixel ArchitecturePhotogate Active-Pixel Architecture CMOS sensor photosite area breakdown 25% = circuitry Sensor fill factor = 75 % Active pixel structure prevents background image noise Image courtesy of Micron

Sensing Color Funnel light to the photosensitive portion of each pixel CFA allows each RGB color to be measured independently Image courtesy of Micron

Bayer CFA Invented by Kodak Able to separately measure red, green, & blue photons 2x2 repeating arrangement Output: sequential RGB (sRGB) G R G R B G B G G R G R B G B G G R G R B G B G Image courtesy of Micron

MI-SOC-0343 Diagram Active Pixel Array (640 x 480) Image Core Register Set Analog Processing ADC Imager Core Image Flow Processor Register Set Data Output & Timing Image Flow Processor Serial Interface (SCLK, SDATA) Color Correction Gamma Control Sharpening Control Saturation Control AWB AE Defect Correction Lens Detection

Image Flow Processor Interpolates 1 color/pixel into 3-colors/pixel by filling in missing data based on adjacent pixels Includes: lens shading, edge detection, aperture correction, color correction, AWB, AE, gamma correction, color saturation control, and zoom features 77 addressable control registers Output either 4:2:2 YCrCb (CCIR656) or 4:4:4 565RGB data

CoolRunner-II Interface MI-SOC-0343 SCLK, SDATA SHIP Interface Control Logic Image Grabber Control Logic FRAME_VALID LINE_VALID PIX_CLK SRAM Interface Logic SRAM Address / Data / Control Address Counter Top Level Control Logic DOUT (7:0) 24 MHz Clock

LCD Best LCD solution for video application: color active matrix TFT Transmissive or transflective polarization type TFT modules offer sub-pixels (3 per pixel) to represent each RGB color LCD (with an integrated driver) accessible via parallel data interface

Optrex LCD 6.4” Transmissive Color TFT Dual CCT backlight 640 x (RGB) x 480 (VGA format) Thin form factor Parallel 6-bit RGB data interface LCD Panel (640 x 480) Timing Logic Vertical IC Driver CCT Horizontal IC Driver CCT

CoolRunner-II Interface Optrex T-51382D LCD Interface Control Logic VSYNC HSYNC CLK SRAM Interface Logic SRAM Address / Data / Control Top Level Control Logic R (5:0) G (5:0) B (5:0) PWM Control Logic K2607 Inverter

System Design LCD Panel LCD Interface Control Logic SRAM Interface Logic SRAM Main Control Logic PWM Logic Inverter CMOS Image Sensor SHIP Interface Control Logic Image Grabber Control Logic

Cell Phone Camera Expansion CMOS Image Sensor uP KEYBOARD DISPLAY DRAM EPROM SRAM DSP D/A A/D SPEECH CODER SPEECH DECODER CHANNEL CODER CHANNEL DECODE CHANNEL SPREAD RAKE RCVR RF MOD RF DEMOD DUPLEXER RF AMP RF RCV/AMP

Summary CoolRunner-II allocation – Design uses a 384 macrocell device (with approx. 80% utilization) – Varies based on internal ROM size for CMOS image sensor register writes CoolRunner-II is ideal for digital camera applications in portable and handheld devices – Provides a low power flexible interface with CMOS image sensors – Provides a low power data allocation and memory resource solution CoolRunner-II can be used to extend existing design (ie. PDA, cell phone) and add digital camera functionality