MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective.

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. I mage P rocessor U nit Nicolas WALCZAK

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Agenda Presentation Architecture Performances & limitations “IPU only” use cases –Display a JPEG picture stored in a SD Card –Viewfinder –Video call “IPU/GPU” use cases –Direct access from MCU –DMA access to GPU internal memory

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Presentation Displays Memory Image Sensors TV Encoder Graphics Accelerator IPU EMI ARM11 CPU Argon LVLT

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Presentation –IPU: I mage P rocessing U nit –The role of IPU is to limit the involvement of SW in video tasks –Linux IPU drivers are already written for SCM-A11 and as the IPU is the same at 95% between Argon LV and SCMA11 so the Argon Linux drivers will be quickly available. A+ P2KLV P2KLV LinuxLVLT Linux Products Carina, Scorpius…Volans, Rocket…Kassos…None GPU SC4/SC12 SC14/SC15None (IPU)

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Agenda Presentation Architecture Performances & limitations “IPU only” use cases –Display a JPEG picture stored in a SD Card –Viewfinder –Video call “IPU/GPU” use cases –Direct access from MCU –DMA access to GPU internal memory

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Smart displays Architecture Dumb displays Camera IPU CSI C MOS S ensor I nterface DI Display Interface SDC Synchronous Display Controller IC Image Converter ADC Asynchronous Display Controller IDMAC Image DMA Controller SDRAM

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Architecture IPU CSI DI SDC IC ADC IDMAC CSI: CMOS Sensor Interface Provide the MCLK to the camera Receive frames from the camera on the 10-bit data bus IC: Image Converter Color space conversion Resizing Rotation, Flip Post-Filtering IDMAC: Image DMA Controller ADC: Asynchronous Display Controller SDC: Synchronous Display Controller DI: Display Interface Provide the display MCLK and Ver/Hor synchros signals according displays specification timings Low Level Access for Smart displays configuration

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. IDMAC dedicated channels (1/2) NumberNameSourceDestinationProcessing Flow Purpose 0DMAIC_0ICMemoryPreprocessing data from IC (encoding task) to memory 1.A1DMAIC_1ICMemoryPreprocessing data from IC (viewfinder task) to memory 1.BDMAADC_0ICADCPreprocessing data from IC (viewfinder task) to smart display 2.A2DMAIC_2ICMemoryPostprocessing data from IC to memory 2.BDMAADC_1ICADCPostprocessing data from IC to smart display 3DMAIC_3MemoryICGraphics data for combining (viewfinder task) 4DMAIC_4MemoryICGraphics data for combining (post-processing task) 5DMAIC_5MemoryICPostprocessing data from memory 6DMAIC_6MemoryICPreprocessing data from sensor stored in memory (for example Bayer) 7DMAIC_7ICMemoryDirect data from IC (sensor data) to memory 8DMAIC_8ICMemoryMemory Preprocessing data after rotation (encoding task) 9DMAIC_9ICMemoryMemory Preprocessing data after rotation (viewfinder task) 10DMAIC_10MemoryICPreprocessing data for rotation (encoding task) 11DMAIC_11MemoryICPreprocessing data for rotation (viewfinder task) 12DMAIC_12ICMemoryPostprocessing data after rotation 13DMAIC_13MemoryICPostprocessing data for rotation 14DMASDC_0MemorySDCBackground data (full refresh) 15DMASDC_1MemorySDCSDC Foreground data

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. IDMAC dedicated channels (2/2) NumberNameSourceDestinationProcessing Flow Purpose 16DMASDC_2MemorySDCMask data 17DMASDC_3MemorySDCBackground data (partial refresh) 18DMAADC_2MemoryADCSystem channel 1 write data 19DMAADC_3MemoryADCSystem channel 2 write data 20DMAADC_4MemoryADCCommands stream for system channel 1 21DMAADC_5MemoryADCCommands stream for system channel 2 22DMAADC_6 ADCMemorySystem channel 1 read data 23DMAADC_7 ADCMemorySystem channel 2 read data 24DMAPF_0MemoryPFPF parameters (quantization parameters for MPEG-4 and H.264 and filter offsets for H.264) 25DMAPF_1MemoryPFPF parameters (boundary strength for H.264) 26DMAPF_2MemoryPFY input data 27DMAPF_3MemoryPFU input data 28DMAPF_4MemoryPFV input data 29DMAPF_5PFMemoryY output data 30DMAPF_6PFMemoryU output data 31DMAPF_7PFMemoryV output data

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Agenda Presentation Architecture Performances & limitations “IPU only” use cases –Display a JPEG picture stored in a SD Card –Viewfinder –Video call “IPU/GPU” use case –Direct access from MCU –DMA access to GPU internal memory

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Performances  Displays interfaces:  Asynchronous and Synchronous interface  Several screen resolutions (240*320, 176*220, …)  Pre-Processing:  Downsizing, Resizing, CSC, blending, rotation  Post-Processing:  From External Memory  Downsizing, Resizing, CSC, Rotation, Flip…  Post-Filtering:  Deblocking/Deringing for MPEG4 & H264

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Limitations  No frame buffer  No decompression (JPEG or MPEG…)  No compression (JPEG or MPEG…)

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Conclusion Current MDB choice is to use:  LV + GPU for Medium and High tier products  LV-LT + IPU for Low tier products ! Unfortunatly: LV-LT MCU PLL runs only to 312MHz so the MCU load is too important to perform video tasks.

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Display a JPEG picture stored in a SD card (1/4) Image Processing Unit ARM11 SW Memories Camera Display Bayer or YUV IPU RGB Image Conversion CSI up to 10 Image DMA Controller EMI ADC SDC DI SDRAMSDHC SSI ATLAS ARGON ARM11 Encode JPEG Decode JPEG Encode MPEG Decode MPEG Encode MP3 Decode MP3 Separate Audio and Video Mux Audio and Video SDMA

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. up to 10 Viewfinder (1/2) Image Processing Unit ARM11 SW Memories Camera Display Bayer or YUV IPU RGB Image Conversion CSI Image DMA Controller EMI ADC SDC DI SDRAMSDHC SSI ATLAS ARGON ARM11 Encode JPEG Decode JPEG Encode MPEG Decode MPEG Encode MP3 Decode MP3 Separate Audio and Video Mux Audio and Video SDMA

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. up to 10 Viewfinder (2/2) Image Processing Unit ARM11 SW Memories Camera Display Bayer or YUV IPU RGB Image Conversion CSI Image DMA Controller EMI ADC SDC DI SDRAMSDHC SSI ATLAS ARGON ARM11 Encode JPEG Decode JPEG Encode MPEG Decode MPEG Encode MP3 Decode MP3 Separate Audio and Video Mux Audio and Video SDMA

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. up to 10 Video call : emission (1/5) Image Processing Unit ARM11 SW Memories Camera Display Bayer or YUV IPU RGB Image Conversion CSI Image DMA Controller EMI ADC SDC DI ARM11 Encode JPEG Decode JPEG Encode MPEG Decode MPEG Encode MP3 Decode MP3 Separate Audio and Video Mux Audio and Video SDRAM Network SDMA SSI ATLAS ARGON

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Video call : reception (1/6) Image Processing Unit ARM11 SW Memories Camera Display Bayer or YUV IPU RGB Image Conversion CSI up to 10 Image DMA Controller EMI ADC SDC DI ARM11 Encode JPEG Decode JPEG Encode MPEG Decode MPEG Encode MP3 Decode MP3 Separate Audio and Video Mux Audio and Video SDRAM Network SDMA SSI ATLAS ARGON

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Direct access to GPU from MCU Image Processing Unit ARM11 SW Memories IPU Image Conversion CSI Image DMA Controller EMI ADC SDC DI ARM11 SDRAM ARGON GPU

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. DMA Access to GPU internal memory Image Processing Unit ARM11 SW Memories IPU Image Conversion CSI Image DMA Controller EMI ADC SDC DI ARM11 SDRAM ARGON GPU

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Thanks Questions?

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Backup Slides

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. LV vs LV-LT IPU On LV-LT 9 pins have been removed on the display interface: –IPU_VSYNC0 (Synchro display type 0) –IPU_LCS0 (CS for display type 0) –IPU_CONTRAST1 –IPU_SD_D_O (read data line in case of 5 wires serial interface) –IPU_WRITE –IPU_READ, –IPU_D3_CLS –IPU_D3_SPL –IPU_D3_REV On camera interface only a 8-bits bus is now supported HANTRO is not present on LV-LT

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Performances De-blocking and De-ringing –Post-filtering for MPEG-4 – proprietary algorithm –In-loop de-blocking for H.264 - as specified in the standard –Rate up to 9M pixels/sec, e.g., VGA @ 30 fps Resizing –Fully flexible resizing ratio –Rescaling of image from sensor is performed on-the-fly, without buffering in system memory –Rescaled image can be sent to display without MCU intervention Color space conversion –Fully flexible: YUV RGB; YUV YUV; color adjustments and special effects Enables up to three independent simultaneous flows: –Sensor to Display (e.g. viewfinder) –Sensor to Memory Buffer (e.g. encoding) –Memory Buffer to Display (e.g. playback) Inversion – horizontal and vertical 90-degree Rotation

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Performance Numbers for JPEG Encode and Decode in Mcycles per frame EncodeDecode VGA11.78.4 1.2 Mpxl49.835.8 3.0 Mpxl114.181.9

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. ARM11 MJPEG Encode Performance 532 3 Mpxl 1.2 Mpxl VGA

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Argon vs. Nvidia SC12 – Performance Argon With no communication Argon With 3.6Mbps HSDPA SC12 Video Playback – MPEG4-SP (including post-filtering) VGA @ 30 fpsVGA @ 15 fpsCIF @ 30 fps Video Playback – H.264-BP, WMV, RVVGA @ 15 fps No Video Record – MPEG4-SPCIF @ 30 fps Video Record – H.264-BPCIF @ 15 fpsQCIF @ 30 fpsNo Two-Way Video – MPEG4-SPCIF @ 30 fps CIF @ 15 fps Two-Way Video – H.264-BPQCIF @ 30 fps No Two MPEG4-SP Video Encodes: 30&15 fpsCIF No Two H.264-BP Video Encodes: 30&15 fpsQCIF No View Finder3 MP @ 10 fps; 2 MP @ 15 fps; 1 MP @ 30 fps Motion JPEG Encode3 MP @ 4 fps3 MP @ 3 fpsVGA @ 30 fps 2D GraphicsYes 3D Graphics 90 KTri/sec 80 MP/sec 70 KTri/sec 60 MP/sec Yes Assuming –ARM11 @ 532 MHz (Turbo mode) –Non video tasks consume ~70 MHz –HSDPA, 3.6 Mbps consumes ~120 MHz

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Argon vs. Nvidia SC12 – Features ArgonSC12 Camera port Data bus10, 8, 4-bit Data rate72MHz = 30MPix/sec + 20% overhead ISP (Bayer processing)YesNo Two sensor supportYes (not simultaneous)No (but possible using Argon controls) Display port ResolutionVGA Data bus up to 18-bit Multiple displaysRAM-less + smart TV + smartMain + Sub(Smart) Internal RAMNo1280 KB Processing ResizingYes, Fully flexibleYes, flexible fine grain from 1/60 to 8 Color conversion and pixel packing Yes, Fully flexible (Incl. YUV YUV conversion, color adjustment and special effects) YUV4:2:2 to RGB 565 Combining with graphicsYes, global/per-pixel alphaYes Inversion and rotationYes (for video)Yes

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Image Capture Rate Bus WidthBayerYUV 4:2:2RGB/YUV 4:4:4 4-bit (nibble mode) 30 Mpixels/sec; e.g.: 1 Mpixels @30fps 2 Mpixels @15fps 15 Mpixels/sec; e.g.: 1 Mpixels @15fps 2 Mpixels @7.5fps 10 Mpixels/sec; e.g.: 1 Mpixels @10fps 2 Mpixels @5fps 8-bit60 Mpixels/sec; e.g.: 2 Mpixels @30fps 4 Mpixels @15fps 30 Mpixels/sec; e.g.: 2 Mpixels @15fps 4 Mpixels @7.5fps 20 Mpixels/sec; e.g.: 2 Mpixels @10fps 4 Mpixels @5fps 10, 12, 16-bit30 Mpixels/sec; e.g.: 1 Mpixels @30fps 2 Mpixels @15fps 15 Mpixels/sec; e.g.: 1 Mpixels @15fps 2 Mpixels @7.5fps 10 Mpixels/sec; e.g.: 1 Mpixels @10fps 2 Mpixels @5fps Notes: Maximal pixel clock rate: 72 MHz Overhead assumed due to blanking intervals: 20% For a bus wider than 8 bits, the capturing rate is limited by the memory bus bandwidth. For Bayer input, the capturing rate is limited further by the image signal processing.

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Video encoder in Argon Rate: –Pixel rate: up to CIF @ 30 fps –Compressed bit-rate: up to 384 Kbps 500+MCPS –Adds 500+MCPS of equivalent ARM11 performance Compression format: –MPEG-4 simple profile – levels 0-3 –H.263 baseline (P0) – levels 10-30 Additional features: –Pre-processing: Picture smoothening (low-pass filter - patented technology) –Camera movement stabilization (patented technology)

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. JPEG compression Convert RGB to YUV: Y = 0.30 R + 0.59 G + 0.11 B U = 0.60 R - 0.28 G - 0.32 B V = 0.21 R - 0.52 G + 0.31 B Filter low frequencies = low colors differences A big compression will introduce 8x8 blocks called “artefacts “. 40x40pixels 5x5 blocks

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. De-Blocking

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Display port Provides a connection to a variety of display devices –RAM-less LCD panels - up to VGA 100 fps, 262K colors –LCD panels with integrated frame buffer - up to 1024x1024, 16M colors –Graphics accelerators –TV encoders Interface types –Parallel – data bus up to 18 bits, bus rate up to 133 MHz –Serial (SPI) - bus rate up to 100 MHz Access modes –Synchronous Generates timing signals for screen refresh and sends pixel data to the display Used for memory-less LCD panels, TV encoders and dual-port smart LCD panels Only parallel interface; up to 30M pixels/sec –Asynchronous The primary interface to an external display controller or graphics accelerator –Data transfer (DMA) – read/write – between system memory and external device –Direct access – read/write – of the ARM11 to the external device Interleaved address and data Simultaneous Functionality –Connectivity: up to two displays (one may be a memory-less LCD) and a TV encoder –Functionality: subject to restrictions by time-sharing the bus and interfaces Examples: two displays; TV + smart displays

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners. © Motorola, Inc. 2003. Motorola Confidential Proprietary Add additional legal text here if required by your local Legal Counsel. Image Sensor port Provides a connection to one or two image sensors Parallel Interface: 10-bit, 8-bit or 4-bit data bus Data formats: –YUV, RGB, 10 or 8 bits/value processed in the IPU (e.g. view-finder generation) –Generic (Bayer, compressed…) transferred directly to memory, for ARM procession Data rate: up to 30M pixels/s –VGA @ 30+ fps (real time) –3M pixel @ 10 fps Camera flash support Timing methods –Using dedicated timing signals: Vsync, Hsync –Timing embedded in the data stream following (loosely) the ITU-R BT.656 standard, progressive or interlaced order

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective.

Similar presentations

Presentation on theme: "MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective.

Similar presentations

Presentation on theme: "MOTOROLA and the Stylized M Logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective."— Presentation transcript:

Similar presentations

About project

Feedback