Image Denoising 成功大學 資訊工程系 陳 培 殷

Outline Embedded System What CSE Students Can Do in Embedded System Design Example -Image Denoising Demo

PCB PC: a general-purpose computing system PC Pentium

PCB Embedded System: a special-purpose computing system Most embedded systems are designed for 1.special purpose (customizing and non-programmable) 2.real time applications 3. stable applications 4. automatic applications Embedded System (1/2)

uP UART MPEG ROM RAM Embedded System (2/2) Traditional embedded systems use low-level processors only. ARM PCI MPEG USB FLASH ROM RAM DSP AMBA Advanced embedded systems Multi-core

Application Everywhere!

Applications Information Appliances (IA): 1. Smart phone, VOIP 2. Digital TV, set-top box 3. PlayStation 4. PDA, mp3 player 5. Camera, DV 6. Air-conditioner, microwave oven, refrigerator, vacuum cleaner, sensor network 7. Motorcycles 8. Car (abs, engine firing, air bag) >100 processors 9. … Ubiquitous computing (many computers for everyone)

Requirements 1. Friendly user interface 2. Short time-to-market 3. Real-time (Speed) 4. Cost 5. Power consummation/dissipation (cooling strategy and battery life)

Synthesis Applications 傳輸距離： 100 m 功率消耗： 2 W 功能：影像傳輸、 語音傳輸 螢 幕： 176 x 220 pixel 色 1.8 吋 TFT 其他 Specifications Hardware: CPU, RAM, I/O… Hardware: CPU, RAM, I/O… Software: C,C++ Software: C,C++ System Design clk) begin if (sel1) begin out=in1; else out=in2; end Component Design Layout Placement & Routing Fabrication Marketing System Development Flow Testing

What CSE students can do in Embedded System Design? My personal Views: 1. Hardware Design -Chip 2. Hardware Design -System 3. Related Tools Design 4. Firmware Design 5. Software Design 6. ESL Design

Design Flow Specification System Architecture Hardware Design Software Design System Integration System Verification/Testing Hardware/software partition is very difficult !!! (cost  time)

IC Industry in Taiwan 晶粒測試及切割 晶圓切割 設計 導線架 測試 封裝 製造 光罩 晶圓 邏輯設計 封 裝 化學品 成品測試 光罩設計 長晶

ARM PCI MPEG USB FLASH ROM RAM DSP AMBA Hardware Design -- Chip (1/4) ASIC The basic design flow for digital cell-based ASIC Describe circuits with hardware description language (HDL 硬體描述語言 )  VHDL and Verilog Synthesis ( 合成 ) the circuits …. Need the concepts of digital logic/system application specific integrated circuit (ASIC 晶片 ) such as USB, MPEG, …. Full custom design vs. semi custom (cell-based) design

Hardware Design -- Chip (2/4)

Hardware Design -- Chip (3/4) if (a==b) if (c==1) d=f; else d=1; else d=0; a b c f d f c a b d Translate into Boolean Representation Optimize + Map HDL Source Target Technology Synthesis = Translation+Optimization+Mapping Process of logic synthesis Software Engineers play a very important role for EDA tools development !!!

Hardware Design -- Chip (4/4) FPGA or CPLD Real ASIC chip less flexible, long design cycle, higher speed, larger-scale production to reduce price more flexible, shorter design cycle, lower speed, lower utilization, suitable for smaller production Standard cell PLD Fab (TSMC, UMC,..) Two implementations : Xilinx, Altera

Hardware Design -- System ARM PCI MPEG USB FLASH ROM RAM DSP AMBA ASIC Input devices: keyboard, touch screen, switch, button,.. Output devices: monitor, LCD, LED, … Extended devices: compact flash card (CF), PCMCIA, SD (for storage, wireless communication, I/O) Power system: Transmission Interface: PCI, USB, IEEE 1394, UART, bluetooth… Bus: AMBA ( Advanced Microcontroller Bus Architecture ) Input devices Output devices

Related Tools Design 晶粒測試及切割 晶圓切割 設計 導線架 測試 封裝 製造 光罩 晶圓 邏輯設計 封 裝 化學品 成品測試 光罩設計 長晶

Firmware Design ARM PCI MPEG USB FLASH ROM RAM DSP AMBA ASIC Devices drivers for I/O devices, extended devices, transmission interface Assembly codes and C codes for some dedicated CPUs (ARM, 8086,..)  Architectures and instruction sets of different CPUs, DMA,… Input devices Output devices

Software Design ARM PCI MPEG USB FLASH ROM RAM DSP AMBA ASIC Input devices Output devices Embedded OS: WinCE, Palm OS, uC/OS, Linux, JAVA  Real time OS (time)  as small as possible (memory)  Distributed embedded system (+ fault tolerance) Application Software: wireless communication, network, multimedia, health, convenience, Web, ….  Porting a customized embedded system to different machines is very difficult (need large modification)

Future Chip: tens of millions of transistors or more (.35,.25,.18,.09) Design shifts from ASIC/board to system System on a Board (printed circuit board) System on a chip uP FPGA MPEG ASIC ATM ROM SW PCB uP Core SRAM ROM ATMMPEG ROM FPGA Glue Logic A/D Block PCB SOC System-on-a chip is possible (the whole system is built in a single chip)

SOC is industry trend multi-core

Example: Mobile Phone Voice only; 2 processors 4 year product life cycle Short talk time Yesterday Voice, data, video, SMS <12 month product life cycle Lower power; longer talk time Today 5~8 Processors Memory Graphics Bluetooth GPS Radio WLAN Single Chip DSP Radio Flash Memory Processor Source: EI-SONICS

What’s ESL tools ? Describe a SOC design in a more abstract way. Provide virtual prototypes for HW and SW implementation. Help system integration in early design phase. Help to verify the result after job partition. Claim that design and verification can be done in minutes. Electronic System Level (ESL) Tools Suitable for students in CSE

Hardware Software Co-Simulation SoC Designer

Example: Image Denoising Noise arises During image acquisition CCD camera, light levels, sensor temperature During image transmission Interference in the transmission channel Noise Detection Noise Removal Noise Suppression Process Hardware Design – Chip !

Drawbacks of conventional algorithms Heavy computation loading Iterations Large computation loading Additional memory space Our goals: Cost-effective Good performance Suitable for VLSI implementation Real-time requirement Motivation

Low-Complexity Algorithms Embedded System 1. Power consumption 2. Computing time (real time vs. high quality) VLSI Implementation 1. Circuit cost 2. Circuit speed 3. Power consumption Low-Complexity algorithms are necessary for embedded systems and VLSI implementations.

Issues for VLSI Implementation Circuit Cost fixed point vs. floating point less complexity, moderate quality vs. high complexity, high quality +, - vs. *, / 2*x left shift x 5*x  4*x + x Functional units sharing Circuit Speed Pipeline architecture Delay reduction Power Consumption Sleep Shut down partially

Cost-Effective Denoising Method (1/4) 1.Find Min & Max in the mask 3.Check Suspected Noise 4.Choose eligible & representative edge directions 5.Find the minimum difference 6.Reconstruct the corrupted pixel 2.Utilize Min & Max info. to detect noise Noise DetectionNoise Removal

Check suspected noise Case 1Case 2Case 3Case 4 Cost-Effective Denoising Method (2/4)

Choose eligible & representative edge directions Case 1Case 2Case 3Case 4 Cost-Effective Denoising Method (3/4)

Find the minimum difference and reconstruct the noisy pixel Case 1 Cost-Effective Denoising Method (4/4)

Architecture of the Denoising Chip

Shift Register Bank 1st Raw2nd Raw 1st Raw 2nd Raw 1st Raw 3rd Raw

Dataflow Max & Min Computation Unit Noise Detection Unit Edge Detection Unit Output Selection Unit

Stage 1 Stage 2 Stage 3 Stage 4 Pipeline Stages

Demo Source Image Sending Image Denoising Circuit FPGA Board Receiving Image Result Image Software Program Hardware Platform PC Hardware Software Co-Simulation/Verification PSNP calculation