1. Design of a Novel Bridge to Interface High Speed Image Sensors In Embedded Systems Tareq Hasan Khan ID: 11083577 ECE, U of S Term Project (EE 800)

2. 2 Outline Motivation Motivation The Image Sensor Interface The Image Sensor Interface Proposed Bridge Proposed Bridge Sensor Control Sensor Control I2C Interface I2C Interface Memory Addressing and Control Memory Addressing and Control Random Access Memory Random Access Memory Clock Generator Clock Generator Results Results Conclusion Conclusion

3. 3 Motivation Problems of Interfacing High-speed Image Sensors in Embedded System Problems of Interfacing High-speed Image Sensors in Embedded System Produce data at such a high rate that it cannot be processed in real time Produce data at such a high rate that it cannot be processed in real time Microcontrollers have limited internal memory and may not be able to store a complete image frame unless external memory is provided Microcontrollers have limited internal memory and may not be able to store a complete image frame unless external memory is provided Data may only be accessed sequentially Data may only be accessed sequentially 2-wire I2C protocol must be implemented in the microcontroller which is an extra overhead 2-wire I2C protocol must be implemented in the microcontroller which is an extra overhead A high speed clock must be provided to the image sensors A high speed clock must be provided to the image sensors Power consumption is typically high Power consumption is typically high Image Sensor Image Processor

4. 4 Motivation Solution… Solution… The image processor can easily initialize the image sensor without implementing the I2C protocol The image processor can easily initialize the image sensor without implementing the I2C protocol The image frame can be captured and stored in the bridge’s internal memory The image frame can be captured and stored in the bridge’s internal memory The pixel values can be accessed by the image processor in a random access fashion through a parallel memory access interface at required speed for processing The pixel values can be accessed by the image processor in a random access fashion through a parallel memory access interface at required speed for processing To reduce the power consumption, the bridge will automatically send sleep mode command to image sensor after an image is captured To reduce the power consumption, the bridge will automatically send sleep mode command to image sensor after an image is captured A clock generator is also included in the bridge to feed the clock signal for the image sensor A clock generator is also included in the bridge to feed the clock signal for the image sensor Image Sensor Bridge Image Processor

5. 5 Image Sensor Interface Commercial image sensors (such as Toshiba, OmniVision) sends image data using a common standard interface known as digital video port (DVP) parallel output interface Commercial image sensors (such as Toshiba, OmniVision) sends image data using a common standard interface known as digital video port (DVP) parallel output interface The initialization and configuration of the image sensor is done by the 2-wire (SCL and SDA) I2C protocol. In the context of image sensor, it is often called as Serial Camera Control Bus (SCCB) interface The initialization and configuration of the image sensor is done by the 2-wire (SCL and SDA) I2C protocol. In the context of image sensor, it is often called as Serial Camera Control Bus (SCCB) interface

6. 6 Outline Motivation Motivation The Image Sensor Interface The Image Sensor Interface Proposed Bridge Proposed Bridge Sensor Control Sensor Control I2C Interface I2C Interface Memory Addressing and Control Memory Addressing and Control Random Access Memory Random Access Memory Clock Generator Clock Generator Results Results Conclusion Conclusion

7. 7 The Proposed Bridge

8. 8 Capturing Image using the Bridge At first, image processor asserts the RST pin to high and then make it low. At first, image processor asserts the RST pin to high and then make it low. Then the required image size and color must be selected by the FrameSize(0:1) and RGB/Gray’ pins. Then the required image size and color must be selected by the FrameSize(0:1) and RGB/Gray’ pins. Holding the FrameSize(0:1) and RGB/Gray’ pins to the desired logic, the image capturing process can be started by asserting the Init pin to high. Holding the FrameSize(0:1) and RGB/Gray’ pins to the desired logic, the image capturing process can be started by asserting the Init pin to high. While the image capturing process is going on, the Adr(0:18) and Data(0:7) goes to high impedance state. While the image capturing process is going on, the Adr(0:18) and Data(0:7) goes to high impedance state. After the image capturing process is completed, the FrameReceived pin goes from low to high and the image sensor goes to sleep mode to save power. After the image capturing process is completed, the FrameReceived pin goes from low to high and the image sensor goes to sleep mode to save power. The Adr(0:18), Data(0:7) and RD’ pins can now be used to access any pixel value of the frame at a slower speed and in a random access fashion. The Adr(0:18), Data(0:7) and RD’ pins can now be used to access any pixel value of the frame at a slower speed and in a random access fashion. The process of capturing the next frame with the same configuration can be repeated by asserting ReqFrame pin from low to high. The process of capturing the next frame with the same configuration can be repeated by asserting ReqFrame pin from low to high.

9. 9 Outline Motivation Motivation The Image Sensor Interface The Image Sensor Interface Proposed Bridge Proposed Bridge Sensor Control Sensor Control I2C Interface I2C Interface Memory Addressing and Control Memory Addressing and Control Random Access Memory Random Access Memory Clock Generator Clock Generator Results Results Conclusion Conclusion

10. 10 Sensor Control Used to configure and control different modes of the image sensor. Used to configure and control different modes of the image sensor. FSM implemented FSM implemented Using the I2C interface block, it sends command to the image sensor Using the I2C interface block, it sends command to the image sensor

11. 11 I2C Interface Used to generate the I2C protocol bit-frames in single master mode Used to generate the I2C protocol bit-frames in single master mode I2C allows communication of data between I2C devices over two wires. It sends information serially using one line for data (SDA) and one for clock (SCL) I2C allows communication of data between I2C devices over two wires. It sends information serially using one line for data (SDA) and one for clock (SCL) For our application, the bridge acts as master and the image sensor acts as the slave device For our application, the bridge acts as master and the image sensor acts as the slave device Only the required subset of the I2C protocol is implemented to reduce the logic usage Only the required subset of the I2C protocol is implemented to reduce the logic usage

12. 12 Memory Addressing and Control This block manages the data pins for the image sensor interface This block manages the data pins for the image sensor interface Generates address and control signals for the Memory block of the bridge. Generates address and control signals for the Memory block of the bridge. It implements a 19 bit address-up counter and it is connected with the address bus of the memory. It implements a 19 bit address-up counter and it is connected with the address bus of the memory. The DOUT (0:7) is directly connected with the data bus of the memory. The DOUT (0:7) is directly connected with the data bus of the memory. In the valid data state, at each negative edge event of DCLK, the address up-counter is incremented, and at each positive edge event of DCLK, WR’ signal for the memory is generated. In the valid data state, at each negative edge event of DCLK, the address up-counter is incremented, and at each positive edge event of DCLK, WR’ signal for the memory is generated. After a complete frame is received, the address up-counter is cleared and FrameReceiced signal is asserted high. After a complete frame is received, the address up-counter is cleared and FrameReceiced signal is asserted high.

13. 13 Random Access Memory A 512KB single port random access memory block is used to store a frame A 512KB single port random access memory block is used to store a frame

14. 14 Clock Generator This block generates the 20 MHz clock at the EXTCLK pin, which must be fed in the image sensor This block generates the 20 MHz clock at the EXTCLK pin, which must be fed in the image sensor An 800 KHz clock is also required to generate the clock for the I2C Interface and the Sensor Control block. The 800 KHz clock can be generated by dividing the EXTCLK by 25 using a mod-25 counter An 800 KHz clock is also required to generate the clock for the I2C Interface and the Sensor Control block. The 800 KHz clock can be generated by dividing the EXTCLK by 25 using a mod-25 counter

15. 15 Outline Motivation Motivation The Image Sensor Interface The Image Sensor Interface Proposed Bridge Proposed Bridge Sensor Control Sensor Control I2C Interface I2C Interface Memory Addressing and Control Memory Addressing and Control Random Access Memory Random Access Memory Clock Generator Clock Generator Results Results Conclusion Conclusion

16. 16 Result The proposed bridge was modeled in VHDL. The proposed bridge was modeled in VHDL. A model for the image sensor implementing the DVP interface and an image processor, which access the pixels using the proposed bridge, was also written. A model for the image sensor implementing the DVP interface and an image processor, which access the pixels using the proposed bridge, was also written. The model was simulated using ModelSim and by observing the simulation timing waveforms, the correctness of the functional behavior of the bridge was verified. The model was simulated using ModelSim and by observing the simulation timing waveforms, the correctness of the functional behavior of the bridge was verified.

17. 17 Synthesis Result The maximum allowable frequency for DCLK can vary from 76.5 MHz to 244.4 MHz. The maximum allowable frequency for DCLK can vary from 76.5 MHz to 244.4 MHz. The area utilizations are almost the same for all the devices, however, Xilinx Virtex5 occupies lesser LUT. The area utilizations are almost the same for all the devices, however, Xilinx Virtex5 occupies lesser LUT. The Memory block can be connected with the embedded RAM blocks of the FPGA device, and the embedded clock generators of the FPGA can be used to generate the required clock signals. The Memory block can be connected with the embedded RAM blocks of the FPGA device, and the embedded clock generators of the FPGA can be used to generate the required clock signals.

18. 18 Conclusion In this work, the design of a novel bridge is proposed to interface high speed image sensors with low power and low speed embedded systems. In this work, the design of a novel bridge is proposed to interface high speed image sensors with low power and low speed embedded systems. By using the proposed bridge, a frame can be captured and stored in the bridge’s memory easily without implementing any I2C protocol in the image processor. Then the pixel values can be accessed by the processor through a parallel memory access interface at desired speed and in a random access fashion. By using the proposed bridge, a frame can be captured and stored in the bridge’s memory easily without implementing any I2C protocol in the image processor. Then the pixel values can be accessed by the processor through a parallel memory access interface at desired speed and in a random access fashion. The bridge can support data rate of image sensors up to 244.4 MHz in Xilinx Vertex 5 FPGA. The bridge can support data rate of image sensors up to 244.4 MHz in Xilinx Vertex 5 FPGA.

19. 19 Thanks

20. 20 Simulation Waveforms (1/2)

21. 21 Simulation Waveforms (2/2)