Low cost FPGA implimentation of tracking system from USB to VGA Characterization presentation Presented by: Alon Tirosh & Jonathan Ezroni Supervisor: Mike Sumszyk

Agenda Project Objectives. SOPC Overview. NIOSII Performance. Timing Considerations. System Architecture. Tracking block interface. Algorithm. System Inputs & Outputs. Memory Considerations. Timeline.

2.Project Objectives Configure SoPC including: I/O, memory, processor, bus and peripherals. Implement tracking algorithm via code in C running on Nios II softcore and via verilog code. The hardware implementation will deal with the computing demanding part of the algorithm. Produce moving red box enclosing the moving object and aggregate it to the input video stream. This is a one semester project, focusing on implementing a tracking algorithm on a simple 50$ board. Project may be extended to be bi- semesteral focusing on the algorithm implementation.

Price Comparison- comparing to an implementation showcased on the TI web site =5 boards X 2 TI DSP’S 10 x 27.15= 271.5 $

SoPC Overview DE2 Altera board. Cyclone II FPGA. Nios 2 Softcore. Avalon Switch fabric. Environment: SoPC builder. Quartus II. IDE II. ModelSim.

3.Nios II Performance

4.Timing considerations Add Timing segment at beginning of architecture(FIFO). VGA output works at 30 bits/clk. 160x120x15=188000clks/sec~0,3Mhz NIOS MIPS/0.3Mhz = 55/0.3= 183. ~ 183 assembly operations- Instructions per pix USB is asynchronous, so external PC will keep the synchronization.

5.System Architecture 15 frames per sec video stream DE2 Board Philips ISP1362 (USB 2.0- device) Cyclone II Nios II Soft Processor Host Avalon Switch Fabric VGA Controller On Chip Memory Custom HW

5.1 Data Flow Finished Int from Philips device controller Idle Start running C code Int from Philips device controller Run custom part on chip Philips ISP1362 Cyclone II Nios II Soft Processor Avalon Switch Fabric VGA Controller On Chip Memory Custom HW block

5.2 Data Flow-cont. USB Controller PC DMA Custom HW SDRAM DMA NIOS DMA RGB ->BW Subtract between 2 pixels Update of center of mass coordinates Integrate to NIOS Video FIFO ADD Box Boundaries VGA Controller VGA D/A DMA Double frame One frame

6.Custome HW block interface Through Avalon bus, VIA legacy PIO modules. Tracking block Data in Data out Interrupt Start Avalon Bus

Tracking calculations 7.Algorithm Tracking calculations Input stream t+1 t t-1 Memory space VGA Adapter

8.Inputs and Outputs Inputs: Video feed input: PC (USB HOST) is responsible to create constant 15 FPS video stream. VGA Controller includes 3 10 bit D/A’s Dot Rate = (Horiz Res)x (Vert Res) x (Refresh Rate)/(Retrace Factor)= 120 [pix]x160 [pix]x 15 [FPS] 120x160x15= 288,000[pix/sec] Unsynchronized inputs from USB and to output video stream will be solved using FIFO’s.

9.Memory considerations Cyclon II has 483Kbits for our use. After compilation of a basic design, of the basic NIOS II and 32K ram, only 211Kbits left(less than one frame), so there is need for other memory resources. Flash memory is very slow in write cycles, and with limited read/write cycles ~ 100K (less then 2 hours of video) so it’s not an option long term usage. 0.5Mbyte SRAM is not enough for possible extended resolution(640x480). 8Mbyte of SDRAM will be enough for: 160x120x3x3 = 172,800Bytes which is the current plan. Extended resolution or more data that will be used for calculations.

10.Timeline Comments Due date Milestone TBD 17/12/08 24/12/08 31/12/08 Ramp-up : Image to SoPC 24/12/08 Algorithm C++ imp. 31/12/08 Veriog imp. 07/01/09 Final debug, improvements ??? Final Presentation