Autonomous Tracking Unit John Berglund Randy Cuaycong Wesley Day Andrew Fikes Kamran Shah Professor: Dr. Rabi Mahapatra CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Project Goals –Develop a 2-dimensional servo control system –Develop a camera platform –Implement an FPGA to camera interface –Develop a memory and camera control system –Develop a motion tracking algorithm –Implement the algorithm in hardware –Integrate components –Graduate Project Status Objective –Develop an autonomous camera that can identify and track an object in motion CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

System Diagram SRAM Algorithm Camera Interface Camera Servo Control Memory & Camera Control Servos CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Servos: Pulse Width Modulation CPSC Spring 1999 Autonomous Tracking Unit 17-20ms 0º160º 1.25ms 1.75ms CPSC Spring 1999 Autonomous Tracking Unit

8-bit input angle with 9-bit internal counter Test the MSB for minimum pulse length Minimum pulse length => 1|0000|0000 b - minimum number of pulses Servos: Final Design CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Servo Control Components CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Camera Interface II Memory and Camera Control Camera Interface Module QuickCam Start Hold Send Valid Data (8) Nibble (4) PCAck Reset Command CamRdy Command (8) Remove asynchronous data exchange Reduce complexity of controlling logic Provide a configurable initialization sequence Goals: CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Camera Interface II Components CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Camera Interface II Success! Testing Issues –BSII cannot trigger on an edge of a signal –Camera’s CCD decays at a rapid rate Performance Measurements –Store a 128x120 image in 32 ms (31 fps) Future Improvements –Implement the byte mode transfer –Implement memory storage logic inside the interface CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Memory & Camera Control (MCC) “Memory System: This will involve implementing an SRAM interface that will store image representations and allow for them to be accessed by other system components.” CPSC Spring 1999 Autonomous Tracking Unit Isolates the Algorithm from the details of memory access and camera control This module went through several design changes

Original MCC Design CPSC Spring 1999 Autonomous Tracking Unit Asynchronous communications with the Algorithm and the Camera Control Module 256x243 pictures and 6-bit pixels Expected to use 256K of memory to store 3 frames Did not unpack the pixels Complex initialization sequence Tested in simulation CPSC Spring 1999 Autonomous Tracking Unit

MCC Final Design Frame (3 bit)Vertical Offset (7 bits)Horizontal Offset (6 bits) Final Memory Addressing Scheme CPSC Spring 1999 Autonomous Tracking Unit Fifth and final design One bit Command Streams data from the camera synchronously Camera Control module writes directly to memory 64K of addressable memory, 8K needed Reduced frame size to 128x120 CPSC Spring 1999 Autonomous Tracking Unit

Tested with BS2 and Camera Control module Observed inconsistent results We made a separate project to unit test the MCC Worked almost perfectly in testing Needed to the camera’s cord! MCC Final Design (Continued) CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

CPSC Spring 1999 Autonomous Tracking Unit MCC Final Block Diagram CPSC Spring 1999 Autonomous Tracking Unit

Our “Simple” Environment Single rigid object Object is darker than its background Object is smaller than its background Object is travelling at a reasonable pace CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Algorithm Test & Development Goal was to develop and test several algorithms Use software to help us visually compare solutions Tests were used to compare performance in different scenarios CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Algorithm Step 1: Create a histogram of the object Step 2: Locate edges by subtracting Step 3: Calculate the center of the object Step 4: Adjust camera CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Algorithm Issues Problems: –No object in frame –Partial object in frame –Handshaking timing issues Solutions: –No object: Use buffers to filter out non-significant deltas –Partial object: Make frame extremes default boundaries of the object –Timing issues: Add more states to ensure correct propagation CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

System Integration Phase 1: Integrate Camera Interface Module and Memory & Camera Control Module –Simulated in Xilinx –BSII used to simulate algorithm module –Capture data with PC serial port Phase 2: Algorithm Module Revision –Image simulated with a Xilinx stub module –Unit integrated with one dimensional movement –Two dimensional movement implemented after testing horizontal and vertical tracking individually CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

System Integration Phase 3: Performance Improvement. –Initial unit integration with the Algorithm Module on a 500kHz clock –Increasing Algorithm Module clock resulted in choppy movement, due to Servo operation –Frame clock used to synchronize Servo operation –Decreasing the frame clock period from 20.2ms to 17.0ms increased the image processing rate CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit

Performance (Frame Clock = 20.2 ms) CPSC Spring 1999 Autonomous Tracking Unit 16.6 images per second CPSC Spring 1999 Autonomous Tracking Unit

Performance Improvement (Frame Clock = 17 ms) CPSC Spring 1999 Autonomous Tracking Unit 19.7 images per second 18.7% improvement CPSC Spring 1999 Autonomous Tracking Unit

Possible Improvements Use 2 SRAM chips simultaneously –Image is retrieved and processed from one chip while the next image is being stored –Pin limitations Implement entire unit in Xilinx –Use internal RAM to store histogram information – bit storage locations required –45 CLBs required to store histogram information CPSC Spring 1999 Autonomous Tracking Unit CPSC Spring 1999 Autonomous Tracking Unit