Acceleration of the Retinal Vascular Tracing Algorithm using FPGAs Direction Filter Design FPGA FIREBIRD BOARD Framegrabber PCI Bus Host Data Packing Design BlockRAM Image Data MEMORY 0 Image Data MEMORY 1 Memory Switching Design Miriam Leeser Shawn Miller Smart Camera:Provides Host PC with image data along with image processing results at frame rate with low latency Results MEMORY 2 Results MEMORY 3 Memory Switching Design

Retinal Vascular Tracing Application Goal: Detection and enhancement of the vascular structure of a patient’s retina from a live video feed Latency and throughput requirements of real-time image processing cannot be provided by software on a general-purpose processor

Timing Issues Return results for each pixel at frame rate of camera Very Low Latency Surgical laser must be shut off immediately after detecting that it is aimed incorrectly Cannot tolerate 1 frame delay (33msec at 30 frames/sec) Complex memory management required to achieve minimum latency Latency currently on the order of 100  sec Storage of a 5x5 pixel image Memory 0Memory 1

Implementation Hardware Acceleration  Template responses are calculated in hardware in parallel  All pixels in the image are processed  Camera connected directly to the board (no host interaction)  Only the results are sent to the host after they become available, and while new results are being calculated Very Low Latency  Surgical laser must be shut off immediately if we detect that it is aimed incorrectly  Cannot tolerate a one frame delay  Complex memory management scheme must be introduced Algorithm What does the algorithm do?  Retinal vascular tracing: detection of blood vessels in images of the retina  The algorithm finds blood vessels and traces out their structure Where is the algorithm used?  Processing live video of the patient retina during laser retinal surgery  Highlighting the vascular structure helps the surgeon avoid damage Why do we need to accelerate it?  Current implementation: software on a general-purpose processor  Images are 512x512 pixels, and need to be processed at frame rate Acceleration of the Retinal Vascular Tracing Algorithm using FPGAs Miriam Leeser Shawn Miller Badrinath Roysam Charles Stewart Ken Fritsche This work was supported in part by CenSSIS, the Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Centers Program of the National Science Foundation (Award Number EEC ). More Information In proceedings: Rapid Automated Tracing and Feature Extraction from Retinal Fundus Images Using Direct Exploratory Algorithms A. Can, H. Shen, J.N. Turner, H.L. Tanenbaum and B. Roysam, IEEE Transactions on Information Technology in Biomedicine, June 99. On the web: Original Image Each pixel is passed through the design unaltered. Direction The direction template with the maximum response for each pixel. The direction is represented by a value between 0 and 15. Response The maximum response that led to the direction decision for each pixel. Results Objective To accelerate retinal vascular tracing by implementing computation of template responses in reconfigurable hardware. Reconfigurable Hardware Firebird reconfigurable computing engine from Annapolis Micro Systems  1 Xilinx VIRTEX E (XCV2000E) FPGA  5 Memory banks (4 x 64-bit, 1 x 32-bit)  5.4 Gbytes/sec of memory bandwidth  66Mhz/64-bit PCI interface to host FIREBIRD BOARD Image Data MEMORY 2 Image Data MEMORY 3 PCI Bus Host Direction Filter Design FPGA Data Packing Design BlockRAM Memory Switching Design Image Data MEMORY 0 Image Data MEMORY 1 Memory Switching Design Framegrabber (Dillon Eng.) Direction Templates Stand-alone camera outputs only image data. Our design outputs not only image data, but directional template responses as well. The cost of additional image processing is a latency on the order of seconds. This is a low cost when considering that at 30 frames/sec, a new frame of image data is introduced every 33 msec. The application for this project is Retinal Vascular Tracing, but the same method can be applied to any problem that requires real-time image processing. Conclusions Memory Management Problems  Must continuously write new data to input memory from the camera and be reading the data to be processed  Cannot read and write from one memory on the same clock cycle  The image is stored row-wise, but must be read column-wise Solution  Store the image in “checkerboard” fashion in two memories. Every other pixel is stored in a different memory  A column of data is read by alternating between the two memories on every clock cycle Input Memory 0 Input Memory 1 Clock Writing data from camera Reading data to be processed Inactive Time Memory 0Memory 1 Checkerboard storage of a 5x5 image