1. Implementing Codesign in Xilinx Virtex II Pro Betim Çiço, Hergys Rexha Department of Informatics Engineering Faculty of Information Technologies Polytechnic University of Tirana

2. Outline Introduction The main algorithm proposed  Spotlights  Static lights  Light pairs System implementation Results Conclusions

3. Introduction Future of automotive systems? => video based driver assistance. Complex algorithm are required. (and they need to be executed faster and faster) One solution would be the use of dedicated HW. Drawback, is the lack of the flexibility. Nowadays systems for driver assistance offer features such, lane departure or cruise control  cameras and radar sensors

4. Introduction But we want more...  We want help in more complex situations, like urban traffic. The proposed vision-based concept Separation, repetitive operations and high level application code. Repetitive operations are accelerated by coprocessors. High level application code is implemented fully programmable on standard CPU cores. Achivments Fast execution Flexibility

5. The main algorithm An algorithm to detect cars by taillights. Tunnel or night time driving on roads with separated lanes for each direction. Properties of the algorithm:  searches for light pairs that stem from cars.  compare the properties of car lights with the lights of the tunnel.  searches for the illumination of a license plate. We demonstrate that the best way for this algorithm to be implemented is the division into a hardware and a software part.

6. Cont` The image is recorded from a video camera (25 grayscale frames per second). The image then is processed by the subsequent engines and software parts. At the end we have as a result, car identification by taillights.

7. Basic Schematic

8. Schematic Summary Hardware implemented parts:  PlateSearchEngine (license of the car)  Spotlight-Engine  LabelingEngine Software implemented parts  Programmable software that performs several actions like shown in the scheme.

9. Spotlight Engine From Camera the frame goes:  To the Spotlight Engine  To the Platesearch Engine Depending on a threshold, the spotlight engine will give a binary image.

10. Spotlights Bright regions in the image that are ideally round and surrounded by dark pixels Taillights of cars typically appear as spotlights Proposed algorithm applies a simple shape filter to the image We define the two pixel sets relative to the current pixel (CP). If all pixels in PF are darker than all pixels in PS, the CP is a spotlight pixel lum(PS) > lum(PF ) + threshold

11. Labeling Engine Takes as input, the binary image given from Spotlight Engine Searches the binary image for regions of connected white pixels Creates a label for each region Gives as an output a list of labels Each label corresponds to one spotlight

12. Static lights We have a list of spotlights that have been extracted We have these properties for each spot: Bounding rectangle Position in the image (coordinates of the center of the bounding rectangle) Total brightness Number of pixels Static lights, i.e. they do not move relative to the road Direction of the light’s motion vector can be used to determine static lights. For each light in the current frame, a close by light is searched in the previous frame

13. Finding light pairs We have to consider Distance of lights, y-component  Both taillights of a car are expected to be on the same height on a flat road Distance of lights, x-component  A vehicle that is far away from the camera appears more in the top of the image than a closer vehicle Existence of additional light in between the pair  In that case, the two outer lights of the four spotlights could be considered as one candidate pair At the end, including the continuity vector, we come up with the best candidates for light pairs.

14. System Implementation The system was implemented on a Xilinx Virtex-II Pro FPGA That FPGA also features two embedded 300 MHz PowerPC CPU cores, one of which was used to run an embedded Linux operating system

15. Results For the Spotlight Engine we have 7 times increase in speed, compared to Pentium 4 processor. 173 time increase in speed compared to a Power PC processor.

16. Conclusions As we have seen through this work in today’s video assistance systems for drivers need to be real time and flexible. We used here a new type of design where hardware and software cooperate to give both real time performance and flexibility. What is more important:  “By implementing hardware coprocessors we let repetitive tasks to be handled by hardware engines while the software part takes care of the interface between parts of the system and some computations on a small amount of data”. With Codesign we release the processor load during the computations.

17. Thank you... bcico@icc-al.orgbcico@icc-al.org, hrexha@gmail.comhrexha@gmail.com