1. Workshop - November 2011 - Toulouse (SoC toolKit for critical Embedded sysTems) Thales Use Case: Pedestrian tracking with smart cameras SoCKET Collaborative Project Paul Brelet paul.brelet@thalesroup.com 23/11/2011 1

2. Introduction Thales interest on the project. SOCKET Global Flow Thales Use case description: Application description. Create Classifiers. C application. Architecture description. 2Workshop - November 2011

3. Thales Interest Thales Research has several interests on the project: Interaction between control and dataflow. Predictability real time comportment. Reduce the programming/re-programming cost. Verification. 3Workshop - November 2011

4. SoCKET Flow Global SoC Req. SoC Architecture Functional validation SW Performance Validation C/C++/ASM Functionality Timing + Functionality Instruction Set Simulator System Requirements Platform Assembly Metrics HLS System Properties Hardware properties Software properties TLM LT TLM AT Software Co-simulation/Co-emulation Silicon Software Execution HLS Traffic generator Metrics IP-Xact SoC Headers generation RTL Software Requirements traceability 4

5. Thales use case – Context Context Safety of critical infrastructures containing image processing Detection of intrusions  pedestrians Description Environment multi-camera (basic scenario using 2 cameras) having a specific visual field Centralized process of the video stream coming from a reduced number of camera in an System on Chip embedded system 5Workshop - November 2011

6. Use case TRT – Description App. Pedestrians detection Algorithm of classification [Viola&Jones] Two steps: Off-line: training by an image database On-line: detection by using the training results Pedestrians tracking in an multi-camera environment Use of the visual covering of the cameras in order to carry out the tracking Utilization of descriptors of forms and/or colors in order to improve the tracking and to manage occlusions 6Workshop - November 2011

7. Thales use case – Appl. description Some examples [Viola&Jones ICCV ’03] Database 7Workshop - November 2011

8. Thales use case – App. description Procedure The training is carried out on a workstation The SoC applies the classifiers and the tracking algorithm on multi-camera Synoptic Application Camera Integral Image Tiler Features Classifiers Cascade Bounding Box Descriptor Table Manage Descriptors Tracking result SoC IOFrontend processingBackend processing Host 8Workshop - November 2011

9. Thales use case – App. Description Schematic view of the pedestrian detection 9Workshop - November 2011

10. To create the classifiers From scratch: A huge database is needed: Minimum 10 000 positive and negative images to have good classifiers. A good workstation and Time: Take about 30 minutes on 8 cores Xeon 2.4GHz and 12Gb RAM. 2 Weeks on Dual core and 2Gb RAM. From existing classifiers: Convert to the Thales classifiers by XML parsing. 10Workshop - November 2011

11. C Application Classifiers (off-line) C Code (on-line) Result 11Workshop - November 2011

12. Thales use Case – Architecture Host Architecture details 12Workshop - November 2011

13. Thales use Case – Architecture Accelerator details (Engine) 13Workshop - November 2011

14. 23/11/2011 Questions? 14Workshop - November 2011