Company Confidential Konnet ViAn Konnet ViAn P. Ltd. 102, Shiv Shakti Compex Baner road, Pune , India /5 Computing for Video Analytics in Surveillance

Company Confidential Konnet ViAn Agenda  Video Surveillance Solutions –Case Study  Analytics in Surveillance  Computing requirement  Optimization for High Performance Computing

Company Confidential Konnet ViAn Surveillance  TV shows like “Person of Interest” and movies like the “Mission Impossible” Series give a whole new perspective to surveillance and crime detection  Its called “mass surveillance” where surveillance cameras, all forms of electronic communications, audio and any digital data is analysed to identify “acts of terrorism” and crime.

Company Confidential Konnet ViAn Video Surveillance – Facts and Myths  Pervasive surveillance is no longer just a part of science fiction  There are merits and demerits of this digital revolution, but security is undoubtable a major concern in present times  The possibilities for pre crime detection are very feasible  There is huge rush in US, specially after the Boston bombings to increase the number of cameras that monitor  But is that sufficient? Cameras are just the eyes  The brains behind the eyes is the analytics

Company Confidential Konnet ViAn Network Infrastructure  In an already existing data network the addition of cameras will load the network band width  It requires a layout of data network  Data servers to store the video, audio and other digital data  Servers to analyse the data collected  Electronics for post analyses actions

Company Confidential Konnet ViAn City Surveillance  Applications like traffic monitoring, parking lot surveillance etc.  Impossible for people to monitor 24x7 and hence need for video analytics  Video Analytics allows real-time alerts by automating the event recognition process.

Company Confidential Konnet ViAn TRAFFIC MONITORING Surveillance application

Company Confidential Konnet ViAn Features of Traffic Monitoring System  Detect vehicles (cars, trucks, 2 wheelers, carriers, etc)  Capture number plates  Capture Driver image  Detect traffic jams –Alerts traffic police  Detect vehicle collisions –Alerts and alarms for collisions  Track vehicles for speeding –Ticket generation for speeding vehicles  Tracking of vehicles through multiple cameras/locations

Company Confidential Konnet ViAn Traffic Shivajinagar

Company Confidential Konnet ViAn SOLUTION DESIGN Traffic Monitoring

Company Confidential Konnet ViAn Designing a Traffic Monitoring Soln  Approximately 100 crossroads (chowks)  4 cameras each chowk for traffic monitoring  100*4 = 400 cameras

Company Confidential Konnet ViAn Traffic Surveillance System Design Central Room The Video Wall Analytics Server Database Storage Network Switch Rack Video Management Servers Cameras

Company Confidential Konnet ViAn Server Configuration CPU coresAnalytics channels RAM in GB HDD for 1 month Analytics data i3 quad core141 Xeon E5 series 10 core blades TB 20 Xeon E5 blades 40064x 20320TB Number of CamerasHDD for 1 month H Analytics Database

Company Confidential Konnet ViAn SOFTWARE DESIGN Analytic Algorithms

Company Confidential Konnet ViAn Video Analytics  Extracts meaningful and relevant information from digital video using algorithms in computer vision, pattern recognition and machine intelligence Detects changes Video Analytics Qualifies and Classifies Video Input Correlates Interprets and Recognizes Alarm Output

Company Confidential Konnet ViAn Event Log Analytics in Traffic Monitoring Capture of frame DetectionRecognitionTracking Activity Recognition Event Notification Configuration Define virtual zones and lines to recognize activities like Speeding, collisions, traffic jams, etc Background Configuration Data Initialization fps Vehicle Classifiers fps Scene History Encoding Recordings

Company Confidential Konnet ViAn Encoding  Video resolution is needed at 15 fps  H.264 Main profile compression for streaming  Challenges to achieve optimal bit rate for 15 fps Compression Bit rate Complexity

Company Confidential Konnet ViAn Detection  Background Model updated dynamically  Edge detection performed on foreground blobs  Edge map used as features for Vehicle detection  Detection done using Adaboost algorithm  Number plate detection performed along with vehicle detection

Company Confidential Konnet ViAn Recognition of Number Plate Characters  Pre-processing requires de shearing of detected Number Plate images  Binarization  Identifying blobs by connecting the binarized image  Cleaning the unconnected and unwanted blobs  Separation of each character  Recognizing each character  Organizing the recognized characters to create a number plate

Company Confidential Konnet ViAn Tracking  Use Particle Tracking Algorithm for all detected number plates (objects)  Each object is tracked for minimum number of N frames, even if it is not detected (Maintain a history)  Every object in the history of N frames is searched in the current image for tracking  The trajectory path (state) of each object is maintained for activity recognition

Company Confidential Konnet ViAn Analytics – Computation Need  One video analytics process for an HD video stream loads 3 CPU cores in an i3  GUI and Post event automation like speed ticket generation and data analytic functions needs extra Processing  The post event alerts and alarms needs real time responses  Thus the need for High Performance Computing

Company Confidential Konnet ViAn Can we optimize?  The presented design processes analytic modules sequentially  The raw video frames of 1280x720x3 bytes causes cache misses –Detection works on grey image and edge map –Recognition on extracted features from an edge map –Tracking uses color image  Analytics Server configuration is not cost effective  Video data needs to be streamed to the central monitoring station and thus needs at least 2 Mbps network bandwidth per camera for real time streaming

Company Confidential Konnet ViAn Optimization  Distribute the computing  Create a multi-threaded flow for the analytics for better real time performance  Use HPC that have accelerators for most used idioms/intrinsics and accelerators for video encoding and decoding  This should reduce the number of cores required for our traffic monitoring

Company Confidential Konnet ViAn DISTRIBUTED COMPUTING

Company Confidential Konnet ViAn Central Vs Distributed Analytics Server Databa se Network of cameras POE switch Control Room Video Streaming to Analytics Server High bandwidth required Network of cameras POE switch + modem Web app user ANPR data and Vehicle data Low Bandwidth required ViAn ANX Web server Cloud ViAn ANX POE switch + modem

Company Confidential Konnet ViAn Distributed  In our specific traffic monitoring application, the analytics process for each camera executes as a separate process on 1 core  A central server communicates with each analytics process and collates the data for event triggering  The central server organizes the analytics and video for post event analysis

Company Confidential Konnet ViAn MULTI-THREADED

Company Confidential Konnet ViAn Analytics in Parallel - Pipelined Configuration Initialization Trigger an event Encoding Recordings Capture Frame i Detection fps Recognition Tracking Frame i-1 Activity Recogn Copy output and Frame i Copy Output C Trigger On C Trigger

Company Confidential Konnet ViAn HPC

Company Confidential Konnet ViAn Hybrid core architecture devices  H.264 encoding on accelerators  The pipelined parallel design enables load distribution on multiple cores  Nearly 4 times better performance in parallel solution –Encoding, Detection, Recognition and Tracking separate threads  Accelerators for frequently used idioms is optimized by the cross compilers

Company Confidential Konnet ViAn Issues  Complexity is high with data sharing within multiple cores  Porting to other devices is difficult  Performance on other devices will be varied depending on the supporting accelerators

Company Confidential Konnet ViAn QUESTIONS?

Company Confidential Konnet ViAn Contact Seva Panda (PhD) Chief Technology Officer Konnet ViAn P. Ltd Pune Thank you!