Utilizing AI & GPUs to Build Cloud-based Real-Time Video Event Detection Solutions Zvika Ashani CTO

Background Agent Vi has been developing video analytics solutions for surveillance applications for the past 10 years Our products provide a diverse set of capabilities such as real time alerts, forensic search and statistical analysis Our enterprise solution is installed on premise using customer provided servers 2 years ago we set out to create a SaaS platform that will provide real time video analytics capabilities to our customers © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

What’s the big deal with VA in the cloud? Deep Learning for object detection and classification – a de-facto standard Image analytics as a service – numerous providers out there (AWS, Azure, Google and others). Applications such as image tagging, facial recognition, image moderation and more. Offline (batch processing) of video is also starting to become available from these same providers. Mostly used for video indexing and search. Real time video analysis is still a challenge, mainly due to bandwidth constraints © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Platform requirements Easy to deploy, does not require any software installation on site Analyzes surveillance video and provides real time alerts based on user defined rules Provide state of the art probability of detection and false alarm rates Support any IP surveillance camera that can provide an RTSP stream Utilize low bandwidth per camera (less than 50kbps) Use low cost HW on site Operate at near real time (latency should not exceed 200ms) Operational cost per camera per month should be very low © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Challenges Cannot upload a high quality video stream to the cloud due to BW limitations Cannot deploy a GPU to the user sites due to cost considerations Solution Use a distributed processing architecture Perform initial processing of the video stream at the edge to reduce the amount of BW required Send the results of the initial processing to the service in the cloud Use a combination of CPU’s and GPU’s in the cloud to complete the processing and detect events © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

All communication is encrypted using SSL Service Architecture Agent Vi Support Internet 30-50 kbps per camera InnoVi Edge Customer Internet Feature streams Customer InnoVi Edge Monitoring Stations Internet Admin All communication is encrypted using SSL © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

At the edge In the cloud Video decoding Foreground detection Blob segmentation Image cropping 50 kbps upstream 10 kbps downstream In the cloud Object detection and tracking Object classification Rules processing and event generation Other tasks such as automated scene learning © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Our Neural Network implementation Mostly use CNN’s for object detection and classification and LSTMs for other tasks Classify images into one of 15 classes Images are B&W 64x64 pixels Publicly available pre-trained models on ImageNet (or other data sets) are irrelevant due to differing image format and fields of view Need to balance network complexity between accuracy and cost per camera Use the Caffe framework for implementing the various network topologies © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Only real surveillance footage is used - people © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Only real surveillance footage is used - vehicles © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Only real surveillance footage is used - noise © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Analytics Server - Initial implementation Customer Analytics Server Internet InnoVi Edge N servers with CPU + GPU Assumption – there is an sufficient number of servers available on AWS to allow the solution to scale to support any number of cameras Runs both CPU intensive and GPU intensive tasks Was actually CPU bound, average GPU utilization was about 25% © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Analytics Server - Micro service implementation Customer Internet InnoVi Edge Analytics Server Analytics Service N servers with CPU Classification Service M servers with CPU + GPU Split analytics server to GPU and CPU tasks Each server cluster scales separately Over 90% utilization achieved on both clusters © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Classifier Service Benchmark g2.2xlarge – NVIDIA GRID GPU $0.7 per hour g3.4xlarge – NVIDIA Tesla M60 GPU $1.21 per hour © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Examples © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

Cost can be further decreased Next obvious step is implementing inference with INT8. AWS has yet to announce support for EC2 instances with INT8 support Cost per image is expected to go down by a factor of 2x – 4x © 2017 Agent Video Intelligence Ltd. Confidential and Proprietary