1. Utilizing AI & GPUs to Build Cloud-based Real-Time Video Event Detection Solutions
Zvika Ashani
CTO

2. Background
Agent Vi has been developing video analytics solutions for surveillance applications for the past 10 years
Our products provide a diverse set of capabilities such as real time alerts, forensic search and statistical analysis
Our enterprise solution is installed on premise using customer provided servers
2 years ago we set out to create a SaaS platform that will provide real time video analytics capabilities to our customers
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

3. What’s the big deal with VA in the cloud?
Deep Learning for object detection and classification – a de-facto standard
Image analytics as a service – numerous providers out there (AWS, Azure, Google and others). Applications such as image tagging, facial recognition, image moderation and more.
Offline (batch processing) of video is also starting to become available from these same providers. Mostly used for video indexing and search.
Real time video analysis is still a challenge, mainly due to bandwidth constraints
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

4. Platform requirements
Easy to deploy, does not require any software installation on site
Analyzes surveillance video and provides real time alerts based on user defined rules
Provide state of the art probability of detection and false alarm rates
Support any IP surveillance camera that can provide an RTSP stream
Utilize low bandwidth per camera (less than 50kbps)
Use low cost HW on site
Operate at near real time (latency should not exceed 200ms)
Operational cost per camera per month should be very low
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

5. Challenges
Cannot upload a high quality video stream to the cloud due to BW limitations
Cannot deploy a GPU to the user sites due to cost considerations
Solution
Use a distributed processing architecture
Perform initial processing of the video stream at the edge to reduce the amount of BW required
Send the results of the initial processing to the service in the cloud
Use a combination of CPU’s and GPU’s in the cloud to complete the processing and detect events
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

6. All communication is encrypted using SSL
Service Architecture
Agent Vi Support
Internet
30-50 kbps per camera
InnoVi Edge
Customer
Internet
Feature streams
Customer
InnoVi Edge
Monitoring Stations
Internet
Admin
All communication is encrypted using SSL
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

7. At the edge In the cloud Video decoding Foreground detection
Blob segmentation
Image cropping
50 kbps upstream
10 kbps downstream
In the cloud
Object detection and tracking
Object classification
Rules processing and event generation
Other tasks such as automated scene learning
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

8. Our Neural Network implementation
Mostly use CNN’s for object detection and classification and LSTMs for other tasks
Classify images into one of 15 classes
Images are B&W 64x64 pixels
Publicly available pre-trained models on ImageNet (or other data sets) are irrelevant due to differing image format and fields of view
Need to balance network complexity between accuracy and cost per camera
Use the Caffe framework for implementing the various network topologies
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

9. Only real surveillance footage is used - people
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

10. Only real surveillance footage is used - vehicles
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

11. Only real surveillance footage is used - noise
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

12. Analytics Server - Initial implementation
Customer
Analytics Server
Internet
InnoVi Edge
N servers with
CPU + GPU
Assumption – there is an sufficient number of servers available on AWS to allow the solution to scale to support any number of cameras
Runs both CPU intensive and GPU intensive tasks
Was actually CPU bound, average GPU utilization was about 25%
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

13. Analytics Server - Micro service implementation
Customer
Internet
InnoVi Edge
Analytics Server
Analytics Service
N servers with
CPU
Classification
Service
M servers with
CPU + GPU
Split analytics server to GPU and CPU tasks
Each server cluster scales separately
Over 90% utilization achieved on both clusters
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

14. Classifier Service Benchmark
g2.2xlarge – NVIDIA GRID GPU $0.7 per hour
g3.4xlarge – NVIDIA Tesla M60 GPU $1.21 per hour
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

15. Examples
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary

16. Cost can be further decreased
Next obvious step is implementing inference with INT8.
AWS has yet to announce support for EC2 instances with INT8 support
Cost per image is expected to go down by a factor of 2x – 4x
© 2017 Agent Video Intelligence Ltd. Confidential and Proprietary