Alain Goossens & Jean-Pierre Van Loo Data scientists – SII Belgium Applying neural networks to anomaly detection Alain Goossens & Jean-Pierre Van Loo Data scientists – SII Belgium Data Innovation Summit 27th June 2018 #DISUMMIT

Underground cabling, infrastructure and pipelines Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Current anomaly detection process Desk top analysis Visual analysis of pictures Risk analysis Verification of permits When necessary send an intervention team Intervention team Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Artificial intelligence Concept of the POC YES there is an anomaly NO there is no anomaly Artificial intelligence INPUT pictures Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Overview Advanced analytics pattern in Azure MODEL TRAINING SERVING STORAGE AZURE ML AZURE ML STUDIO ML SERVER AZURE DATABRICKS (Spark ML) SQL Server (In-database ML) DATA SCIENCE VM BATCH AI SENSORS AND IOT (UNSTRUCTURED) COSMOS DB APPLICATIONS LONG TERM STORAGE DATA PROCESSING SQL DB r LOGS, FILES AND MEDIA (UNSTRUCTURED) SQL DB DATA LAKE STORE AZURE STORAGE COSMOS DB DATA LAKE ANALYTICS AZURE DATABRICKS HDINSIGHT SQL DW TRAINED MODEL HOSTING ORCHESTRATION AZURE ANALYSIS SERVICES BUSINESS / CUSTOM APPS (STRUCTURED) DASHBOARDS SQL Server (In-database ML) AZURE CONTAINER SERVICE DATA FACTORY Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Data preparation Categorization of the images Rural - Industrial 523 with bulldozers 567 with no anomalies Enrichment to have more images Cropping, flipping or translating images Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

The models used A simple neural network with Microsoft Azure Machine Learning Studio  Convolutional neural network (CNN) and R-CNN on a deep learning Virtual machine Custom Vision Service Vision API + Microsoft Azure Machine Learning Studio based on output of the Vision API Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

The models used A simple neural network with Microsoft Azure Machine Learning Studio  Convolutional neural network (CNN) and R-CNN on a deep learning Virtual machine Custom Vision Service Vision API + Microsoft Azure Machine Learning Studio based on output of the Vision API Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Two-class Neural network – AUC optimized

Principle of convolutional neural network (CNN) anomaly no anomaly Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Deep learning virtual machine Labelled data set, anomaly Labelled data set, normal condition Two class classification/prediction: YES anomaly No anomaly Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Convolutional neural network (CNN) Results 50 epochs Accuracy: Training = 98,5 % Validation = 92,9 % Test = 93 % True positives : 94,4 % False negatives : 5,6 % False positives : 8,4 % True negatives : 91,6 % Bulldozer detected when there is one No bulldozer detected when there is no No bulldozer detected but there is one Bulldozer detected but there is no Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Regions of interest on big images Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Custom Vision Service Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Computer Vision API 2000 known tags Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Machine learning studio on vision API tags Find a correlation between tags identified and the presence of a bulldozer Tested models: Two-class boosted decision tree Two-class logistic regression Two-class support vector machine Two-class decision jungle Two-class neural network Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Business conclusions Fairly good results with the main models used but with a dataset: trivial limited in variability Customer learned about AI, ML and NN in a real world context Further steps: Extend dataset variability Apply other models Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Business conclusions Fairly good results with the main models used but with a dataset: trivial limited in variability Customer learned about AI, ML and NN in a real world context Further steps: Extend dataset variability Apply other models Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Scientific conclusions Cropping images is absolutely required Learning speed of the neural network is quite satisfactory (for a relatively simple dataset) Ad hoc trained models perform better but pre- trained models are not to be excluded Using classifiers on metatags from a generic Vision API gives a fairly good result Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT

Scientific conclusions Cropping images is absolutely required Learning speed of the neural network is quite satisfactory (for a relatively simple dataset) Ad hoc trained models perform better but pre- trained models are not to be excluded Using classifiers on metatags from a generic Vision API gives a fairly good result Applying neural networks to anomaly detection, Alain Goossens & Jean-Pierre Van Loo, @alaingoossens4 @jpvl1 #DISUMMIT