Deploying machine learning models at scale

Slides:



Advertisements
Similar presentations
Web RoleWorker Role At runtime each Role will execute on one or more instances A role instance is a set of code, configuration, and local data, deployed.
Advertisements

Windows Azure Role Cloud Computing Soup to Nuts Mike Benkovich Microsoft Corporation btlod-71.
Windows Azure Conference 2014 Running Docker on Windows Azure.
Operating System for the Cloud Runs applications in the cloud Provides Storage Application Management Windows Azure ideal for applications needing:
WINDOWS AZURE Scott Guthrie Corporate Vice President Windows Azure
#msitconf. Damien Caro Technical Evangelist Manager, Что будет, если приложение поместить в контейнер? What happens if the application.
Microsoft Build /9/2017 5:00 AM © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY,
IT Operations Management
1/26/2018 Hosting Windows Desktops and Applications Using Remote Desktop Services and Azure Windows Server Azure Resource Manager © 2014 Microsoft.
Welcome to the Hands on Lab!
Run Azure Services in your datacenter
Building ARM IaaS Application Environment
Containers as a Service with Docker to Extend an Open Platform
Fundamentals Sunny Sharma Microsoft
Scalable Web Apps Target this solution to brand leaders responsible for customer engagement and roll-out of global marketing campaigns. Implement scenarios.
Accelerate your DevOps with OpenShift by Red Hat
DL (Deep Learning) Workspace
Azure Machine Learning Deploying and Managing Models in production
Docker and Azure Container Service
Docker Birthday #3.
Introducing Azure Functions
In-Depth Introduction to Docker
Azure Functions and Automation: The SQL Agent in the Cloud
Windows Azure Cloud Visit – Ravindra verma.
Introduction to R Programming with AzureML
Modernizing Application Delivery with Containers & Kubernetes
IT Operations Management
Serverless Architecture in Azure
Scalable Web Apps Target this solution to brand leaders responsible for customer engagement and roll-out of global marketing campaigns. Implement scenarios.
Microsoft Azure Service Fabric Overview
Exploring Azure Event Grid
Service Fabric Patterns & Best Practices
Windows Server & Hyper-V Containers Vaggelis Kappas
Microsoft Connect /18/ :32 PM
9/20/ :55 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.
Azure Container Instances
CloudSimplified.IO.
Introduction to Windows Azure Web Sites
Open Source Toolkit for Turn-Key AI Cluster (Introduction)
Intro to Docker Containers and Orchestration in the Cloud
Microsoft Ignite NZ October 2016 SKYCITY, Auckland.
Azure Event Grid with Custom Events
11/27/2018 4:20 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.
Developing for the cloud with Visual Studio
Docker Workflows with Visual Studio
12/5/ :36 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.
Learn. Imagine. Build. .NET Conf
M318.
System Center Application Management
From Source to Production: The Latest in Container Dev
1/2/2019 5:18 PM THR3016 Customer stories: Plan and orchestrate large resource deployments on Azure infrastructure Igal Figlin Principal PM Manager – Azure.
Azure Container Registry
Serverless Architecture in the Cloud
2/16/2019 9:42 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.
MDC-B203 Deploying Applications in Microsoft System Center Virtual Machine Manager Using Services John Messec Program Manager Microsoft.
Microsoft Connect /25/2019 1:20 PM
Windows Azure Overview
ML in Azure Databricks Mahesh Balija 4/15/2019 1:36 PM
Introduction to Docker
TechEd /23/2019 9:23 AM © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks.
Developing Windows Azure Applications with Visual Studio
SQL Server using Amazon Web Services EC2 Instances
Azure Container Service
Harrison Howell CSCE 824 Dr. Farkas
Microsoft Virtual Academy
A DevOps process for deploying R to production
Docker for DBAs SQL Saturday 8/17/2019.
Containers on Azure Peter Lasne Sr. Software Development Engineer
9/16/2019 6:55 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN.
Presentation transcript:

Deploying machine learning models at scale 8/24/2019 2:58 PM Deploying machine learning models at scale Angus Taylor © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Introduction Deploy machine learning: Put machine learning models into production Score data and serve the results to users or other systems Production systems for business applications need to be: scalable elastic available consistent Cloud-based architectures meet these requirements

Outline Docker for R Architectures for batch inferencing Architectures for real-time inferencing

Docker for R

Why Docker? Production applications need to be consistent consistent R code consistent R packages consistent R version consistent OS dependencies Production applications need to be portable transition seamlessly from development to production run on multiple (virtual) machines

Docker containers A Docker container encompasses everything needed to run an application Deploys anywhere that Docker is installed Development environment matches production score.R R installation R packages OS kernel OS dependencies R models?

Azure Container Registry Docker concepts Git repository Docker Hub / Azure Container Registry build Dockerfile Docker image pull run VM VM VM Docker containers

Dockerfile Instructions for building a docker image Use rocker base images Install all necessary dependencies Specify the scoring script to run FROM rocker/tidyverse:3.5.1 RUN R –e “devtools::install_github( “Azure/doAzureParallel”) RUN R install2.r jsonlite dotenv CMD [“Rscript”, “score.R”]

Package management for R Recommendations Packrat package Snapshot package MRAN (daily archive of CRAN) devtools::install_github(“package”, ref = “commit”) Anaconda Use these to “pin” package versions in your Dockerfile!

Architectures for batch inferencing

Example use case: product sales forecasting Forecast sales of 1,000s of products Produce forecasts on a schedule every week Size of product range changes over time Requirements: Efficiency Scalability Cost!

Azure Batch Cluster of VMs in the cloud Each VM runs scoring job in a docker container Only pay while they’re running Cluster auto-scales

Azure Container Instance Serverless compute Runs a docker container Use as the “master node” to trigger scoring jobs Use doAzureParallel package within ACI job to trigger scoring jobs on Azure Batch Only pay for the time it runs

Batch inferencing architecture Azure Container Registry Azure Container Instance triggers parallel scoring jobs Master image doAzureParallel run_jobs.R Worker image ML packages score.R Azure Batch cluster runs score.R Azure blob storage holds data and R models

Architectures for real-time inferencing

Example use case: product recommendation Provide product recommendations to users of a retail website Potentially 1,000s of users at any one time Highly elastic demand Requirements: Availability Latency Elasticity Cost!

Azure Kubernetes Service Highly available containerized deployment Serves HTTP requests at very low latency Load balancing for elasticity

Microsoft Machine Learning Server Host a machine learning model as a web service Listens for HTTPs requests Makes predictions with scoring code Returns response Alternative: plumber package library(mrsdeploy) model <- readRDS("model.rds") publishService( “model-service”, v=“1.0.0”, code=“score.R” model=model )

Real-time inferencing architecture Predictions HTTP response Ingress controller for load balancing Input HTTP request Data Predictions Azure Container Registry Docker image Model object Prediction function MMLS / plumber

Conclusion Use Docker to create consistent and portable ML deployments Use containerized VM clusters for batch inferencing Use Kubernetes clusters for real-time inferencing

https://github.com/Azure/RealtimeRDeployment 8/24/2019 2:58 PM https://github.com/Azure/RealtimeRDeployment https://github.com/Azure/RBatchScoring © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.