Machine Learning Services In-database and Stand-alone Andrew Loree

Got a question? Andrew Loree www.andyloree.com andy@andyloree.com @LowOnDiskSpace

Expectations & Assumptions What is your primary role? SQL Developer App Developer SQL sysadmin/Platform DBA Not a Python/R language tutorial Not a Machine Learning (ML) how-to

Goals – To Understand… What is Machine Learning Server? Different deployment designs In-database - SQL Server Machine Learning Services Stand-alone Administrative features Language features Simple use-cases

Does the Machine really Learn? Pattern recognition is from learning and past experiences, and we use it every day Which of these charges were fraudulent for my credit card? Ship to Address Type Merchant Date Amount Mine Computers & Accessories AMAZON 04-09 $734.95 $12.98 Not Mine Books 04-10 $468.03 Electronics NEWEGG 04-12 $198.32 When is there enough data and when do you have too much? Enter ML

Types of Questions (about Data) Descriptive - how many of X did I sell? Associative - is there an association between temperature and sales? (hypothesis) Comparative - how many X sells versus Y? Predicative - using associations and comparatives to predicate sales of X? Machine Learning can answer predictive questions

Current Microsoft ML Landscape Machine Learning Server – operationalized ML services, parallel and distributed processing SQL Server Machine Learning Services – functionally similar to ML Server, in-database Data Science Virtual Machine (DSVM) – preconfigured, multi- scenario, primarily desktop focused Spark MLLib – ML for HDInsight Azure Machine Learning Studio – Drag-and-drop web environment to build, test and publish predictive analytics Cognitive Toolkit – Deep-learning, useable by all of the above Cognitive Services – APIs to pre-trained models covering sentiment detection, vision, speech, language and knowledge search https://docs.microsoft.com/en-us/azure/machine-learning/preview/overview-more-machine-learning

Why Machine Learning Server? How do we consume our model?

Why Machine Learning Server? Operationalized Model Management

What is Machine Learning Server? Operationalized model management for Machine Learning in the enterprise Answers to how to deploy and consume ML models Scalable language support MicrosoftML and revoscalepy Two different deployment methods (as well as how they are consumed and managed) In-database – data proximity to SQL Server Stand-alone – portable, remote compute-context

revoscalepy Package Scalable parallel predictive modeling algorithms Linear regression Logistic regression Decision tree Boosted tree Random forest Predictions Also contains functions: Compute context Data sources Data manipulation Analytic functions Job functions (for Spark) Serialization Utilities https://docs.microsoft.com/en-us/machine-learning-server/python-reference/revoscalepy/revoscalepy-package

MicrosoftML Package Optimized ML algorithms and pre-trained models Multi-threaded, scalable beyond memory https://docs.microsoft.com/en-us/machine-learning-server/r/concept-what-is-the-microsoftml-package

Demo: In-Database Scenario: Data cleansing street names from addresses sp_execute_external_scripts Setup Launch Azure portal Launch RDP to mlsservicessql and mlserver Show quick setup by using Azure Portal Template

Maintenance: In-database Two paths for component version upgrades: Default – SQL Server SP and update cadence Binding – upgrades using Machine Learning Server release cadence SqlBindR.exe (part of ML Server installation) to control binding/unbinding Adding packages Run python C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES PIP - make sure your path has -\PYTHON_SERVICES\Scripts] R - use CREATE EXTERNAL LIBRARY, not supported with Python yet https://docs.microsoft.com/en-us/sql/advanced-analytics/r/use-sqlbindr-exe-to-upgrade-an-instance-of-sql-server https://docs.microsoft.com/en-us/sql/advanced-analytics/python/install-additional-python-packages-on-sql-server

Components: In-database Launchpad – starts satellite processes BxlServer – binary exchange to lanagauge SqlSatellite – handles data communication Operationalization - https://docs.microsoft.com/en-us/machine-learning-server/install/r-server-install-supported-platforms https://docs.microsoft.com/en-us/sql/advanced-analytics/python/new-components-in-sql-server-to-support-python-integration

Security & Performance: In-database Script execution via Launchpad uses local accounts Configuration Manager -> Launchpad Service EXECUTE ANY EXTERNAL SCRIPT granted per db Performance Manage resources with EXTERNAL resource governor pool Balance CPU/memory usage with default pool Increase number of Launchpad pool users (default 20) Some tasks better are better left to an application tier https://docs.microsoft.com/en-us/sql/advanced-analytics/r/how-to-create-a-resource-pool-for-r

Demo: Pre-trained Model Scenario: Generate positive/negative summary of reviews for a product Pre-trained sentiment analysis from microsoftml sentiment – general feeling or opinion

Demo: Models and PREDICT Scenario: Using bike rental data, dates and weather to build a model and predict future rentals Revoscalepy rx_lin_mod() Serialize trained model T-SQL PREDICT https://github.com/Microsoft/sql-server-samples/tree/master/samples/features/machine-learning-services/python/getting-started/rental-prediction

Deployment: In-database Models are data (varbinary) R/Python scripts are data (nvarchar) Many ways to version and control, e.g. Restrict arbitrary scripts via EXECUTE AS within sprocs Store scripts and models in tables control with row-level security In-database available only on Windows SQL Server installation today PREDICT is available on all SQL platforms and versions Monitored with extended events Operationalization - https://docs.microsoft.com/en-us/machine-learning-server/install/r-server-install-supported-platforms

Administration: In-Database DMV’s sys.dm_external_script_requests – current workers sys.dm_external_script_execution_stats – execution counters Process execution SELECT * FROM sys.dm_os_performance_counters WHERE object_name LIKE '%External Scripts%’ Resource pools – Enterprise Edition sys.resource_governor_external_resource_pools sys.dm_resource_governor_external_resource_pool_affinity https://docs.microsoft.com/en-us/sql/advanced-analytics/r/dmvs-for-sql-server-r-services Job objects - https://msdn.microsoft.com/library/windows/desktop/ms684161.aspx https://docs.microsoft.com/en-us/sql/advanced-analytics/r/managing-and-monitoring-r-solutions

Compute Context: In-database Allows users to run code within SQL Server as a remote compute context RxInSqlServer Use-cases: Avoid data movement More powerful calculation Maintain data security https://docs.microsoft.com/en-us/machine-learning-server/r/concept-what-is-compute-context https://docs.microsoft.com/en-us/machine-learning-server/python-reference/revoscalepy/rxinsqlserver https://blogs.technet.microsoft.com/dataplatforminsider/2017/09/26/in-database-machine-learning-in-sql-server-2017/

Stand-alone Machine Learning Server Setup one-box configuration > az ml admin boostrap Jupyter notebook to create and deploy, and consume Samples – https://github.com/Microsoft/ML-Server-Python-Samples One-box configuration - https://docs.microsoft.com/en-us/machine-learning-server/operationalize/configure-machine-learning-server-one-box Kestrel web server - https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel?tabs=aspnetcore2x ML Server Admin with CLI - https://docs.microsoft.com/en-us/machine-learning-server/operationalize/configure-admin-cli-launch Python samples on Windows Download just the .ipynb files from the Github repo https://github.com/Microsoft/ML-Server-Python-Samples Different deployment methods - https://blogs.msdn.microsoft.com/mlserver/2017/12/13/getting-started-with-python-web-services-using-machine-learning-server/ Rest calls with Postman - https://blogs.msdn.microsoft.com/mlserver/2017/02/22/rest-calls-using-postman-for-r-server-o16n-2/ Setup Launch Azure portal Launch RDP to mlsservicessql and mlserver Show quick setup by using Azure Portal Template to create Launch python.exe Run example demo summary script to verify installation Configure our node using az ml admin commands > az ml admin –h Setup our one-box config with the bootstrap command > az ml admin bootstrap Show nodes list port list Launch Jupyter notebook.exe Run quickstart demoy

Multi-server Configuration Leverage load- balancing Web and compute nodes separate Typical web scale- out

Administration: Stand-alone az ml admin is primary source of configuration/control Monitoring and configuration tuning Kestrl web service https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel?tabs=aspnetcore2x https://docs.microsoft.com/en-us/machine-learning-server/what-is-operationalization

Got a question? Andrew Loree www.andyloree.com andy@andyloree.com @LowOnDiskSpace