1. Machine Learning Services
In-database and Stand-alone
Andrew Loree

2. Got a question? Andrew Loree
@LowOnDiskSpace

3. Expectations & Assumptions
What is your primary role?
SQL Developer
App Developer
SQL sysadmin/Platform DBA
Not a Python/R language tutorial
Not a Machine Learning (ML) how-to

4. Goals – To Understand… What is Machine Learning Server?
Different deployment designs
In-database - SQL Server Machine Learning Services
Stand-alone
Administrative features
Language features
Simple use-cases

5. Does the Machine really Learn?
Pattern recognition is from learning and past experiences, and we use it every day
Which of these charges were fraudulent for my credit card?
Ship to Address
Type
Merchant
Date
Amount
Mine
Computers & Accessories
AMAZON
04-09
$734.95
$12.98
Not Mine
Books
04-10
$468.03
Electronics
NEWEGG
04-12
$198.32
When is there enough data and when do you have too much? Enter ML

6. Types of Questions (about Data)
Descriptive - how many of X did I sell?
Associative - is there an association between temperature and sales? (hypothesis)
Comparative - how many X sells versus Y?
Predicative - using associations and comparatives to predicate sales of X?
Machine Learning can answer predictive questions

7. Current Microsoft ML Landscape
Machine Learning Server – operationalized ML services, parallel and distributed processing
SQL Server Machine Learning Services – functionally similar to ML Server, in-database
Data Science Virtual Machine (DSVM) – preconfigured, multi- scenario, primarily desktop focused
Spark MLLib – ML for HDInsight
Azure Machine Learning Studio – Drag-and-drop web environment to build, test and publish predictive analytics
Cognitive Toolkit – Deep-learning, useable by all of the above
Cognitive Services – APIs to pre-trained models covering sentiment detection, vision, speech, language and knowledge search

8. Why Machine Learning Server?
How do we consume our model?

9. Why Machine Learning Server?
Operationalized Model Management

10. What is Machine Learning Server?
Operationalized model management for Machine Learning in the enterprise
Answers to how to deploy and consume ML models
Scalable language support
MicrosoftML and revoscalepy
Two different deployment methods (as well as how they are consumed and managed)
In-database – data proximity to SQL Server
Stand-alone – portable, remote compute-context

11. revoscalepy Package Scalable parallel predictive modeling algorithms
Linear regression
Logistic regression
Decision tree
Boosted tree
Random forest
Predictions
Also contains functions:
Compute context
Data sources
Data manipulation
Analytic functions
Job functions (for Spark)
Serialization
Utilities

12. MicrosoftML Package Optimized ML algorithms and pre-trained models
Multi-threaded, scalable beyond memory

13. Demo: In-Database Scenario: Data cleansing street names from addresses
sp_execute_external_scripts
Setup
Launch Azure portal
Launch RDP to mlsservicessql and mlserver
Show quick setup by using Azure Portal Template

14. Maintenance: In-database
Two paths for component version upgrades:
Default – SQL Server SP and update cadence
Binding – upgrades using Machine Learning Server release cadence
SqlBindR.exe (part of ML Server installation) to control binding/unbinding
Adding packages
Run python
C:\Program Files\Microsoft SQL Server\MSSQL14.MSSQLSERVER\PYTHON_SERVICES
PIP - make sure your path has -\PYTHON_SERVICES\Scripts]
R - use CREATE EXTERNAL LIBRARY, not supported with Python yet

15. Components: In-database
Launchpad – starts satellite processes
BxlServer – binary exchange to lanagauge
SqlSatellite – handles data communication
Operationalization -

16. Security & Performance: In-database
Script execution via Launchpad uses local accounts
Configuration Manager -> Launchpad Service
EXECUTE ANY EXTERNAL SCRIPT granted per db
Performance
Manage resources with EXTERNAL resource governor pool
Balance CPU/memory usage with default pool
Increase number of Launchpad pool users (default 20)
Some tasks better are better left to an application tier

17. Demo: Pre-trained Model
Scenario: Generate positive/negative summary of reviews for a product
Pre-trained sentiment analysis from microsoftml
sentiment – general feeling or opinion

18. Demo: Models and PREDICT
Scenario: Using bike rental data, dates and weather to build a model and predict future rentals
Revoscalepy rx_lin_mod()
Serialize trained model
T-SQL PREDICT

19. Deployment: In-database
Models are data (varbinary)
R/Python scripts are data (nvarchar)
Many ways to version and control, e.g.
Restrict arbitrary scripts via EXECUTE AS within sprocs
Store scripts and models in tables control with row-level security
In-database available only on Windows SQL Server installation today
PREDICT is available on all SQL platforms and versions
Monitored with extended events
Operationalization -

20. Administration: In-Database
DMV’s
sys.dm_external_script_requests – current workers
sys.dm_external_script_execution_stats – execution counters
Process execution
SELECT * FROM sys.dm_os_performance_counters WHERE object_name LIKE '%External Scripts%’
Resource pools – Enterprise Edition
sys.resource_governor_external_resource_pools
sys.dm_resource_governor_external_resource_pool_affinity
Job objects -

21. Compute Context: In-database
Allows users to run code within SQL Server as a remote compute context
RxInSqlServer
Use-cases:
Avoid data movement
More powerful calculation
Maintain data security

22. Stand-alone Machine Learning Server
Setup one-box configuration > az ml admin boostrap
Jupyter notebook to create and deploy, and consume
Samples –
One-box configuration -
Kestrel web server -
ML Server Admin with CLI -
Python samples on Windows
Download just the .ipynb files from the Github repo
Different deployment methods -
Rest calls with Postman -
Setup
Launch Azure portal
Launch RDP to mlsservicessql and mlserver
Show quick setup by using Azure Portal Template to create
Launch python.exe
Run example demo summary script to verify installation
Configure our node using az ml admin commands > az ml admin –h
Setup our one-box config with the bootstrap command > az ml admin bootstrap
Show nodes list port list
Launch Jupyter notebook.exe
Run quickstart demoy

23. Multi-server Configuration
Leverage load- balancing
Web and compute nodes separate
Typical web scale- out

24. Administration: Stand-alone
az ml admin is primary source of configuration/control
Monitoring and configuration tuning Kestrl web service

25. Got a question? Andrew Loree
@LowOnDiskSpace