Advanced Analytics with Azure Machine Learning Microsoft C+E Technology Training Solution Area Data Analytics Solution Advanced Analytics Technology Azure Machine Learning Session Title Advanced Analytics with Azure Machine Learning Buck Woody

Learning Objectives Understand Machine Learning Use the Cortana Intelligence Suite (CIS) to create a Machine Learning Solution Publish and consume an Azure ML model At the end of this Module, you will: Understand Azure ML and how experiments are created Understand how MRS can be used to perform Machine Learning experiments Use ADF to schedule Azure ML Activities

Course Module List Machine Learning Overview The Cortana Intelligence Suite Machine Learning Tools and Setup Azure ML Studio and the Team Data Science Process Ingesting and Preparing Data Algorithms Model Scoring and Evaluation Lab Publishing and Using the Model

Machine Learning Overview What is Machine Learning: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-what-is- machine-learning/

Machine Learning in 5 Minutes The Formal one: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, as measured by P, improves with experience E.” A Practical Example: What is Machine Learning: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-what-is- machine-learning/ Look at data. Try something. Get the right answer? No? Look at the data. Do something different. Better? Yes? Do that again. (Repeat)

Machine Learning Algorithms Split into two main categories: Supervised learning Predicting the future Learn from known past examples to predict future Labels provided Unsupervised learning Making sense of data Understanding the past Learning the structure of data Labels no provided Algorithm Documentation: https://msdn.microsoft.com/library/dn905974.aspx Exploring: https://azuremlsimpleds.azurewebsites.net/simpleds/

Machine Learning Capabilities Which category (Classification) How much/many (Regression) Which group (Clustering, Recommender) Is it odd (Anomaly) Which action (Reinforcement Learning) Classification: Assign a category to each item (Chinese | French | Indian | Italian | Japanese restaurant). – Which Category? Regression: Predict a real value for each item (stock/currency value, temperature). – How much/how many? Clustering/Recommendation: Partition items into homogeneous groups (clustering twitter posts by topic). – Which Groups? Anomaly: Identify when something unexpected happens. – Is this weird? Reinforcement Learning: Make an appropriate action for some new data. – Which action?

The Cortana Intelligence Suite Example paths for using Azure ML: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-data-science- plan-sample-scenarios/

Cortana Intelligence in a Sentence: Cortana Intelligence is a Platform and a Process to perform advanced analytics from start to finish What you can do with CIS: https://www.microsoft.com/en- us/server-cloud/cortana-intelligence-suite/why-cortana- intelligence.aspx More about the process: https://channel9.msdn.com/Blogs/Seth- Juarez/Understanding-Data-Science-for-building- Predictive-Analytics-Solutions-by-Francesca-Lazzeri Data Science Blog: https://buckwoody.wordpress.com/

The Team Data Science Process Cross Industry Standard Process for Data Mining The Team Data Science Process Consume Deploying Train Models Evaluating Create Models Modeling Generate Features Data Preparation This process largely follows the CRISP-DM model: http://www.sv-europe.com/crisp-dm-methodology/ It also references the Cortana Intelligence process: https://azure.microsoft.com/en- us/documentation/articles/data-science-process-overview/ A complete process diagram is here: https://azure.microsoft.com/en- us/documentation/learning-paths/cortana-analytics- process/ Some walkthrough’s of the various services: https://azure.microsoft.com/en- us/documentation/articles/data-science-process- walkthroughs/ Explore and Visualize Data Understanding Planning, Environment, Ingest Business Understanding Data Science Blog: https://buckwoody.wordpress.com/

The Cortana Intelligence Platform Cortana, Cognitive Services, Bot Framework Power BI Azure Stream Analytics Azure HDInsight Azure Machine Learning and MRS Azure SQL DB, Data Warehouse, DocumentDB Azure Data Lake Platform and Storage: Microsoft Azure – http://microsoftazure.com Storage: https://azure.microsoft.com/en-us/documentation/services/storage/ (Host It) Azure Data Catalog: http://azure.microsoft.com/en-us/services/data-catalog (Doc It) Azure Data Factory: http://azure.microsoft.com/en-us/services/data-factory/ (Move It) Azure Event Hubs: http://azure.microsoft.com/en-us/services/event-hubs/ (Bring It) Azure Data Lake: http://azure.microsoft.com/en-us/campaigns/data-lake/ (Store It) Azure DocumentDB: https://azure.microsoft.com/en-us/services/documentdb/ , Azure SQL Data Warehouse: http://azure.microsoft.com/en-us/services/sql-data- warehouse/ (Relate It) Azure Machine Learning: http://azure.microsoft.com/en-us/services/machine- learning/ (Learn It) Azure HDInsight: http://azure.microsoft.com/en-us/services/hdinsight/ (Scale It) Azure Stream Analytics: http://azure.microsoft.com/en-us/services/stream-analytics/ (Stream It) Power BI: https://powerbi.microsoft.com/ (See It) Cortana: http://blogs.windows.com/buildingapps/2014/09/23/cortana-integration- and-speech-recognition-new-code-samples/  and https://blogs.windows.com/buildingapps/2015/08/25/using-cortana-to-interact-with- your-customers-10-by-10/ and https://developer.microsoft.com/en-us/Cortana  (Say It) Cognitive Services: https://www.microsoft.com/cognitive-services Bot Framework: https://dev.botframework.com/ Azure Event Hubs Azure Data Factory Azure Data Catalog Microsoft Azure

Machine Learning Tools and Setup Data Science for Beginners: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-data- science-for-beginners-the-5-questions-data-science- answers/

The Azure ML Environment Development Environment Creating Experiments Sharing a Workspace Deployment Environment Publishing the Model Using the API Consuming in various tools The Azure Machine Learning Studio: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-what-is-ml- studio/ Guided tutorials: https://azure.microsoft.com/en- us/documentation/services/machine-learning/ Microsoft Azure Virtual Academy course: https://mva.microsoft.com/en-US/training- courses/microsoft-azure-machine-learning-jump-start- 8425?l=ehQZFoKz_7904984382

Azure ML Elements Import Data Preprocess Algorithm Split Data Designing an experiment in the Studio: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-what-is-ml- studio/ Train Model Score Model

Azure ML Studio and the Team Data Science Process Walkthroughs of each step: https://azure.microsoft.com/en- us/documentation/learning-paths/data-science-process/

Creating an Experiment Create Workspace Deploy Model Consume Model Build and Model Get/Prepare Data Build/Edit Experiment Create/Update Model Evaluate Model Results Beginning Series: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-data- science-for-beginners-the-5-questions-data-science- answers/

Ingesting and Preparing Data Importing Data to Azure ML: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-data- science-import-data/

Inspecting data Keys to quality source data Authority Spread Consistency Types and Units Representation 1. In reference to machine learning, but applicable to all data usage: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-data-science- prepare-data/

Azure Storage Types: Create with: Blobs Tables Queues Files Azure Portal Azure PowerShell Azure Command Line Interface (CLI) Service Management REST API Azure Storage Resource Provider REST API Azure Portal - https://portal.azure.com/ Azure PowerShell - https://azure.microsoft.com/en- us/documentation/articles/storage-powershell-guide-full/ AZCOPY - https://azure.microsoft.com/en- us/documentation/articles/storage-use-azcopy/ Azure CLI - https://azure.microsoft.com/en- us/documentation/articles/storage-azure-cli/ Service management REST API - http://msdn.microsoft.com/library/azure/ee460799.aspx Azure Storage Resource Provider REST API - https://msdn.microsoft.com/library/azure/mt163683.aspx

Redundancy and Location LRS: 3 Copies, 1 Datacenter GRS: 6 Copies, 2 Datacenters ZRS: 3 Copies, 2-3 Datacenters Locations and Redundancy Overview: https://azure.microsoft.com/en- us/documentation/articles/storage-introduction/ Affects on Scalability and Performance Targets: https://azure.microsoft.com/en- us/documentation/articles/storage-scalability-targets/ Pricing Details: https://azure.microsoft.com/en- us/pricing/details/storage/

Tag the data descriptions Make it easy to find data in context Azure Data Catalog Register data sources Tag the data descriptions Make it easy to find data in context Use the data – keep it secure Full example: https://azure.microsoft.com/en- us/documentation/articles/data-catalog-get-started/

Options for Data Sourcing Import from local Import from Online Import from Experiment Getting data: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-data-science- import-data/ Data Science Blog: https://buckwoody.wordpress.com/

Algorithms

Clustering Grouping items based on defined Features https://azure.microsoft.com/en- us/documentation/articles/machine-learning-algorithm- choice/ https://msdn.microsoft.com/en- US/library/azure/dn906012.aspx https://msdn.microsoft.com/en- us/library/azure/dn913055.aspx https://msdn.microsoft.com/en- us/library/azure/dn905970.aspx https://msdn.microsoft.com/en- us/library/azure/dn905995.aspx

Classification Predicting the class or category for a single instance of data https://azure.microsoft.com/en- us/documentation/articles/machine-learning-algorithm- choice/ https://msdn.microsoft.com/en- US/library/azure/dn906012.aspx https://msdn.microsoft.com/en- us/library/azure/dn913055.aspx https://msdn.microsoft.com/en- us/library/azure/dn905970.aspx https://msdn.microsoft.com/en- us/library/azure/dn905995.aspx

Anomaly Detection Selecting items based on unusual or suspicious patterns https://azure.microsoft.com/en- us/documentation/articles/machine-learning-algorithm- choice/ https://msdn.microsoft.com/en- US/library/azure/dn906012.aspx https://msdn.microsoft.com/en- us/library/azure/dn913055.aspx https://msdn.microsoft.com/en- us/library/azure/dn905970.aspx https://msdn.microsoft.com/en- us/library/azure/dn905995.aspx

Regression Predicting the value of a datum given its history https://azure.microsoft.com/en- us/documentation/articles/machine-learning-algorithm- choice/ https://msdn.microsoft.com/en- US/library/azure/dn906012.aspx https://msdn.microsoft.com/en- us/library/azure/dn913055.aspx https://msdn.microsoft.com/en- us/library/azure/dn905970.aspx https://msdn.microsoft.com/en- us/library/azure/dn905995.aspx

Model Scoring and Evaluation Train and Evaluate your Model: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-walkthrough-4- train-and-evaluate-models/

Scoring a Model Apply a trained model to: A list of recommended items Forecasts for time series models Estimates of projected demand, volume, or other numeric quantity, for regression models Cluster assignments A predicted class or outcome, for classification models Probability scores associated with these outputs https://azure.microsoft.com/en- us/documentation/articles/machine-learning-algorithm- choice/ https://msdn.microsoft.com/en- US/library/azure/dn906012.aspx https://msdn.microsoft.com/en- us/library/azure/dn913055.aspx https://msdn.microsoft.com/en- us/library/azure/dn905970.aspx https://msdn.microsoft.com/en- us/library/azure/dn905995.aspx

Evaluating a Model Metrics for Classification Models Accuracy, Recall, Precision, F1-Score AUC Average Log Loss Training Log Loss Metrics for Regression Models Mean absolute error (MAE) Root mean squared error (RMSE) Relative absolute error (RAE) Relative squared error (RSE) Coefficient of determination Simple explanation of the ROC Curve: http://blog.revolutionanalytics.com/2016/08/roc-curves-in- two-lines-of-code.html https://msdn.microsoft.com/en- us/library/azure/dn906026.aspx https://azure.microsoft.com/en- us/documentation/articles/machine-learning-evaluate- model-performance/ https://msdn.microsoft.com/library/azure/75fb875d-6b86- 4d46-8bcc-74261ade5826 https://msdn.microsoft.com/library/azure/927d65ac-3b50- 4694-9903-20f6c1672089 https://msdn.microsoft.com/library/azure/e9ad68a7-e91b- 4ae6-800e-b5ee7e22cd17

Lab Create and Run an Azure ML Experiment Open this site, and follow the steps listed there: https://azure.microsoft.com/en- us/documentation/articles/machine-learning-walkthrough- develop-predictive-solution/

Publishing and Using the Model

Options for Data Access Azure ML API Code Push to Storage Power BI / Excel Access and read through this page: https://support.office.com/en-us/article/Connect-to- Microsoft-Azure-Blob-Storage-Power-Query-f8165faa- 4589-47b1-86b6-7015b330d13e?ui=en-US&rs=en- US&ad=US&fromAR=1 Access and read through this page: http://social.technet.microsoft.com/wiki/contents/articles/2 128.azure-and-sql-database-tutorials-tutorial-1-using- azure-web-role-and-azure-table-service.aspx Accessing storage using Code: https://azure.microsoft.com/en- us/documentation/articles/storage-dotnet-how-to-use- blobs/ Working with Azure Storage: https://azure.microsoft.com/en- us/documentation/services/storage/ Data Science Blog: https://buckwoody.wordpress.com/

© 2016 Microsoft Corporation. All rights reserved © 2016 Microsoft Corporation. All rights reserved. Microsoft, Windows, Microsoft Azure, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION