Microsoft Ignite 2016 11/17/2018 2:42 AM BRK2223 Achieve Extreme Advanced Analytics Optimization using SQL Server R Services Justin Silver Scientist, PROS, Inc. Joe Sack Principal PM, Microsoft © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Session Agenda How PROS optimized their process with SQL Server R Services Legacy pain points Understanding the PROS Advanced Analytics usage scenario

Revenue & Profit Realization PROS provides a real-time software solution platform to help companies drive pricing & sales effectiveness Reduce Quote Turnaround Time Simplify Complex Product Catalogs Raise Win Rates Grow Deal Sizes Increase Margins Reduce Sales Cycle Duration Increase Quota Attainment

Segmentation Foundation for PROS smart pricing guidance Pricing Optimization Expert Target Floor Group customers, products & transactions into micro-segments of similar willingness-to-pay Apply optimization algorithms to target the pricing envelope ‘sweet spot’ in every segment

Pricing segmentation process Identify appropriate set of business attributes and model how they can be used with statistical and practical soundness to provide intelligent pricing benchmarks for future transactions. Customer data export Handoff to PROS Data science process Configuration files Sent to customer and consumed by app Attribute Selection Segmentation Scoring

Pricing segmentation: Pain points Bring data science to the data Sometimes difficult to get our current tools in same place as where customer data lives Especially challenging for Cloud implementations, which are now the norm Connect data science with products Lack of integration of segmentation analysis with some products Generate valid results quickly Current homegrown tools can struggle with performance for large data sets

Why R? What is R? Who uses R? Why use R? 11/17/2018 2:42 AM Why R? What is R? A language and an open-source statistical computing environment R language + graphical facilities + data analysis tools and methods Who uses R? 2M+ statisticians, scientists, data scientists, data analysts, engineers Why use R? Free and easy to start prototyping/exploring/modeling Embraced by academics and research Extensible – > 8,000 available community contributed packages (CRAN: Comprehensive R Archive Network) © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Common Pain Points with Open Source R 11/17/2018 2:42 AM Common Pain Points with Open Source R Performance of data exploration activities may be slow (data locality | single-threaded | memory constraints) Data may leave “trust boundary” Solutions may not be easily operationalized, requiring re-coding in other solutions Data Scientist may not be aware of already-cleansed and collected data Data Scientist may be using data cleansing techniques which would be more efficiently performed by SQL Server Database Engineer may be facilitating analytic and statistical operations which would be better performed using R © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Interact directly with data 11/17/2018 2:42 AM SQL Server R Services Example Solutions Fraud detection Sales forecasting Warehouse efficiency Predictive maintenance Extensibility R R Integration Data Scientist Package Library Interact directly with data 010010 100100 010101 010010 100100 010101 Data Developer/DBA Manage data and analytics together T-SQL Interface Relational Data Built-in to SQL Server 2016 © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SQL Server R Services and PROS Participated in Microsoft SQLCAT lab engagement 1 week lab engagement (PROS, MSFT) Evaluate performance improvement potential of SQL/R using Segmentation process as test case; comparing to Excel/C++/Python tools as baseline Expand PROS science modeling utilizing ScaleR libraries Opportunity for Microsoft PMs and Engineering to work directly with PROS, get feedback, identify bugs, and optimize

11/17/2018 2:42 AM Lab Workflow Verify R scripts are in a working state with local 20M row data set (5GB compressed) via IDE Migrate to sp_execute_external_script Confirm correct results Measure baseline performance characteristics Weave in performance optimizations Re-test and re-measure, ensuring correct results along the way © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Attribute Selection Tool Segmentation & Scoring Tool 11/17/2018 2:42 AM Before After Attribute Selection Tool Excel Tool SQL Server DB SQL Server DB C++ code Attribute Selection sp_execute_external_script ‘R code’ Data tables Data tables Segmentation & Scoring Tool Excel Tool Segmentation sp_execute_external_script ‘R code’ Stored Proc Scoring sp_execute_external_script ‘R code’ C++ code © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/17/2018 2:42 AM Before After © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2+ days in current system 11/17/2018 2:42 AM The Results Attribute Selection Segmentation Batch Scoring 2+ days in current system 45 mins 5 mins 2 mins Current System SQL Server 2016 R © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

How was this achieved? 11/17/2018 2:42 AM Leveraged SQL Server machine’s compute context to benefit from the resources of the SQL Server machine Leveraged ScaleR function equivalents whenever possible (rxDForest, rxDTree, rxPredict) Adjusting parameters for ScaleR functions further improved performance (described next) © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Number of Trees for rxDForest 11/17/2018 2:42 AM Number of Trees for rxDForest © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Number of Tasks and rxForest 11/17/2018 2:42 AM Number of Tasks and rxForest © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Number of Tasks and rxDTree 11/17/2018 2:42 AM Number of Tasks and rxDTree © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Tips for successful migrations 11/17/2018 2:42 AM Tips for successful migrations Ensure both the scientists and DBAs are engaged For scenarios where multiple packages are needed, see “Create a Local Package Repository Using miniCRAN” https://aka.ms/minicran Ensure validity of scripts in sp_execute_external_script before optimizing Make one change at a time – testing the impact of changes on results and performance Leverage SQL Server native optimizations and R-function equivalents where available Narrow down your SELECT list to only the attributes needed (13 hours vs. 5 minutes in one rxCreateColInfo scenario) Your results will vary based on your data characteristics, so be sure to experiment with thread counts, batch size, and algorithm-based configurations © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Call to Action Consider which scenarios may benefit from SQL Server R Services within your company or with your customers Want to get started? See Early Customer Experiences with SQL Server R Services https://aka.ms/earlysqlr

Free IT Pro resources To advance your career in cloud technology Microsoft Ignite 2016 11/17/2018 2:42 AM Free IT Pro resources To advance your career in cloud technology Plan your career path Microsoft IT Pro Career Center www.microsoft.com/itprocareercenter Cloud role mapping Expert advice on skills needed Self-paced curriculum by cloud role $300 Azure credits and extended trials Pluralsight 3 month subscription (10 courses) Phone support incident Weekly short videos and insights from Microsoft’s leaders and engineers Connect with community of peers and Microsoft experts Get started with Azure Microsoft IT Pro Cloud Essentials www.microsoft.com/itprocloudessentials Demos and how-to videos Microsoft Mechanics www.microsoft.com/mechanics Connect with peers and experts Microsoft Tech Community https://techcommunity.microsoft.com © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Please evaluate this session 11/17/2018 2:42 AM Please evaluate this session Your feedback is important to us! From your PC or Tablet visit MyIgnite at http://myignite.microsoft.com From your phone download and use the Ignite Mobile App by scanning the QR code above or visiting https://aka.ms/ignite.mobileapp © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11/17/2018 2:42 AM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.