6/10/2018 4:48 AM THR2155 A/B Testing, Canary releases and dark launching: Implementing continuous delivery on Azure Marcel de Vries CTO, Xpirit © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Required technologies Feature toggles Deployment Slots 6/10/2018 4:48 AM Agenda Required technologies Release techniques Feature toggles Deployment Slots Traffic Management Telemetry A/B Testing Canary Releasing / Flighting Dark Launch © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Required technology

Feature toggles What is it? Why do you need it? Mechanism to separate feature deployment from feature exposure A.k.a. feature flippers, feature flags, feature switch, conditional feature, etc. Why do you need it? It enables you to give control back to the business on when to release the feature It enables test techniques (more on this later) It provides an alternative to keeping multiple branches in version control

Deployment slots What is it? Why do you need it? A way to set up multiple environments and swap between environments Why do you need it? When you want to deploy with zero downtime When you need to test in production When you want easy rollback

Traffic Management What is it? Why do you need it? Traffic manager provides the ability to route traffic between Azure app services Within an app service, routes traffic between deployment slots Why do you need it? It enables failover and load distribution capabilities It enables you to deploy to a slot and then slowly move traffic over to the other slot

Telemetry Get insights in performance and health of your system 6/10/2018 4:48 AM Telemetry Get insights in performance and health of your system Example: Application insights © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo Marcel de Vries CTO, Xpirit

Release techniques

A/B Testing What is it? Why do you need it? How do you do it? Experimental approach to identify changes that increase or maximize an outcome of interest Why do you need it? To collect data on a hypothesis to diminish or substantiate the hypothesis How do you do it? Combine deployment slots/traffic routing, feature toggles and feedback loops

Canary Release What is it? Why do you need it? How do you do it? Releasing a feature to a limited subset of end users Why do you need it? Validates there is no break When you want to gradually roll out a feature to ensure enough capacity How do you do it? Use a combination of feature toggles, traffic routing and deployment slots Random selection of users:

Dark Launch What is it? Why do you need it? Dark launch is like canary release, but the feature is invisible to the end user Why do you need it? To test a feature in production without the end user noticing it When you anticipate a big impact on production resources

Demo Marcel de Vries CTO, Xpirit

Xpirit mdevries@xpirit.com Fluentbytes.com @marcelv 6/10/2018 4:48 AM Xpirit All Photos used in this presentation come from unsplash.com mdevries@xpirit.com Fluentbytes.com @marcelv © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Xpirit xpir.it/XpiritMagazine mdevries@xpirit.com Fluentbytes.com Building a CD pipeline with VS 2015 Test Automation with CodedUI Testing Web Applications with CodedUI Introduction to Docker on Windows with Visual Studio 2017 All Photos used in this presentation come from unsplash.com mdevries@xpirit.com Fluentbytes.com @marcelv

Please evaluate this session Tech Ready 15 6/10/2018 Please evaluate this session From your Please expand notes window at bottom of slide and read. Then Delete this text box. PC or tablet: visit MyIgnite https://myignite.microsoft.com/evaluations Phone: download and use the Microsoft Ignite mobile app https://aka.ms/ignite.mobileapp Your input is important! © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6/10/2018 4:48 AM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.