7/30/2018 8:21 AM BRK3209 Managing, securing, and scaling Azure Service Fabric clusters and applications Chacko Daniel Principal Program Manager © Microsoft.

7/30/2018 8:21 AM BRK3209 Managing, securing, and scaling Azure Service Fabric clusters and applications Chacko Daniel Principal Program Manager © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Service Fabric Azure Client Modules
Build 2014 7/30/2018 8:21 AM Agenda Service Fabric Azure Client Modules Best practices for planning your cluster Best practices for business continuity planning Best practices for securing your cluster Application and Services as ARM resources Monitoring and diagnostics Road map © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Azure Service Fabric Any OS, Any Cloud Dev Box Azure
Microsoft Build 2017 7/30/2018 8:21 AM Any OS, Any Cloud Lifecycle Management Always On Availability Programming Models Health & Monitoring Dev & Ops Tooling Auto scaling Orchestration SDK Available Managed Service Deployment Packages Coming soon Deployment Packages Coming soon Dev Box Azure On-Premise Data centers Other Clouds Azure Stack Coming soon © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Service Fabric Client Modules
Microsoft Build 2017 7/30/2018 8:21 AM Service Fabric Client Modules AzureRM.ServiceFabric powershell module az sf cli module Automates many E2E scenarios Creating a secure cluster Adding or removing nodes from a cluster Adding or removing NodeTypes from a cluster Adding new cluster and client certificates to a cluster Switching the cluster upgrade mode from Automatic to Manual or vice-versa Updating service fabric settings Ships as a part of Azure PowerShell and CLI respectively Both the modules are supported in cloud shell In preview for now, so feedback is appreciated © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

New-AzureRmServiceFabricCluster
Microsoft Build 2017 7/30/2018 8:21 AM New-AzureRmServiceFabricCluster Ships with four parameter sets Supports majority of ‘create new service fabric cluster’ scenarios Example with minimum set of parameters - create a secure cluster with a self signed cert, and optionally downloading it locally: New-AzureRmServiceFabricCluster -ResourceGroupName $RGname -Location $clusterloc -ClusterSize $numNodes -VmPassword $pwd -CertificateSubjectName $subname -CertificatePassword $pwd -CertificateOutputFolder $pfxfolder Let us see this in action. © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Create a secure cluster using PowerShell
Microsoft Build 2017 7/30/2018 8:21 AM Demo: Create a secure cluster using PowerShell Demo #1 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scaling an Azure cluster in or out
Microsoft 2016 7/30/2018 8:21 AM Scaling an Azure cluster in or out Safest option: Use the Azure PS or CLI to perform this operation Add nodes Add-AzureRmServiceFabricNode -ResourceGroupName $RGname -Name $clusterName -NodeType $nodeType -Number $addNumNodes az sf cluster node add --cluster-name --node-type --number-of-nodes-to-add --resource-group Remove nodes Remove-AzureRmServiceFabricNode -ResourceGroupName $RGname -Name $clusterName -NodeType $nodeType -Number $addNumNodes az sf cluster node remove --cluster-name --node-type --number-of-nodes-to-remove --resource-group Let us see this in action. © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scaling out a cluster using CLI
Microsoft Build 2017 7/30/2018 8:21 AM Demo: Scaling out a cluster using CLI Demo #2 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Service Fabric Cluster Planning
Microsoft 2016 7/30/2018 8:21 AM Service Fabric Cluster Planning Key points: Capacity planning is not an easy exercise Capacity planning requires periodic reassessment Add capacity on demand is not instantaneous Incurring downtime in the future to change capacity may not work Read more on cluster capacity planning in this document © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft 2016 7/30/2018 8:21 AM Service Fabric Cluster Planning Define what the cluster will be used for Is this to be used for Test? Is this a part of the CI/CD pipeline? Is this for Production use? Determine the node types and sizes For each application planned to be deployed – do a sizing exercise Ports to be opened etc. Are there any unique compliance or security requirements? Compliance expectations from infrastructure and applications © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Microsoft 2016 7/30/2018 8:21 AM Service Fabric Cluster Planning Where do you want this cluster hosted? On Azure? On-Premise, in your data center? On some other cloud provider? Choose the # of Fault Domains (FDs) This determines the headroom needed in case of unplanned failures Choose the # of Upgrade Domains (UDs) This determines the headroom needed in case of planned failures © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

System service replica set size
7/30/2018 8:21 AM Reliability tier Sets the number of system services replicas. Determines who reliable your entire cluster is. Determines what the primary nodetype size is The node SKU/Size has no bearing on reliability tier Reliability Tier System service replica set size Bronze 3 Silver 5 Gold 7 Platinum 9 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Best Practices : Reliability tier
7/30/2018 8:21 AM Best Practices : Reliability tier Run production workloads in Azure at Silver or higher Choose the highest tier you can afford Adjust the reliability tier on scale out or in For a one node test cluster, do not specify the reliability tier Use the same guidance for your stateful services as well Reliability Tier System service replica set size Recommended for Cluster Size of Bronze 3 Silver 5 5 or 6 Gold 7 7 or 8 Platinum 9 9 and up; Cross AZ or Cross Region © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Full node SKUs like D15_V2, G5
7/30/2018 8:21 AM Durability tier Indicates to the system the privileges that your VMs have with the underlying Azure infrastructure Indicates to the system to favor data safety over operational speed Will delay all infrastructure operations Scale out and in operations become easier Does not apply to the on premise clusters Durability Tier Infra privilege time out Available on VMs SKUs Bronze 0 mins All supported VM SKUs Silver 10 mins Gold 120 mins Full node SKUs like D15_V2, G5 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Best Practices : Durability tier
7/30/2018 8:21 AM Best Practices : Durability tier Keep your cluster and applications healthy at all times Respond to all Service replica lifecycle events timely Adopt safer ways to make a VM SKU change (Scale up/down) Set expectation :The cluster level and infra operations will slow down Durability Tier Infra privilege time out Available on VMs SKUs Bronze 0 mins All supported VM SKUs Silver 10 mins Gold 120 mins Full node SKUs like D15_V2, G5 Read this carefully : © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Business Continuity Planning
Microsoft Build 2017 7/30/2018 8:21 AM Business Continuity Planning Keep a written, updated, Business Continuity Define what your RPO and RTO are RPO - The Recovery Point Objective (RPO) determines the amount of data you can afford to lose in a disaster RTO - The Recovery Time Objective (RTO) is the maximum tolerable length of time that your service can be down after a disaster occurs Backup your application state to meet your RPO Set up clusters across Availability Zones or Even Regions. © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What is an “Availability Zones” (AZ)
Availability Zones (AZ) are physically separated locations within an Azure region 3 zones in each AZ-enabled region Each zone has independent power, network, and cooling. Each AZ has 1 to infinite # of DCs. No DC shared by two zones. VM to VM network latency: Intra-AZ < 1ms, inter-AZ < 2ms

Cluster across three Availability Zones.
7/30/2018 8:21 AM Cluster across three Availability Zones. Traffic Manager VMSS #1 Subnet #1 Single VNet VMSS #2 Availability Zone #1 Availability Zone #2 Subnet #3 Availability Zone #3 Service Fabric Cluster VMSS #3 Single Vnet Three zonal VMSS Thee load balancers One traffic manager © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Cross Availability Zones Service Fabric cluster
Microsoft Build 2017 7/30/2018 8:21 AM Cross Availability Zones Service Fabric cluster Use a cross AZ cluster when Want to survive an azure zone going down RPO and RTO = 0 Slightly higher write latency than a cluster in a single AZ Back up your application State irrespective To be used for regional failures and OOPS recovery Minimum Cluster size Has to be across 3 or more AZs Primary node type - 9 VMs minimum © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Cross regional Service Fabric cluster
Microsoft Build 2017 7/30/2018 8:21 AM Cross regional Service Fabric cluster Has to span at least 3 regions. Use it when the Want to survive an azure region going down RPO and RTO = 0 Higher write latency than across AZ is acceptable Back up your application State To be used for OOPS recovery Minimum Cluster size Has to be across 3 or more Regions Primary node type – 9 VMs minimum © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Cross regional Service Fabric Cluster
Microsoft 2016 7/30/2018 8:21 AM Demo: Cross regional Service Fabric Cluster Demo #3 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Backup & Restore – Azure & on-premise
Backup and Restore Service Supports both Reliable Actors and reliable Services. Allows for custom backup policies– backup schedule, retention, backup location etc Supports backup policies at application, service or partition level. Works on both Azure and on-premise clusters. Supports enumerating backups and viewing them in Backup Explorer Supports restoration of a partition from a backups. How do I enable it in my Cluster ? Create xstore account (Azure) or file share for uploading backups Enable Backup Restore service by adding it as addOnFeature. Best Practice Enable backup service on your cluster, if you run stateful services in your cluster Preview – Mid October

Cluster Manifest – Code Snippet
7/30/2018 8:21 AM Cluster Manifest – Code Snippet "apiVersion": " preview", "type": "Microsoft.ServiceFabric/clusters", ... "properties": { "addonFeatures": [ “BackupRestoreService” ], © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Back up Explorer

Back up and Restore service
Microsoft 2016 7/30/2018 8:21 AM Demo: Back up and Restore service Review the ARM template Show SFX and the system service Demo #4 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Disasters and suggested mitigations
Microsoft Build 2017 7/30/2018 8:21 AM Disasters and suggested mitigations Types of Disasters RPO and RTO = 0, Write latency acceptable RPO and RTO > 0 Regional Outage Cross-regional SF cluster Stand up a new cluster, restore from backup Data Center Outage Cross AZ (3+) SF cluster Cluster down (Very low probability for cross-regional clusters) Machine / Node down Deploy across 5+ FDs, 5+ UDs, Design for write quorum losses Other sources of data loss or “oops” Restore from backup © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Best practices: Cluster setup in Azure
Microsoft 2016 7/30/2018 8:21 AM Best practices: Cluster setup in Azure Use the ARM template to customize your cluster Setup managed storage for VM VHDs Use the ARM template to drive changes to your Resource Group Easy configuration management Auditing Avoid using implicit commands to tweak your resources Treat your cluster configuration as code Be thorough in checking the configurations you choose to deploy © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Applications and Services as Resources
7/30/2018 8:21 AM Applications and Services as Resources What is it ? Applications and Services are now ARM resources. Allows you to leverage features like RBAC, Auditing of operations etc. How do I get it ? Use the API version preview Cluster runtime is 6.0 or above.. When do I use it RBAC © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Applications and Services
Microsoft 2016 7/30/2018 8:21 AM Demo: Applications and Services Demo #5 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Best practices: Cluster Security in Azure
7/30/2018 8:21 AM Best practices: Cluster Security in Azure Always use a secure cluster Cluster node to node security – use Certificates Client access (Admin and Read only) – use AAD Use automated deployments Use scripts to generate, deploy and rollover secrets Keep the secrets in KV, use AD for all other client access No human should have access to them without authentication. Additionally consider the following: Create DMZs using Network Security Groups (NSGs) Use Jump servers to RDP into cluster VMs or to manage your cluster © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Best practices: Cluster Security in Windows Server (Standalone)
7/30/2018 8:21 AM Best practices: Cluster Security in Windows Server (Standalone) Always use a secure cluster: use AD Cluster security: gMSA Client access (Admin and Read-only): use AD Use automated deployments Use scripts to generate, deploy, and rollover secrets No human should have access to them without authentication Additionally consider the following: Create DMZs using Network Security Groups (NSGs) in your load balancer Use Jump servers to RDP into cluster VMs or to manage the cluster © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Monitoring and Diagnostics
Microsoft 2016 7/30/2018 8:21 AM Monitoring and Diagnostics Service Fabric out-of-the-box monitoring Operational events (high level cluster and node events) Health reports and load balancing decisions made by the system Data and messaging events Reliable Services events Reliable Actors events System logs (used by us to provide support for your cluster) Best practices Always enable diagnostics Generate custom traces in your applications and services Set up automated monitoring alerts Watchdog Service (github.com/Azure-Samples/service-fabric-watchdog-service) Read more on monitoring and diagnostics in this document © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo: Sample – Watchdog Demo Demo #7 Philips Hue Microsoft 2016
© 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Scenarios to monitor and why
Microsoft Build 2017 7/30/2018 8:21 AM Scenarios to monitor and why Cluster and Node state Is the cluster healthy? Are all the nodes up? Detect and diagnose hardware and infrastructure issues Application and Service state Upgrade status, number of services and replicas Detect software and app issues, reduce service downtime Resource Usage Do all the nodes need to be up? What is the average CPU usage? Understand resource consumption and drive better business decisions Performance Tracking Is there any unexpected latency? Are the services responsive? Optimize application, service, and infrastructure performance Custom Application Metrics Is your app being used in the way that you expected? Is solution effective? Generate business insights and improvements © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Near real time alerting with Azure Monitor
Microsoft 2016 7/30/2018 8:21 AM Demo: Near real time alerting with Azure Monitor Demo #6 © 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

The Road Ahead Patch Orchestration Application
Microsoft Build 2017 7/30/2018 8:21 AM The Road Ahead Patch Orchestration Application Stand alone offering for Linux clusters © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Recap Service Fabric Azure PowerShell or CLI Module
Microsoft Build 2017 7/30/2018 8:21 AM Recap Service Fabric Azure PowerShell or CLI Module Best practices for planning your cluster Best practices for securing your cluster Best practices for business continuity planning Application and services as ARM resources Monitoring and diagnostics Road map © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Service Fabric Sessions at Ignite
7/30/2018 8:21 AM Service Fabric Sessions at Ignite Day Time Session Code Session Room Monday Sept 25th 4 PM – 5.15 PM BRK3208 Azure Service Fabric overview and the road ahead OCCC S310 Tuesday Sept 26th 12.30 PM – 1.45 PM BRK3331 Azure Service Fabric for Linux OCCC W224 Wednesday Sept 27th 12:05 PM - 12:25 PM THR2159 Monitor your microservices with Application Insights OCCC South Expo Theater #1 2:15 PM - 3:30 PM BRK3189 Modernizing existing .NET applications with Windows Containers and Azure cloud 2.15 PM – 3.30 PM BRK2190 Orchestrating one million containers with Azure Service Fabric OCCC W208 AB BRK3209 Managing, securing, and scaling Azure Service Fabric clusters and applications Any time HOL3041 Deploying and scaling Windows and Linux containers on Azure Service Fabric Hands-on Labs Room You are already here © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Service Fabric resources
7/30/2018 8:21 AM Service Fabric resources Service Fabric developer SDK Service Fabric documentation Learn from samples, free clusters and labs Comprehensive resources list Questions, comments, issues

7/30/2018 8:21 AM BRK3209 Managing, securing, and scaling Azure Service Fabric clusters and applications Chacko Daniel Principal Program Manager © Microsoft.

Similar presentations

Presentation on theme: "7/30/2018 8:21 AM BRK3209 Managing, securing, and scaling Azure Service Fabric clusters and applications Chacko Daniel Principal Program Manager © Microsoft."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

7/30/2018 8:21 AM BRK3209 Managing, securing, and scaling Azure Service Fabric clusters and applications Chacko Daniel Principal Program Manager © Microsoft.

Similar presentations

Presentation on theme: "7/30/2018 8:21 AM BRK3209 Managing, securing, and scaling Azure Service Fabric clusters and applications Chacko Daniel Principal Program Manager © Microsoft."— Presentation transcript:

Similar presentations

About project

Feedback