Scott Schnoll Exchange Server 2013 Site Resilience

Agenda The Preferred Architecture Namespace Planning and Principles Datacenter Switchovers and Failovers Dynamic Quorum and DAGs

The Preferred Architecture

Site Resilience changes in Exchange 2013 Tech Ready 15 4/13/2017 Site Resilience changes in Exchange 2013 Frontend/Backend recovery are independent Most protocol access in Exchange Server 2013 is HTTP DNS resolves to multiple IP addresses HTTP clients have built-in IP failover capabilities Clients skip past IPs that produce hard TCP failures Namespace no longer a single point of failure Single or multiple namespace options Admins can switchover by removing VIP from DNS or disabling No dealing with DNS latency © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Preferred Architecture Namespace Design 4/13/2017 9:08 PM Preferred Architecture Namespace Design For a site resilient datacenter pair, a single namespace / protocol is deployed across both datacenters autodiscover.contoso.com HTTP: mail.contoso.com IMAP: imap.contoso.com SMTP: smtp.contoso.com Load balancers are configured without session affinity, one VIP / datacenter Round-robin, geo-DNS, or other solutions are used to distribute traffic equally across both datacenters mail VIP mail VIP © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Preferred Architecture DAG Design 4/13/2017 9:08 PM Preferred Architecture DAG Design Each datacenter should be its own Active Directory site Deploy unbound DAG model spanning each DAG across two datacenters Distribute active copies across all servers in the DAG Deploy 4 copies, 2 copies in each datacenter One copy will be a lagged copy (7 days) with automatic play down enabled Native Data Protection is used Single network is used for MAPI and replication traffic Third datacenter used for Witness server, if possible Increase DAG size density before creating new DAGs mail VIP mail VIP DAG Witness Server © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Preferred Architecture Selina (somewhere in NA) Batman (somewhere in Europe) na.contoso.com eur.contoso.com DNS Resolution DNS Resolution na VIP na VIP eur VIP eur VIP DAG DAG

Namespace Planning & Principles

Namespace Planning No need for namespaces required by Exchange 2010 Can still deploy regional namespaces to control traffic Can still have specific namespaces for protocols Two namespace models Bound Model Unbound Model Leverage split-DNS to minimize namespaces and control connectivity Deploy separate namespaces for internal and external Outlook Anywhere host names

Bound Model Sue mail.contoso.com mail2.contoso.com Jane mail VIP (somewhere in NA) mail.contoso.com mail2.contoso.com Jane (somewhere in NA) DNS Resolution DNS Resolution mail VIP mail2 VIP DAG1 Active Passive DAG2 Passive Active

Unbound Model Sue mail.contoso.com VIP #1 VIP #2 DAG DNS Resolution (somewhere in NA) mail.contoso.com DNS Resolution Round-Robin between # of VIPs VIP #1 VIP #2 DAG

Load Balancing Exchange 2013 no longer requires session affinity to be maintained on the load balancer For each protocol session, CAS now maintains a 1:1 relationship with the Mailbox server hosting the user’s data Load balancer configuration and health probes will factor into namespace design Remember to configure health probes to monitor healthcheck.htm, otherwise LB and MA will be out of sync

Single Namespace / Layer 4 TechReady 16 4/13/2017 Single Namespace / Layer 4 CAS OWA ECP EWS EAS OAB MAPI RPC AutoD health check User mail.contoso.com Layer 4LB autodiscover.contoso.com © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Single Namespace / Layer 7 TechReady 16 4/13/2017 Single Namespace / Layer 7 CAS OWA ECP EWS EAS OAB MAPI RPC AutoD Health check executes against each virtual directory health check User mail.contoso.com Layer 7LB autodiscover.contoso.com © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Multiple Namespaces / Layer 4 TechReady 16 4/13/2017 Multiple Namespaces / Layer 4 User CAS OWA ECP EWS EAS OAB MAPI RPC AutoD mail.contoso.com ecp.contoso.com ews.contoso.com eas.contoso.com Layer 4LB oab.contoso.com oa.contoso.com mapi.contoso.com autodiscover.contoso.com © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Datacenter Switchovers and Failovers

Witness Server Placement Microsoft Exchange 4/13/2017 Witness Server Placement New Witness Server placement options available Choose based on business needs and available options Third location DAG witness server improves DAG recovery behaviors Automatic recovery on datacenter loss; Third location network infrastructure must have independent failure modes Deployment scenario Recommendations DAG(s) deployed in a single datacenter Locate witness server in the same datacenter as DAG members; can share one server across DAGs DAG(s) deployed across two datacenters; No additional locations available Locate witness server in primary datacenter; can share one server across DAGs DAG(s) deployed across two+ datacenters Locate witness server in third location; can share one server across DAGs © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Site Resilience - CAS With multiple VIP endpoints sharing the same namespace, if one VIP fails, clients automatically failover to alternate VIP! Removing failing IP from DNS puts you in control of in service time of VIP X mail.contoso.com: 10.0.1.50 mail.contoso.com: 192.168.1.50, 10.0.1.50 primary datacenter: Redmond alternate datacenter: Portland VIP: 192.168.1.50 VIP: 10.0.1.50 cas1 cas2 cas3 cas4

Site Resilience - Mailbox Assuming MBX3 and MBX4 are operating and one of them can lock the witness.log file, automatic failover should occur X primary datacenter: Redmond alternate datacenter: Portland mbx1 mbx2 mbx3 mbx4 third datacenter: Stockholm witness

Site Resilience - Mailbox Tech Ready 15 4/13/2017 Site Resilience - Mailbox Mark the failed servers/site as down: Stop-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite:Redmond Stop the Cluster Service on Remaining DAG members: Stop-Clussvc Activate DAG members in 2nd datacenter: Restore-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite:Portland X X X primary datacenter: Redmond alternate datacenter: Portland mbx1 mbx2 mbx3 mbx4 witness © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Site Resilience - Mailbox Mark the failed servers/site as down: Stop-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite:Redmond Stop the Cluster Service on Remaining DAG members: Stop-Clussvc Activate DAG members in 2nd datacenter: Restore-DatabaseAvailabilityGroup DAG1 –ActiveDirectorySite:Portland X primary datacenter: Redmond alternate datacenter: Portland mbx1 mbx2 mbx3 mbx4 alternate witness witness

Activation Block Comparison Tool Parameter Value Instance Usage Suspend-MailboxDatabaseCopy ActivationOnly N/A Per database copy Keep active off a working but questionable drive Set-MailboxServer DatabaseCopyAutoActivationPolicy “Blocked” or “Unrestricted” Per server Used to control active/passive SR configurations and maintenance Can force admin move DatabaseCopyActivationDisabledAndMoveNow $true or $false Used to do faster site failovers and maintain database availability Databases are not blocked from failing back Continuous move-off operation

DatabaseDisabledAndMoveNow New server setting to improve site resilience Get all active databases off server – FAST! Last resort to not move an active! Proactively continue move databases attempts Server can still be in service Databases mounted and mail delivery!

Best Practices Automate your recovery logic; make it reliable Think of it as rack/site maintenance Exercise it regularly Recovery times directly dependent on detection & decision times! Flip the bit! Don’t ask repair times, “if outage go…” Humans are the biggest threat to recovery times

Dynamic Quorum and DAGs MEC 2014 4/13/2017 9:08 PM Dynamic Quorum and DAGs © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Dynamic Quorum In Windows Server 2008 R2, quorum majority is fixed, based on the initial cluster configuration In Windows Server 2012 (and later), cluster quorum majority is determined by the set of nodes that are active members of the cluster at a given time This new feature is called Dynamic Quorum, and it is enabled for all clusters by default

Dynamic Quorum Cluster dynamically manages vote assignment to nodes, based on the state of each node When a node shuts down or crashes, the node loses its quorum vote When a node rejoins the cluster, it regains its quorum vote By adjusting the assignment of quorum votes, the cluster can dynamically increase or decrease the number of quorum votes required to keep running

Dynamic Quorum By dynamically adjusting the quorum majority requirement, a cluster can sustain sequential node shutdowns to a single node This is referred to as a “Last Man Standing” scenario

Dynamic Quorum Does not allow a cluster to sustain a simultaneous failure of majority of voting members To continue running, the cluster must always maintain quorum after a node shutdown or failure If you manually remove a node’s vote, the cluster does not dynamically add the vote back

Dynamic Quorum Majority of 7 required

Dynamic Quorum Majority of 7 required Majority of 4 required X X X

Dynamic Quorum Majority of 3 required X

Dynamic Quorum Majority of 2 required X

Dynamic Quorum Majority of 2 required X X X X X

Dynamic Quorum Majority of 2 required X X X X X 1

Dynamic Quorum Majority of 2 required X X 1 X X X

Dynamic Quorum Majority of 2 required X X X 1 X X X

Dynamic Quorum Majority of 2 required X X X 1 X X X X

Dynamic Quorum Name DynamicWeight NodeWeight State ---- ------------- Use Get-ClusterNode to verify votes 0 = does not have quorum vote 1 = has quorum vote Get-ClusterNode <Name> | ft name, *weight, state Name DynamicWeight NodeWeight State ---- ------------- ---------- ----- EX1 1 Up

Dynamic Quorum Works with most DAGs Third-party replication DAGs not tested All internal testing has it enabled Office 365 servers use it Exchange is not dynamic quorum-aware Does not change quorum requirements

Dynamic Quorum Cluster team guidance: Exchange team guidance: Generally increases the availability of the cluster Enabled by default, strongly recommended to leave enabled Allows the cluster to continue running in failure scenarios that are not possible when this option is disabled Exchange team guidance: Leave it enabled for majority of DAG members In some cases where a Windows 2008 R2 DAG would have lost quorum, a Windows 2012 DAG can maintain quorum Don’t factor it into availability plans

Dynamic Witness Witness Offline Witness Failure Witness Online 4/13/2017 Dynamic Witness Windows Server 2012 R2 and later Witness Offline Witness vote gets removed by the cluster Witness Failure Witness vote gets removed by the cluster Witness Online If necessary, Witness vote is added back by the cluster © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Questions?