1. 02 | Cluster Deployment & Upgrades
Elden Christensen | Principal Program Manager Lead | Microsoft
Symon Perriman | Vice President | 5nine Software

2. Module Overview Installation Active Directory Considerations
Cluster Validation
Cluster Creation
Cluster Management
Quorum
Cluster Upgrades

3. Installation

4. SKU Selection Microsoft Hyper-V Server 2012 R2
Windows Server 2012 R2 Standard
Windows Server 2012 R2 Datacenter
No guest OS licenses
2 guest OS licenses
Unlimited guest licenses
Host OS is FREE
Licensed per 2 CPUs
Licensed per 2 CPUs
CLI & Remote Management only
Full installation or Server Core
Full installation or Server Core

5. Microsoft Hyper-V Server 2012 R2
Consistent core functionality with Windows Server 2012 R2
Free Server SKU: aka.ms/HyperVserver (RTM)
Enterprise-class Microsoft hypervisor
CLI, remote GUI management or 3rd party add-ons
Does not include guest OS licenses
Contains all Hyper-V & Clustering features
8,000 VMs/cluster
Cluster Shared Volumes (CSV) 2
All types of live migration

6. OS Installation
Core
MinShell
Full
Full with Desktop Experience

7. Failover Cluster Installation
Install Failover Clustering Feature
Launch Failover Cluster Manager

8. Active Directory Considerations

9. Cluster Domain Integration Modes
AD Integrated
Traditional Clusters
Computer objects created for each cluster name
Enables Kerberos authentication against network names
AD-detached
New in Windows Server 2012 R2
Only DNS records created for each cluster name
Targeted primarily for SQL Server deployments

10. Cluster Fundamental Domain Requirements
Nodes must all be joined to an Active Directory domain
Nodes must all be members of the same domain
Nodes must be Member servers
Not supported for cluster nodes to Domain Controllers
Must be at least Domain Functional Level of Windows Server 2003
Recommended to be Domain Functional Level of Windows Server 2008 or higher
Recommended to have all node computer objects in same OU
Ensures group policies are applied consistently
Windows Server 2012 R2 DC required if using BitLocker

11. Active Directory Objects Overview
Organizational Unit (OU)
Computer Object (CO)
Cluster Name Object (CNO)
Computer Object (CO)
Virtual Computer Object (VCO)
Computer Object (CO)
Virtual Computer Object (VCO)
Computer Object (CO)
Member Servers

12. Cluster Integration with Active Directory
Computer Objects created for names associated with clusters
Enables Kerberos authentication against clusters
Cluster Name Object (CNO)
Computer Object associated with the “Cluster Name”
Common identity across cluster (similar to a Managed Service Account)
Identity to create DNS records in a secure DNS zone
Used for cluster management and intra-cluster authentication
Virtual Computer Object (VCO)
Computer Object associated with all other cluster Network Name resources
Used by external clients to authenticate against the clustered applications

13. Creating a New Cluster - CNO
Privileges of locally logged on user running admin tool used to access AD and create computer objects
Admin UI
Active Directory
CNO
Administrator logs on with domain account with domain permissions
Admin credentials used to access AD to create CNO
Computer account with same name as cluster
Cluster Name Object (CNO)

14. Making Roles Highly Available - VCO
Privileges of CNO used to access AD and create VCO computer objects
Admin UI
CNO
Active Directory
CNO
VCO
Cluster Service uses CNO credentials to access AD
CNO credentials used to create VCO
VCO created in Same OU as CNO
Administrator logs on with account with Cluster permissions
Virtual Computer Object (VCO)

15. OU Permissions for Cluster Setup
User locally logged on creating cluster
Create Computer objects
Read All Properties
Reset Password
Note: CNO created in user’s context (aka. They will be Creator / Owner)
CNO
Create Computer Objects
Note: VCO created under security context of CNO

16. Where are Cluster Computer Objects are Created?
Co-located placement of computer objects in AD Organizational Units (OUs)
Nodes
Created in default Computers container OU
CNO
Created in same OU as nodes computer objects
VCO’s
Created in same OU as CNO
Disabled Object for deleted role
Virtual Computer Object (VCO)
Cluster Name Object (CNO)

17. Clusters in Secure DNS Zones
Clusters support Active Directory integrated DNS zones with secure dynamic updates enabled
CNO is used as security principal to create the DNS records
If the CNO is ever deleted / recreated, it will have a different SID and will be unable to update the DNS record
Generates the following error to the System event log::
Cluster Service
DNS
CNO
DNS Record
Event ID: Level: Error Description: Cluster network name resource 'Resource Name' failed registration of one or more associated DNS name(s) for the following reason:
DNS operation refused.

18. Read-Only Domain Controller (RODC) Support
Enables perimeter network and Branch deployments
Read-Write DCs are inside the firewall
Supported with Win2012 and later
Pre-Win2012 requires access to a Read-Write Domain Controller
Computer Object password management is chained from RODC to RWDC
Required Steps:
Pre-stage computer objects (CNO and VCOs) on a RWDC
Grant appropriate permissions on the OUs
Set Cluster SPNs for the Computer Objects
Replicate Computer Objects from RWDC to the RODC
Reference

19. Value of Active Directory-detached Clusters
Simplifies cluster deployments
No need to contact AD admin to setup a cluster
Flexibility to create clusters with or without AD integration
Still required that Nodes are domain joined
Resiliency
Reduced dependencies and failure points
Active Directory
CNO
VCO
Cluster
Reference

20. Considerations Considerations when choosing model
No computer objects implies no Kerberos authentication to name
NTLM only client authentication against cluster names
Intra-cluster authentication still leverages Kerberos
Role
Position
Notes
MSMQ Clusters
Not Supported
MSMQ stores properties in AD
File Server Clusters
Not Recommended
Kerberos for SMB preferred
Hyper-V Clusters
No live migration support
SQL Server Clusters
Supported
Best fit if using SQL Authentication

21. Cluster Validation

22. Cluster Validation
Functional test tool built into the product that verifies the configuration
Run during testing, deployment or production
Best practices analyzed if run on configured cluster
Series of end-to-end tests on all cluster components
Configuration info for support and documentation
Networking issues
Troubleshoot in-production clusters

23. Cluster Validation Tests
Cluster Configuration
Existing clusters only
Hyper-V Configuration
If role is enabled
Inventory
Network
Storage
System Configuration

24. Validate a Failover Cluster

25. Cluster Creation

26. Failover Cluster Manager
Integrated cluster validation
Optionally add cluster disks

27. Create a Failover Cluster

28. Cluster Management

29. HA Roles and Features Common Server Roles Generic Containers
Hyper-V
Exchange
File Server
SQL
Generic Containers
Generic Application
Generic Script
Generic Service
Other Server Roles
DFS-Namespace
DFS-Replication
DHCP
DTC
Hyper-V Replica Broker
IIS
iSCSI Target
iSNS
MSMQ
NFS
Remote Desktop
WINS
3rd Party

30. Cluster Groups Network Name (NetName) IP Address
Easy name for clients to use when accessing clustered service / app
IP Address
Underlying IP address(es) for use with NetName
Uses DHCP if enabled in environment
Workload / Application
Feature role made “highly-available”
Physical Disk Resource
Shared storage for all nodes to access data

31. Virtual Machine Groups
Virtual Hard Disk
Traditional Disk
Cluster Shared Volume (CSV)
Virtual Machine Resource
Configuration File

32. Network Planning NIC Teaming on all adapters recommended
Most cluster have 4-6 NICs with 1 GB connections
Hyper-V clusters should consider NIC hardware enhancements like VMQ, SR-IOV and IPSec Task Offload
Networks uses
Heartbeats & cluster communication
Client / Application / VM access
VMs: Live migration, CSV, Deployment
Storage: SMB3, Shared VHDX, iSCSI, Fibre Channel over Ethernet (FCoE)
Backup

33. Cluster Patch Management
Cluster Aware Updating (CAU) is a feature included in Windows Server which enables patching orchestration
Coordinator serially updates all nodes
Windows Update Agent (WUA)
Windows Server Update Services (WSUS)
Windows Update
Workflow
Scan nodes to find which patches are needed
Identify node with fewest workloads
Move workloads or live migrate VMs
Call to WUA to patch
Verify patch is successful
Repeat steps 2 – 5 on next node
Repeat on remaining nodes
Update Coordinator
Windows Update

34. Create a Failover Cluster

35. Quorum

36. Quorum Witness Failure Node Shutdown for Patching
Cluster challenges
Site Power Outage
Network Disconnect
Node Crash 1 2 3 4 5
Quorum Witness Failure
Node Shutdown for Patching
Add/Evict Node
How do I make sure my Cluster stays up ??...

37. Why Quorum Faster Start & Recovery of Cluster
Effective quorum policy helps faster start of cluster
Determines the set of nodes that have latest cluster database
Identifying point when to start workload
Determines the point when cluster can host applications
Effective quorum policy prevents unnecessary downtime
Addressing split-brain
Prevent two disjointed instances of the same cluster

38. Voting Elements in Quorum
Nodes
Witness
Every cluster node has 1 vote
User configurable per node
Witness has 1 vote
Disk Witness
File Share Witness
User configurable
Single witness per cluster
Cluster needs majority of participating votes to survive

39. Quorum Types Majority is greater than 50%
Always have an odd number of votes
Node Majority
Node and File Share Majority
Disk Only (not recommended)
Node and Disk Majority
Vote
Vote
Vote
Vote
Vote

40. Disk Witness Considerations
Dedicated LUN for internal cluster use
Quorum Disk
Used as arbitration point
Stores a copy of cluster database
Recommendations:
Small disk at least 512 MB in size
Dedicated LUN
NTFS or ReFS formatted
No need for drive letter

41. Windows Server 2012 R2: Quorum Goals
Simplify Quorum Configuration
Quorum shouldn’t affect number of nodes in cluster
Simplified quorum witness selection
Updated wizard for quorum configuration
Increase Cluster High Availability
Cluster more resilient to node/witness failures
Cluster can now survive with <50% majority nodes with Dynamic Quorum
Cluster can now survive even split 50% nodes
Enable more disaster recovery quorum scenarios

42. File Share Witness Considerations
Simple Windows File Server
Easy to deploy
Single File Server can be used for multiple clusters
Unique File Share per clusters
CNO requires write permissions on the File Share
File Server Location
Recommended at 3rd separate site
Not on a node in the same cluster
Not inside VM running in the same cluster
HA File Server configured in a separate cluster
File Share Witness
No copy of cluster database Minimal network traffic – Cluster membership change only

43. Quorum Changes Removed legacy concepts of ‘quorum modes’ Node Majority
Node and Disk Majority
Node and File Share Witness Majority
Nodes: Define which nodes have a quorum vote
Configurable for 1 vote or 0 votes
Disk: Always configure a witness disk with WS 2012 R2
Clustering will determine when it is best to use it
Witness vote dynamically/automatically adjusted based on cluster membership with dynamic quorum
Odd node votes (3) + no witness vote (0) = 3
Even node votes (2) + witness vote (1) = 3
Node
1 / 0
Node
1 / 0
Node
1 / 0
Disk Witness
1 / 0

44. Deciding Which Witness to Use
Witness: Disk vs. File Share
Disk
File Share
Prevents Split-Brain P
Prevents Partition-in-Time
Solves Partition-in-Time
Arbitration Type
SCSI Persistent Reservation
Witness.log file on SMB Share
Recommended: Use Disk Witness if you have shared storage

45. Node Vote Weights Granular control of which nodes have votes
Directly affects quorum calculations
Limit impact on cluster quorum
Cluster quorum does not change if nodes with no vote go down
Nodes with No-Vote continue to be part of the cluster
Receive cluster database updates
Ability to host applications

46. Why modify Node Vote?
Not all nodes in your cluster are equally important
Typically nodes from Disaster Recovery Backup site
Primarily used for multi-site clusters
Recommended only for manual failover across sites
More about this in later slides …
Vote
Vote
Vote
No Vote
No Vote 1 2 3 4 5
Site A
Site B

47. Adjusting majority votes using Node Votes
Original: Total Votes = Majority Votes = 3
Updated: Total Votes = Majority Votes = 2
Vote
Vote
Vote
No Vote
Quorum Maintained!
Cluster Survives! 1 2 3 4

48. Adjusting Node Vote Weights
Granular control of which nodes have votes
Configurable per cluster node
Can be modified with no downtime
Use PowerShell or Configure Quorum Wizard
NodeWeight
Default = 1
Remove Vote = 0
Cluster Assigned = 1
(Get-ClusterNode <name>).NodeWeight = 0

49. Viewing Node Vote Weights
Updated Nodes Page For Easy Viewing
User configured node vote weights in “Assigned Vote” column
Cluster assigned dynamic vote weights in “Current Vote” column

50. Dynamic Quorum Automatic Node Vote Adjustment Dynamic Quorum Majority
Automatic adjustment of Node Vote based on node’ state
Active Node : Dynamic Vote = 1
Down Node : Dynamic Vote = 0
No change for node with no assigned vote
Dynamic Quorum Majority
Quorum majority is dynamically determined by active cluster nodes
Increase High Availability of Cluster Itself
Sustain sequential node failures or shutdowns
Enables cluster to survive with <50% active nodes

51. Dynamic Quorum Functionality
11/21/2018
Dynamic Quorum Functionality
Last Man Standing
Cluster can now survive with only 1 node
64-node cluster all the way down to 1 node
Seamless Integration
With existing cluster quorum features & configurations
With multisite disaster recovery deployments
Enabled By Default
Configurable via PowerShell
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

52. Dynamic Quorum for Witness
Automatic Witness Vote Adjustment
Automatic adjustment of Witness Vote based on active cluster membership
Even Active Nodes with Dynamic Vote of 1 : Witness Dynamic Vote = 1
Odd Active Nodes with Dynamic Vote of 1 : Witness Dynamic Vote = 0
Cluster now has the smarts to determine when to use Witness Vote!
State of Witness
Witness Offline or Failed will automatically make Witness Dynamic Vote = 0
New Recommendation
Always configure a witness with Windows Server 2012 R2
Clustering will determine when it is best to use the Witness
Configure Disk Witness if shared storage, otherwise FSW

53. User Configurable Quorum Properties
DynamicQuorum
NodeWeight
Cluster Common Prop
Default: Enabled
1: Enabled
0: Disabled
Node Common Prop
Default: Vote assigned
1: Cluster Managed
0: Disable Vote
PowerShell
(Get-Cluster).DynamicQuorum = 1
(Get-ClusterNode “name”).NodeWeight = 1

54. Cluster Managed Quorum Properties
DynamicWeight
WitnessDynamicWeight
Node Common Prop
Value Adjusted by Cluster
1: Node Has Vote
0: Node Has No Vote
Cluster Common Prop
Value Adjusted By Cluster
1: Witness Has Vote
0: Witness Has No Vote
PowerShell
(Get-ClusterNode “name”).DynamicWeight (read only)
(Get-Cluster).WitnessDynamicWeight (read only)

55. Dynamic Quorum : Node Scenarios
11/21/2018
Dynamic Quorum : Node Scenarios
Node Shutdown
Node removes its own vote
Node Crash
Remaining active nodes remove vote of the downed node
Node Join
On successful join the node gets its vote back
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

56. Dynamic Quorum : Witness Scenarios
11/21/2018
Dynamic Quorum : Witness Scenarios
Witness Offline
Witness vote gets removed by the cluster
Witness Failure
Witness vote gets removed by the cluster
Witness Online
If necessary, Witness vote is added back by the cluster
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

57. Tie Breaker Cluster will survive simultaneous loss of 50% votes
Especially useful in multi-site DR scenarios with even split
Cluster always ensures total number of votes are Odd
One site automatically elected to win
By default, cluster randomly selects a node to take its vote out
LowerQuorumPriorityNodeID cluster common property identifies a node to take its vote out
Cluster
Site1
Site2

58. Dynamic Quorum Considerations
Simultaneous Loss of Majority Nodes
Need existing majority votes to update new majority votes
Custer cannot sustain simultaneous loss of majority nodes
Always Configure Witness
Witness helps cluster to sustain one extra node failure
Witness helps in giving equal opportunity to survive in DR scenarios (more details later)
Cluster running with <50% majority nodes
The remaining <50% nodes become more important
“Last Man Standing” node becomes necessary for cluster start
Helps prevent partition in time

59. Configured via Cluster Manager GUI and PowerShell
Cluster Quorum Wizard
Updated PowerShell
PowerShell
Set-ClusterQuorum –NoWitness
Set-ClusterQuorum –DiskWitness “DiskResourceName”
Set-ClusterQuorum –FileShareWitness “FileShareName”
Set-ClusterQuorum –DiskOnly “DiskResourceName”

60. Force Quorum Manual Override
Allows to start cluster without majority votes
Cluster starts in a special “forced quorum” mode
Remains in this mode till majority votes achieved
Cluster automatically switches to normal functioning
Caution
Always understand why quorum was lost
Split-brain between nodes possible
You are now in control!
Prevent Quorum Flag
Command Line:
net start clussvc /ForceQuorum
PowerShell:
Start-ClusterNode –ForceQuorum

61. Prevent Quorum Helps prevent nodes with vote to form cluster
Nodes started with ‘Prevent Quorum’ always join existing cluster
Applicable to cluster in “Force Quorum”
Always start remaining nodes with ‘Prevent Quorum’
Helps prevent overwriting of latest cluster database
Forward progress made by nodes in ‘Force Quorum’ is not lost
Prevent Quorum Flag
Command Line:
net start clussvc /PQ
PowerShell:
Start-ClusterNode –PreventQuorum
Most applicable in multisite DR setup

62. Force Quorum Resiliency
Cluster detects partitions after a manual Force Quorum
Cluster has the built-in logic to track Force Quorum started partition
Partition started with Force Quorum is deemed authoritative
Other partitions automatically restart up on detecting a FQ cluster
Restarted nodes in other partition join the FQ cluster
Cluster automatically restarts the nodes with Prevent Quorum
Cluster
Manual Override with ForceQuorum
Site1
Site2
Nodes Restarted
When Site2 partition detected

63. Quorum Modes

64. Cluster Upgrades

65. Clean Installation Only
In-Place Upgrades not supported
Must evict node and perform ‘clean installation’
No mixed OS clusters today
More Information: KB

66. Cluster Workload Migration
Use built-in Migration Wizard
Requires 2 clusters
Supports N to N-2 releases
Migrate by resource group
Workloads stay online on original cluster
Pre- and Post- Migration Reports
Win2008 R2
Win2012 R2
Win2012

67. VM Migration VMs can live migrate from 2012 to 2012 R2
TechEd 2013
11/21/2018 9:45 AM
VM Migration
VMs can live migrate from to 2012 R2
Requires 2 clusters
Need to upgrade ICs in VMs
May want to upgrade other clusters in the stack, such as Scale Out File Server
Standard Disks ↔ CSV Disks
Remap VM Networks
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

68. Migration is Flexible Test Migration Enterprise ↔ Datacenter
Core ↔ Full
Physical Cluster ↔ Guest (Virtual) Cluster
Static IP ↔ Dynamic IP
IPv4 ↔ IPv6
Different Subnets

69. Wizard does not Migrate Everything
Several Roles:
DFS-R
Replicate Configuration to new Cluster
Exchange
Move Mailbox & Public Folder Replication
Print
PrintBRM Tool (UI or CMD)
Remote Desktop CB
TBD
SQL
SQL Server Installation Wizard Upgrade
Groups
Available Storage
Cluster Group (Cluster Core Resources)
Storage Mount Points
No PowerShell/Automation Support
Data Migration

71. AD Cluster Resources
Failover Cluster Step-by-Step Guide: Configuring Accounts in Active Directory
How to Setup a Failover Cluster in a RODC Environment
How to Create a Cluster in a Restrictive Active Directory Environment
Identifying Stale Cluster Computer Objects
How to Troubleshoot Create Cluster failures in Windows Server 2012