1. Dynamic Partitioning in Windows Longhorn Santosh Jodh Software Design Engineer Windows Kernel Platform Group santoshj @ microsoft.com Microsoft Corporation Mike Tricker Program Manager Windows Kernel Platform Group miketri @ microsoft.com Microsoft Corporation

2. Session Outline Introduction to Dynamic Partitioning (DP) Clarifying the terminology Reliability, Availability & Serviceability (RAS) Capacity on Demand (CoD) Resource Management (RM) Hot Add, Replace & Remove Goals and non-goals for DP on Windows codenamed “Longhorn” What we’re expecting others to do to support this

3. Session Goals Attendees should leave this session with a good understanding of the following: What Microsoft means by Dynamic Partitioning DP-related terminology and acronyms Microsoft’s goals and non-goals for DP in Windows Longhorn Knowledge of where to find resources for DP

4. An Introduction to Dynamic Partitioning A hardware partitionable server has the ability to create one or more isolated hardware partitions comprising processors, memory and I/O, each supporting a single Windows instance A dynamically partitionable server has the ability to add, replace or remove hardware within a partition without needing to reboot the OS instance within the partition Why is this interesting? Hardware partition support has been available on some large servers for a number of years Windows is supported on hardware partitionable systems today, but does not support dynamic hardware partitioning With the projected increase in processor performance Microsoft expects a number of these features to become available on mid-range systems Microsoft plans to add support for dynamically partitionable hardware in Windows Longhorn

5. Why You Should Care About DP Microsoft believes that the capabilities that have previously been limited to expensive high end systems are moving into the mainstream Together with the introduction of multi-core processors this will make relatively small and inexpensive systems as powerful and reliable as today’s high end systems This will push highly fault-tolerant enterprise-critical applications such as large databases and management information applications onto less expensive platforms Which means that a range of hardware that has not previously had to consider some of the issues with dynamic hardware will now need to In the same way that RAID 5 changed the way in which we considered disks in the 1990’s

6. What Do We Mean By a Partition? One scale-up application, e.g., Database OS Cell 1 Cell 2 Cell 3 Cell 4 OS Cell 5 Cell 6 SQL Resource Management SQLExchange Virtual Server VM1VM2VM3 Multiple applications running on one OS Multiple Virtual Machines running on one OS All running on a single system

7. Reliability, Availability and Serviceability Minimizing unplanned downtime due to failing hardware E.g. if a processor starts to show signs of failing (increasing number of corrected errors or thermal events) swap it with one that’s on standby without needing to reboot the computer (similar to a hot spare disk in RAID 5)

8. Capacity on Demand The ability to enable processors that are physically present in the computer but not enabled by default E.g. buy a system with 8 processors, only 4 of which are initially paid for, enabled and used by the OS, and then when the workload grows pay to enable 2 or 4 more

9. Resource Management Sharing resources between two or more partitions E.g. If the load on partition 1 is increasing whilst the workload on partition 2 is decreasing move processors and/or memory from partition 2 to partition 1 to better handle the increasing workload

10. More Terminology Socket A physical socket into which a processor and/or memory may be plugged mechanically Sockets may also be independently powered Partition Unit (PU) A collection of system resources that form the smallest building blocks that can be assigned to a partition E.g. processors, memory and I/O host bridges More than one PU may be required to boot a partition

11. Yet More Terminology Hot Add Adding a socket or cell to a running partition Hot Remove Removing a socket or cell from a running partition Hot Replace Replacing a socket or cell in a running partition with one that is already physically present in the system but offline before the operation is started Note that Hot Replace is NOT the same as Hot Remove followed by Hot Add

12. And Yet More Terminology Hot Swap Some vendors support a model that does not require the stand-by hardware to be physically present before the Replace operation is started, and thus IS equivalent to a Hot Remove followed by a Hot Add Hot Plug A term typically covering Hot Add and Hot Remove

13. Assumptions We’re Making About Hardware Future partitionable machines will contain PUs which comprise Processors and memory together Processors Memory I/O host bridges The ACPI tables in those systems will be updated to expose specific methods required to support changing the hardware configuration without needing to reboot The firmware will be able to assist the OS during Hot Add and Hot Replace operations

14. More Hardware Assumptions Systems will include a Service Processor (SP) or Baseboard Management Controller (BMC) PUs can be electrically isolated when not in use No hardware assigned to a specific PU can be shared with other partitions, ensuring that a single failure cannot affect more than one partition

15. Dynamic Hardware Partitioning Memory IO Bridge Service Processor 1. Partition Manager provides the UI for partition creation and management 2. Service Processor controls the inter processor and IO connections Partition Manager 3. Platforms partitionable to the socket level. Virtualization used for sub socket partitioning 4. Support for dynamic partitioning and socket replacement PCI Express Core Cache … Core Cache … Core Cache … Core Cache …... IO Bridge... IO Bridge... IO Bridge... Longhorn dynamic hardware partitioning features are focused on improving server RAS Future Hardware Partitionable Server

16. Goals For Windows Longhorn Support the Hot Add of: Processors Memory I/O host bridges Support the Hot Replace of: Processors Memory OS support only x64 and Itanium only – no 32-bit support will be provided Server SKUs only - for SKUs supporting 4 processors or more only

17. Non-Goals For Windows Longhorn Hot Remove Windows Longhorn will not support the Hot Remove of processors or memory However tools will be supplied to allow both device driver and application developers to validate that they behave correctly in the case of a Hot Remove operation for either processors or memory Partition Manager Today’s Partition Managers are proprietary to each major OEM’s platform, and Microsoft will not be providing equivalent functionality in Windows Longhorn Microsoft will work with the system vendors to enable Windows DP support via their partition management tools SP & BMC “drivers” SPs and BMCs are devices that can be accessed from Windows via a device driver Windows Longhorn will include an IPMI driver which can communicate with SPs and BMCs via a standard interface, but will not provide specific drivers for any vendor’s SP or BMC

18. Supporting Technologies Windows Hardware Error Architecture (WHEA) Error infrastructure designed to support (amongst other things) DP, especially Hot Replace operations Making hardware error information more easily available for management applications to analyze and make failure predictions Extends the Machine Check Architecture available with the Intel Itanium platform Multi-level rebalance Windows Longhorn offers more sophisticated and extensive rebalance operations when hardware is added or removed This is not specific to DP, but will be leveraged by DP to make these operations as efficient as possible PCI Express and specifically Advanced Error Reporting The PCI bus is unable to report many errors, and most end up as NMIs PCI Express introduces AER and supports error correction, which will be exposed by WHEA for error prediction by management applications

19. Status of the Various Components Hot Add of memory is already supported by Windows Server 2003 x86 support shipped in Windows Server 2003 RTM x64 & Itanium support was added in Windows Server 2003 Service Pack 1 Hot Add of I/O Various device classes supporting Hot Plug are already available With Windows Longhorn the extended support for PCI Express devices makes this a very compelling feature Hot Add Processor support is now in test On x64 and Itanium Hot Replace for processors and memory is under development

20. What DP Implies to an Application Developer Add: applications can register for plug & play notifications of new hardware arriving Application developers with hard dependencies on memory or number of threads should watch for these notifications and update their behavior accordingly Resource management software, such as Microsoft’s WSRM, can abstract these changes such that the majority of applications do not need to explicitly handle these notifications Replace: applications will be unaffected and will see no change in the system Application developers need do nothing Remove: applications cannot make hard assumptions about memory or thread affinity Application developers cannot make assumptions about memory being fixed that they may do today, and should not rely upon thread affinity or the size of thread pools being static

21. What DP Implies to a Driver Developer Add: drivers can register for plug & play notifications of new hardware arriving Driver developers have fewer memory size limitations than application developers, and pool sizes will not change even if overall memory grows. The addition of processors and the related interrupt routing changes should also be invisible to drivers So in the Add case most drivers will not do anything new Replace: drivers will be unaffected and will see no change in the system There are implications around device timeouts, as it will be necessary to quiesce the system whilst the replace operation completes Remove: drivers cannot make hard assumptions about memory or thread affinity Drivers cannot make any assumptions around thread affinity, or even that the affinity mask will remain contiguous as it is today

22. Logo Requirements and Testing NOTE: DP is a Server-only feature, so there are no new Client requirements arising from this feature A number of new requirements are being proposed for the Microsoft logo program for Server to support DP Most apply to either platform firmware or device drivers Specific ACPI method support Device drivers must not assume that the processor affinity mask is contiguous We will also be providing test tools to ensure that you’re ready for Hot Remove support in a subsequent Windows release These will apply to both applications and device drivers

23. Other Implications of DP What about NUMA? What happens to the System Resource Affinity Table (SRAT) or System Locality Distance Information Table (SLIT) when new hardware gets added? Nothing happens to the SRAT as it’s a static table updated (and read by Windows) only at boot time So it will be updated the first time the system reboots after hardware is added For Windows Longhorn we’re not making use of the SLIT nor supporting the _SLI method to update locality information dynamically, so again nothing needs to be done here

24. Summary Windows Longhorn is planned to contain support for: Hot Add of processors, memory and I/O host bridges Hot Replace of processors and memory Windows Longhorn will not contain support for: Hot Remove of memory and processors An in-box Partition Manager There are things you’ll need to do to: Enable DP on your systems If your application is hardware-aware you may make use of the benefits offered by DP, and to not fail when hardware changes underneath you Ensure that your device drivers work correctly on DP-capable systems

25. Call to Action Application developers can benefit from DP if they make their application DP-aware Driver developers need to make their drivers DP- aware to work well on DP-capable systems Any may fail completely if they are badly behaved when hardware changes beneath them You may already be talking to us if you’re interested in DP If you’re interested and aren’t yet talking to us then please do!

26. Community Resources Windows Hardware & Driver Central (WHDC) www.microsoft.com/whdc/default.mspx Technical Communities www.microsoft.com/communities/products/default.mspx Non-Microsoft Community Sites www.microsoft.com/communities/related/default.mspx Microsoft Public Newsgroups www.microsoft.com/communities/newsgroups Technical Chats and Webcasts www.microsoft.com/communities/chats/default.mspx www.microsoft.com/webcasts Microsoft Blogs www.microsoft.com/communities/blogs

27. Additional Resources Email: dpfb @ microsoft.com Related Sessions Windows Hardware Error Architecture Error Management Solutions Synergy with WHEA

29. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.