Pedro Miguel Teixeira Senior Software Developer Microsoft Corporation 1/15/2019 6:44 AM SVR18 Developing Applications for Scale-Up Servers Running Windows Server 2008 R2 Pedro Miguel Teixeira Senior Software Developer Microsoft Corporation © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Developing Applications for Scale-Up 1/15/2019 6:44 AM Agenda Developing Applications for Scale-Up Processes and Threads in systems with more than 64 Logical Processors* Processor Groups and CPU distribution Processes and Threads System Topology New and revised APIs User Mode Scheduling Writing a User-Mode Scheduler User Mode Scheduler API © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Processor Groups for Scale Up 1/15/2019 6:44 AM Processor Groups for Scale Up Segmented specification – “groups” of CPUs Windows 7/R2 Supports more than 64 Logical Processors CPUs identified in software by Group#: CPU# Allows backward compatibility with 64-bit affinity New applications have full CPU range using new APIs © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Processor Group Assignment Maximum 64 LPs per Group Minimum possible that fits all LPs Currently a Maximum of 4 LPs distributed by proximity: All LPs (e.g., hyper-threads) in a Core; All Cores in a Socket; Closer Sockets*; NUMA Nodes**; Closer NUMA Nodes;

Processor Groups Example: 4 LPs/core, 4 cores/socket, 2 sockets/node, 4 nodes: 128 LPs Group NUMA Node Socket Core LP Group NUMA Node Socket Core LP

Processes and Threads Each Thread can only be in one Group Threads can be assigned to Groups: CreateRemoteThreadEx SetThreadGroupAffinity Processes are assigned to Groups in Round Robin* By default: Threads inherit the Group from the Parent Thread First thread inherits the Group from its Process

System Topology Processor topology exposed Memory topology exposed Groups  NUMA nodes  Sockets  Cores  Logical Processors (e.g., hyper-threads) Includes details of CPU caches (L0/L1/L2) Memory topology exposed Device location exposed (e.g., network and storage controllers)

New APIs GetActiveProcessorCount GetActiveProcessorGroupCount GetMaximumProcessorCount GetMaximumProcessorGroupCount GetProcessGroupAffinity GetProcessorSystemCycleTime GetThreadGroupAffinity SetThreadGroupAffinity GetNumaNodeNumberFromHandle

Revised APIs CreateRemoteThreadEx GetCurrentProcessorNumberEx GetLogicalProcessorInformationEx GetNumaAvailableMemoryNodeEx GetNumaProcessorNodeEx GetNumaProximityNodeEx GetNumaNodeProcessorMaskEx GetThreadIdealProcessorEx SetThreadIdealProcessorEx QueryIdleProcessorCycleTimeEx

User Mode Scheduling (UMS) Provide a comprehensive infrastructure for Applications to implement their own thread scheduling Support both Kernel and User synchronization events* Maximize CPU time spend executing application’s work Complete Win32 compatibility* UMS solves thread state problem by separating user-mode thread and kernel-mode thread

Cooperative Scheduling Avoiding lock contention gives the best scaling Cooperative scheduling in user-mode avoids contention and context switches Non-running threads User Thread 3 User Thread 4 User Thread 5 User Thread 6 Core 1 Core 1 Core 2 Core 2 User Thread 1 Thread 1 User Thread 2 Thread 2 Thread 3 Thread 4 Thread 5 Thread 6 Kernel Thread 1 Kernel Thread 2 Kernel Thread 3 Kernel Thread 4 Kernel Thread 5 Kernel Thread 6 Thread Scheduling

User Mode Scheduling Reason: Yield Reason: Yield Reason: Blocked Scheduler’s Ready List Wait Reason: Yield Reason: Yield Reason: Blocked Reason: Created Core 1 Core 2 UMS completion list W1 W2 W3 W4 S1 S2 User Kernel Scheduler Threads Blocked Worker Threads

UMS API CreateUmsCompletionList DeleteUmsCompletionList GetUmsCompletionListEvent DequeueUmsCompletionListItems GetNextUmsListItem CreateUmsThreadContext DeleteUmsThreadContext EnterUmsSchedulingMode ExecuteUmsThread UmsThreadYield GetCurrentUmsThread QueryUmsThreadInformation SetUmsThreadInformation

Scheduler writing No locks should be shared between Scheduler Threads and Worker Threads Adds complexity to writing user-mode synchronization objects/primitives Includes not only application locks but also OS implicit (and maybe internal) locks, reducing the operations that can be performed within the scheduler* Schedulers may need to create auxiliary worker threads for housekeeping tasks

YOUR FEEDBACK IS IMPORTANT TO US! Please fill out session evaluation forms online at MicrosoftPDC.com

channel9.msdn.com/learn 1/15/2019 6:44 AM Learn More On Channel 9 Expand your PDC experience through Channel 9 Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses channel9.msdn.com/learn Built by Developers for Developers…. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

1/15/2019 6:44 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

1/15/2019 6:44 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.