Phil Pennington Microsoft WSV317

What will you look for? Overall Solution Scalability

Agenda Windows Server 2008 R2 New NUMA APIs New User-Mode Scheduling APIs New C++ Concurrency Runtime

Example NUMA Hardware Today A 256 Logical Processor System – HP SuperDome A 64 Logical Processor System - Unisys ES dual-core hyper-threaded “Montvale” 1.6 GHz Itanium2 32 dual-core hyper-threaded “Tulsa” 3.4 GHz Xeon

NUMA Hardware Tommorrow 2, 4, 8 Cores-per-Socket "Commodity" CPU Architectures Expect systems with logical processors PCIExpress*PCIExpress* Nehalem I/OHub I/OHub

NUMA Node Groups New with Win7 and R2 GROUPGROUP NUMA NODE SocketSocketSocketSocket Core LP

NUMA Node Groups Example: 2 Groups, 4 Nodes, 8 Sockets, 32 Cores, 4 LPs/Core = 128 LPs GroupGroup NUMA Node SocketSocket Core LP Core LP Core LP Core LP SocketSocket Core LP Core LP Core LP Core LP NUMA Node SocketSocket Core LP Core LP Core LP Core LP SocketSocket Core LP Core LP Core LP Core LP GroupGroup NUMA Node SocketSocket Core LP Core LP Core LP Core LP SocketSocket Core LP Core LP Core LP Core LP NUMA Node SocketSocket Core LP Core LP Core LP Core LP SocketSocket Core LP Core LP Core LP Core LP

Sample SQL Server Scaling 64P To 128P 1.7X 64P 128P 1.3X

P1P1 Cache 1 Mem A Node Interconnect Mem B Disk A P3P3 Cache 3 P4P4 Cache 4 Cache(s) (0) (3) (4) (1) (7) I/O InitiatorI/O InitiatorISR I/O Buffer Home DPC (2)(6) (5) P2P2 Cache 2 Disk B Locked out for I/O Initiation outLocked out for I/O Initiation Bad Case Disk Write Software and Hardware Locality NOT Optimal

P1P1 Cache 1 Mem A Node Interconnect Mem B Disk A P3P3 Cache 3 P4P4 Cache 4 Cache(s) (3) I/O Initiator ISRDPC (2) P2P2 Cache 2 Disk B ISR (2) Windows Server 2008 R2 Optimization for NUMA Topology

NUMA Aware Applications Non-Uniform Memory Architecture Minimize Contention, Maximize Locality Apps scaling beyond even 8-16 logical processors should be NUMA aware A process or thread can set a preferred NUMA node Use the Node Group scheme for Task or Process partitioning Performance-optimize within Node Groups

NUMA API's “Minimize Contention and Maximize Locality”

Agenda Windows Server 2008 R2 New NUMA APIs New User-Mode Scheduling APIs New C++ Concurrency Runtime

Cooperative Scheduling Conceptual Model Avoiding lock contention gives the best scaling Cooperative scheduling in user-mode avoids contention and context switches Core 2 Thread 3 Thread 3 Non-running threads Core 1 Thread 4 Thread 4 Thread 5 Thread 5 Thread 1 Thread 1 Thread 2 Thread 2 Thread 6 Thread 6 Core 2 Core 1 User Thread 2 User Thread 2 Kernel Thread 2 Kernel Thread 2 User Thread 1 User Thread 1 Kernel Thread 1 Kernel Thread 1 User Thread 3 User Thread 3 Kernel Thread 3 Kernel Thread 3 User Thread 4 User Thread 4 Kernel Thread 4 Kernel Thread 4 User Thread 5 User Thread 5 Kernel Thread 5 Kernel Thread 5 User Thread 6 User Thread 6 Kernel Thread 6 Kernel Thread 6

User Mode Scheduling (UMS) System Call Servicing User Kernel KT(P 1 ) KT(P 2 ) UT(P 1 ) UT(P 1 ) UT(P 2 ) Primary Threads Core 1Core 2 KT(1) KT(2) KT(3) KT(4) UT(1) UT(2) UT(3) UT(4) UMS KT (Backing threads) USched ready list Parked SYSCALL Migrate request to appropriate KT Running Blocked Wake primary to regain core UMS completion list Kernel User

User Mode Context Switch Benefit Lower context switch time means scheduling finer-grained items UMS-based yield: 370 cycles Signal-and-wait: 2600 cycles Direct impact synchronization-heavy fine-grained work speeds up Indirect impact finer grains means more workloads are candidates for parallelization

Getting the Processor Back Benefit The scheduler keeps control of the processor when work blocks in the kernel Direct impact More deterministic scheduling and better use of a thread’s quantum Indirect impact Better cache locality when algorithmic libraries take advantage of the determinism to manage available resources

Agenda Windows Server 2008 R2 New NUMA APIs New User-Mode Scheduling New C++ Concurrency Runtime

Visual Studio 2010 Tools, Programming Models, Runtimes Parallel Pattern library Resource manager Task scheduler Task Parallel library Task Parallel library PLINQ Managed library Native library Key: Threads/UMS Operating system Concurrency runtime Programming models Agents library Agents library Thread pool Task scheduler Resource manager Data structures Tools Parallel Debugger Parallel Debugger Profiler and concurrency analyzer Profiler and concurrency analyzer

UMS Threads ConcRT’s use of UMS is an enabler for: Finer-grained parallelism More deterministic behavior Better cache locality UMS allows ConcRT to boost performance in certain situations: Apps that have a lot of blocking

Task Parallelism, For Example Key Concepts Task A computation that may be internally decomposed into additional tasks task_handle Task group A collection of tasks that form a logical computation or sub-computation task_group

Task Scheduling Tasks are run by worker threads, which the scheduler controls Dead Zone WT 0 WT 1 WT 2 WT 3 Without UMS (signal-and-wait) With UMS (UMS yield) WT 0 WT 1 WT 2 WT 3

User-Mode Scheduling API's and the C++ Concurrency Runtime “Cooperative Thread-Scheduling”

Summary Call-to-action Consider how your solution will scale on NUMA systems Utilize the NUMA API’s to Maximize Node Locality Leverage UMS for custom user-mode thread scheduling Use the C++ Concurrency Runtime for most native Parallel Computing scenarios and gain benefits of NUMA/UMS implicitly

Resources MSDN Concurrency Dev-Center MSDN Channel9 MSDN Code Gallery MSDN Server Dev Center LP and NUMA API Support Dev-Team Blogs

Sessions On-Demand & Community Resources for IT Professionals Resources for Developers Microsoft Certification and Training Resources Microsoft Certification & Training Resources Resources Required Slide Speakers, TechEd 2009 is not producing a DVD. Please announce that attendees can access session recordings at TechEd Online. Required Slide Speakers, TechEd 2009 is not producing a DVD. Please announce that attendees can access session recordings at TechEd Online.

Related Content DTL203 "The Manycore Shift: Making Parallel Computing Mainstream" Monday 5/11, 2:45-4:00, Room 404, Stephen Toub DTL310 Parallel Computing with Native C++ in Microsoft Visual Studio 2010 Friday 5/15, 2:45-4:00, Room 515A, Josh Phillips DTL403 "Microsoft Visual C++ Library, Language, and IDE : Now and Next" Thursday 5/14, 4:30-5:45, Room 408A, Kate Gregory DTL06-INT "Task-Based Parallel Programming with the Microsoft.NET Framework 4" Thursday 5/14, 1:00-2:15, Blue Thr 2, Stephen Toub Required Slide Speakers, please list the Breakout Sessions, TLC Interactive Theaters and Labs that are related to your session. Required Slide Speakers, please list the Breakout Sessions, TLC Interactive Theaters and Labs that are related to your session.

Windows Server Resources Make sure you pick up your copy of Windows Server 2008 R2 RC from the Materials Distribution Counter Learn More about Windows Server 2008 R2: Technical Learning Center (Orange Section): Highlighting Windows Server 2008 and R2 technologies Over 15 booths and experts from Microsoft and our partners Over 15 booths and experts from Microsoft and our partners Required Slide Track PMs will supply the content for this slide, which will be inserted during the final scrub. Required Slide Track PMs will supply the content for this slide, which will be inserted during the final scrub.

Complete an evaluation on CommNet and enter to win! Required Slide

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. Required Slide