Lighting Up Windows Server 2008 R2 Using the ConcRT on UMS 7/21/2018 3:18 AM SVR10 Lighting Up Windows Server 2008 R2 Using the ConcRT on UMS Dana Groff Senior Program Manager Microsoft Corporation © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Agenda Agenda Key Take Aways Architecture Overview Defining and Controlling Execution Work Stealing Schedulers Policies Demo multiple schedulers + Work Stealing UMS Composition Support for > 64 Cores You can control affinity without paying attention to the hardware architecture Cooperative blocking allows my application to accomplish more work Composition of CPU resources helps my application run faster

Caveats… This is a C++ talk only! 300 Level talk focusing on the Concurrency Runtime’s Tasks, Schedulers, and resource managment We will start simply but quickly go deeper The focus of this talk is NOT the programming model Prerequisites Familiarity with multithreaded code It’s a really good idea If you have a basic knowledge of ConcRT

Parallel Pattern Library Concurrency for C++ Tools Native Libraries Visual Studio 2010 Parallel Pattern Library Asynchronous Agents Library Intel® OpenMP Intel ® TBB Parallel Debugger Profiler Concurrency Analysis Data Structures Intel Parallel Studio Parallel Composer Task Scheduler Parallel Inspector Resource Manager Parallel Amplifier Native Concurrency Runtime Operating System UMS Threads Threads Intel ® Parallel Studio Visual Studio 2010 / CLR Windows 7 Key:

Performance Productivity Platform Enable software to scale without the need to code explicitly to the hardware Improve runtime efficiency and scalability Performance Make it easier to express concurrency Make it more natural to design for concurrency Focus on your application, not the hardware Productivity Provide dynamic resource management, user-mode load-balancing scheduling with cooperative blocking Code once and run on Windows XP and later Enables compositional resource management Platform

Image Processing Scalable Performance using ConcRT Demo… Processing RAW Images

Express Concurrency User Responsibility Runtime Responsibility Decompose your algorithm into tasks Group related tasks together into groups Schedule related groups together Runtime Responsibility Cooperatively handle blocking operations Related work should stay local to the socket Manage CPU resources

Modular Decomposition into Tasks Decompose your algorithms into “tasks” A Task Pointer to a function Function Object Lambda Lightweight Tasks Fire and forget Group with ScheduleGroup Task_Handle Manage the lifetime of the task Schedule the task in one or more schedulers Group into task groups Cancellable

Making Tasks Run Associate a task with a scheduler Create and tune if Schedulers run tasks Work-stealing queues keep all the CPUs busy Create and tune if Schedulers run tasks efficiently together through schedule groups and task groups Multiple schedulers allow you to segment your CPU through policies You can tune the scheduler through policies

Tasks Are Run by Schedulers Work Stealing Queues Dynamic Resource Manager Scheduler 1 Scheduler 2 Thread 0 Thread 2 Thread 3 Thread 1 Thread 4 Thread 6 Thread 7 Thread 5 Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task Task

Group Tasks for Affinity ScheduleGroup A grouping mechanism for lightweight tasks (such as agents and messages) When a scheduler looks for work, it prefers tasks from the same ScheduleGroup Thread safe Task_Group Has all the features of a schedule group You can wait on all tasks in the group together You cancel all the tasks together Thread safe and reusable Structured_Task_Group Lightweight Task_Group that is not thread safe and not reusable

Dynamic Resource Management Core 1 Core 2 Core 3 Core 4 Sched1: Min 1, Max 4 Active Tasks Sched2: Min 1, Max 4 Active Tasks Sched3: Min 2, Max 4, High Pri: Active Tasks Sched1: Min 1, Max 4 Active Tasks Sched1: Min 1, Max 4 Idle Sched1: Min 1, Max 4 Active Tasks Sched3: Min 2, Max 4, High Pri: Active Tasks Sched1: Min 1, Max 4 Active Tasks Sched2: Min 1, Max 4 Stolen Tasks Sched2: Min 1, Max 4 Active Tasks

Scheduler Policies MinConcurrency, MaxConcurrency Min, Max number of threads associated with the scheduler (note this does not include threads created when the task is cooperatively blocked) TargetOversubscriptionFactor Suggested amount of oversubscription due to other schedulers ContextPriority Threads inherit the priority of the creating thread SchedulerKind UmsThreadDefault or ThreadScheduler SchedulingProtocol EnhanceForwardProgress, EnhanceScheduleGroupLocality ContextStackSize The task’s (thread context) stack size LocalContextCacheSize The size of the internal LIFO queue of unblocked contexts

Demonstrates work stealing and multiple schedulers segment CPU usage Ray Tracer Demo… Demonstrates work stealing and multiple schedulers segment CPU usage

Cooperative Blocking Use runtime awareness of blocking events to Schedule or reschedule work Oversubscribe Parallel Pattern Library wait() for a task_group: wait() for an event critical_section and reader_writer_lock Agent Library receive agent::wait, agent::wait_for_* Concurrency Runtime Concurrency::wait(N) Context::Block and Context::Unblock Context::Oversubscribe

Improving the Platform User Mode Scheduled (UMS) Threads What they are: Enable User Mode Scheduling of Threads Preserve TLS Provide runtime awareness of kernel blocking X64 platforms only ConcRT’s use of UMS is an enabler for: Finer-grained parallelism More deterministic behavior Better cache locality UMS allows the application to define scheduling behavior instead of relying on the OS to handle blocked threads

User-Mode Scheduling (UMS) kernel proper KT0 blocks KT0 KT1 KT2 Primary Thread trap code Thread Parking kernel user UT Completion list UT0 User-mode Scheduler Only primary thread runs in user-mode Trap code switches to parked KT KT blocks  primary returns to user-mode KT unblocks & parks  queue UT completion UT0 UT1 Microsoft Confidential

Demonstrating ConcRT cooperative blocking and UMS

Support More Than >64 Processors Windows 7 x64 and Windows Server R2 support more than 64 processors To set affinity for threads on these machines you need to use new APIs found in the Win7 SDK; see: SetThreadIdealProcessorEx You do not need to do anything special with ConcRT Schedulers, Task_Groups and ScheduleGroups affinitize over all nodes without code changes

Parallel Computing and PDC09 Tools Managed Languages Overviews: FT07 & WKSP08 Visual F# FT20 VTL02 Axum FT03 FT19 Visual Studio 2010 Parallel Debugger Windows Managed Libraries Native Libraries DryadLINQ SVR17 FT19 Parallel Pattern Library Async Agents Library Profiler Concurrency Analysis FT21 Parallel LINQ VTL04 Rx Task Parallel Library FT03 VTL32 Microsoft Research Data Structures Data Structures Native Concurrency Runtime Race Detection SVR10 Task Scheduler Managed Concurrency Runtime Fuzzing ThreadPool Resource Manager Operating System SVR01 HPC Server Threads SVR18 UMS Threads Operating System Key: Research / Incubation Visual Studio 2010 / .NET 4 Windows 7 / Server 2008 R2

Parallel Computing and PDC09 Overview FT07: The State of Parallel Programming WKSP08: Patterns of Parallel Programming Managed code in Visual Studio 2010 FT03: Manycore and .NET 4: A Match Made in Visual Studio 2010 FT21: PLINQ: LINQ, but Faster! FT20: F# for Parallel and Asynchronous Programming Native code in Visual Studio 2010 SVR18: Developing Applications for Scale-Up Servers Running Windows Server 2008 R2 SVR10: Lighting up Windows Server 2008 R2 Using the ConcRT on UMS FT19: C++ Forever: Interactive Applications in the Age of Manycore HPC Server SVR01: Accelerating Applications Using Windows HPC Server 2008 Research and Incubation VTL02: Axum: A .NET Language for Safe and Scalable Concurrency VTL32: Concurrency Fuzzing & Data Races SVR17: Data-Intensive Computing on Windows HPC Server with DryadLINQ VTL04: Rx: Reactive Extensions for .NET FT36: Future of Garbage Collection

YOUR FEEDBACK IS IMPORTANT TO US! Please fill out session evaluation forms online at MicrosoftPDC.com

channel9.msdn.com/learn 7/21/2018 3:18 AM Learn More On Channel 9 Expand your PDC experience through Channel 9. Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses. channel9.msdn.com/learn Built by Developers for Developers…. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

7/21/2018 3:18 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

7/21/2018 3:18 AM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.