1. Lighting Up Windows Server 2008 R2 Using the ConcRT on UMS
7/21/2018 3:18 AM
SVR10
Lighting Up Windows Server 2008 R2 Using the ConcRT on UMS
Dana Groff
Senior Program Manager
Microsoft Corporation
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Agenda Agenda Key Take Aways Architecture Overview
Defining and Controlling Execution
Work Stealing
Schedulers
Policies
Demo multiple schedulers + Work Stealing
UMS Composition
Support for > 64 Cores
You can control affinity without paying attention to the hardware architecture
Cooperative blocking allows my application to accomplish more work
Composition of CPU resources helps my application run faster

3. Caveats… This is a C++ talk only!
300 Level talk focusing on the Concurrency Runtime’s Tasks, Schedulers, and resource managment
We will start simply but quickly go deeper
The focus of this talk is NOT the programming model
Prerequisites
Familiarity with multithreaded code
It’s a really good idea
If you have a basic knowledge of ConcRT

4. Parallel Pattern Library
Concurrency for C++
Tools
Native Libraries
Visual Studio 2010
Parallel Pattern Library
Asynchronous
Agents Library
Intel® OpenMP
Intel ® TBB
Parallel
Debugger
Profiler Concurrency
Analysis
Data Structures
Intel Parallel Studio
Parallel Composer
Task Scheduler
Parallel Inspector
Resource Manager
Parallel Amplifier
Native Concurrency Runtime
Operating System
UMS Threads
Threads
Intel ® Parallel Studio
Visual Studio 2010 / CLR
Windows 7
Key:

5. Performance Productivity Platform
Enable software to scale without the need to code explicitly to the hardware
Improve runtime efficiency and scalability
Performance
Make it easier to express concurrency
Make it more natural to design for concurrency
Focus on your application, not the hardware
Productivity
Provide dynamic resource management, user-mode load-balancing scheduling with cooperative blocking
Code once and run on Windows XP and later
Enables compositional resource management
Platform

6. Image Processing Scalable Performance using ConcRT
Demo…
Processing RAW Images

7. Express Concurrency User Responsibility Runtime Responsibility
Decompose your algorithm into tasks
Group related tasks together into groups
Schedule related groups together
Runtime Responsibility
Cooperatively handle blocking operations
Related work should stay local to the socket
Manage CPU resources

8. Modular Decomposition into Tasks
Decompose your algorithms into “tasks”
A Task
Pointer to a function
Function Object
Lambda
Lightweight Tasks
Fire and forget
Group with ScheduleGroup
Task_Handle
Manage the lifetime of the task
Schedule the task in one or more schedulers
Group into task groups
Cancellable

9. Making Tasks Run Associate a task with a scheduler Create and tune if
Schedulers run tasks
Work-stealing queues keep all the CPUs busy
Create and tune if
Schedulers run tasks efficiently together through schedule groups and task groups
Multiple schedulers allow you to segment your CPU through policies
You can tune the scheduler through policies

10. Tasks Are Run by Schedulers Work Stealing Queues
Dynamic Resource Manager
Scheduler 1
Scheduler 2
Thread 0
Thread 2
Thread 3
Thread 1
Thread 4
Thread 6
Thread 7
Thread 5
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task
Task

11. Group Tasks for Affinity
ScheduleGroup
A grouping mechanism for lightweight tasks (such as agents and messages)
When a scheduler looks for work, it prefers tasks from the same ScheduleGroup
Thread safe
Task_Group
Has all the features of a schedule group
You can wait on all tasks in the group together
You cancel all the tasks together
Thread safe and reusable
Structured_Task_Group
Lightweight Task_Group that is not thread safe and not reusable

12. Dynamic Resource Management
Core 1
Core 2
Core 3
Core 4
Sched1: Min 1, Max 4 Active Tasks
Sched2: Min 1, Max 4
Active Tasks
Sched3: Min 2, Max 4, High Pri: Active Tasks
Sched1: Min 1, Max 4
Active Tasks
Sched1: Min 1, Max 4 Idle
Sched1: Min 1, Max 4 Active Tasks
Sched3: Min 2, Max 4, High Pri: Active Tasks
Sched1: Min 1, Max 4
Active Tasks
Sched2: Min 1, Max 4
Stolen Tasks
Sched2: Min 1, Max 4
Active Tasks

13. Scheduler Policies MinConcurrency, MaxConcurrency
Min, Max number of threads associated with the scheduler (note this does not include threads created when the task is cooperatively blocked)
TargetOversubscriptionFactor
Suggested amount of oversubscription due to other schedulers
ContextPriority
Threads inherit the priority of the creating thread
SchedulerKind
UmsThreadDefault or ThreadScheduler
SchedulingProtocol
EnhanceForwardProgress, EnhanceScheduleGroupLocality
ContextStackSize
The task’s (thread context) stack size
LocalContextCacheSize
The size of the internal LIFO queue of unblocked contexts

14. Demonstrates work stealing and multiple schedulers segment CPU usage
Ray Tracer
Demo…
Demonstrates work stealing and multiple schedulers segment CPU usage

15. Cooperative Blocking Use runtime awareness of blocking events to
Schedule or reschedule work
Oversubscribe
Parallel Pattern Library
wait() for a task_group:
wait() for an event
critical_section and reader_writer_lock
Agent Library
receive
agent::wait, agent::wait_for_*
Concurrency Runtime
Concurrency::wait(N)
Context::Block and Context::Unblock
Context::Oversubscribe

16. Improving the Platform User Mode Scheduled (UMS) Threads
What they are:
Enable User Mode Scheduling of Threads
Preserve TLS
Provide runtime awareness of kernel blocking
X64 platforms only
ConcRT’s use of UMS is an enabler for:
Finer-grained parallelism
More deterministic behavior
Better cache locality
UMS allows the application to define scheduling behavior instead of relying on the OS to handle blocked threads

17. User-Mode Scheduling (UMS)
kernel proper
KT0 blocks
KT0
KT1
KT2
Primary
Thread
trap code
Thread Parking
kernel
user
UT Completion list
UT0
User-mode
Scheduler
Only primary thread runs in user-mode
Trap code switches to parked KT
KT blocks  primary returns to user-mode
KT unblocks & parks  queue UT completion
UT0
UT1
Microsoft Confidential

18. Demonstrating ConcRT cooperative blocking and UMS

19. Support More Than >64 Processors
Windows 7 x64 and Windows Server R2 support more than 64 processors
To set affinity for threads on these machines you need to use new APIs found in the Win7 SDK; see: SetThreadIdealProcessorEx
You do not need to do anything special with ConcRT
Schedulers, Task_Groups and ScheduleGroups affinitize over all nodes without code changes

20. Parallel Computing and PDC09
Tools
Managed Languages
Overviews:
FT07 & WKSP08
Visual F#
FT20
VTL02
Axum
FT03
FT19
Visual Studio 2010
Parallel
Debugger Windows
Managed Libraries
Native Libraries
DryadLINQ
SVR17
FT19
Parallel Pattern Library
Async
Agents Library
Profiler Concurrency
Analysis
FT21
Parallel LINQ
VTL04 Rx
Task Parallel Library
FT03
VTL32
Microsoft
Research
Data Structures
Data Structures
Native Concurrency Runtime
Race Detection
SVR10
Task Scheduler
Managed Concurrency Runtime
Fuzzing
ThreadPool
Resource Manager
Operating System
SVR01
HPC Server
Threads
SVR18
UMS Threads
Operating System
Key:
Research / Incubation
Visual Studio 2010 / .NET 4
Windows 7 / Server 2008 R2

21. Parallel Computing and PDC09
Overview
FT07: The State of Parallel Programming
WKSP08: Patterns of Parallel Programming
Managed code in Visual Studio 2010
FT03: Manycore and .NET 4: A Match Made in Visual Studio 2010
FT21: PLINQ: LINQ, but Faster!
FT20: F# for Parallel and Asynchronous Programming
Native code in Visual Studio 2010
SVR18: Developing Applications for Scale-Up Servers Running Windows Server 2008 R2
SVR10: Lighting up Windows Server 2008 R2 Using the ConcRT on UMS
FT19: C++ Forever: Interactive Applications in the Age of Manycore
HPC Server
SVR01: Accelerating Applications Using Windows HPC Server 2008
Research and Incubation
VTL02: Axum: A .NET Language for Safe and Scalable Concurrency
VTL32: Concurrency Fuzzing & Data Races
SVR17: Data-Intensive Computing on Windows HPC Server with DryadLINQ
VTL04: Rx: Reactive Extensions for .NET
FT36: Future of Garbage Collection

22. YOUR FEEDBACK IS IMPORTANT TO US!
Please fill out session evaluation forms online at
MicrosoftPDC.com

23. channel9.msdn.com/learn
7/21/2018 3:18 AM
Learn More On Channel 9
Expand your PDC experience through Channel 9.
Explore videos, hands-on labs, sample code and demos through the new Channel 9 training courses.
channel9.msdn.com/learn
Built by Developers for Developers….
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

24. 7/21/2018 3:18 AM
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25. 7/21/2018 3:18 AM
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.