1. Task Parallel Library: Design Principles and Best Practices
Tech Ed North America 2010
9/11/ :44 PM
Required Slide
SESSION CODE: DEV408
Task Parallel Library: Design Principles and Best Practices
Shy Cohen
Principal
Shy Cohen Consulting
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. … ABOUT ME Shy Cohen (Ex) MSFT WCF, Cloud
Tech Ed North America 2010
9/11/ :44 PM
ABOUT ME
Shy Cohen
(Ex) MSFT
WCF, Cloud
20 Years in Parallel and Distributed Computing …
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. WHY SHOULD I CARE ABOUT PARALLELISM?
Computers are changing
Moore’s Law is taking an interesting turn
SMP and Multi Core
Users are changing
They expect applications to be snappy and responsive
CPU
CPU
CPU
Core
Core
Memory
Memory

4. OK, I’M INTERESTED – WHAT SHOULD I DO ABOUT IT?
Parallelize!
Thread A
Thread B
Thread C
Quantum Expired
Memory Access
I/O Operation
Any “wait state”

5. USING THREADS TO PARALLELIZE OPERATIONS
Tech Ed North America 2010
9/11/ :44 PM
USING THREADS TO PARALLELIZE OPERATIONS
Using threads on a single core machine may expedite processing when entering a wait state
Using threads on multi-core machine may expedite processing by doing things in parallel
DEMO
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6. THE PROBLEM WITH THREADS
Threads are heavyweight
Each thread is allocated 1 MB of VM for its stack – this doesn’t scale
Context switching is expensive
Optimizing threads usage is not trivial!
Not so good for “quick” parallelism
Threads are hard to compose
Passing data to and from threads is not simple
Managing errors that happen on other threads is not trivial
Coordinating the work of multiple threads is not simple
Especially when you need to stop/cancel work

7. OK, WHAT’S THE ALTERNATIVE?
PLINQ
A Task-oriented Parallel Library (TPL)
Parallel-Safe Data Structures
Tasks
Task Factories
Continuations
Cancellation tokens
Scheduling control

8. TPL GOALS / THEMES Making parallelism Simpler
Implementing common scenarios
Passing data in & out of parallel operations
Handling errors in parallel operations
Interrupting work in progress
Making parallelism Efficient
Work stealing and variable work-chunk sizes
Auto-adjusting the thread-pool size to address varying loads
Reduced overhead for parallel work
Giving developers Control, if they want it
Creation and continuation options
Scheduling tasks outside of the thread-pool

9. WHAT IS A TASK? Representation of an asynchronous operation
A lightweight parallelism construct
A way to get some value (or error ) that will exist in the future
A consistent set of abstractions to program against
An easy way to utilize the thread pool for parallelism
Efficient for “quick” parallelism
Also efficient for long-running activities

10. TASKS & QUEUES Global Queue Local Queues Worker Thread #1
Worker Thread #n
Task #4
Task #5
Application Thread
Task #1
Task #2

11. TASK DESIGN POINTS Quick and fine-grained creation of tasks
Action and Func tasks
Use of closures to pass data in and out
Waiting for tasks to complete
Timeouts and cancellation of tasks
Nesting of Tasks
Scheduling control
Long running operation do not starve other operations
Controlling on what work-queue a task is scheduled on
Structured parallelism through parent-child relationship

12. OTHER TPL GOODIES Support for large collections Thread-local state
Overloads for Int32 and Int64 ranges in loop operations
Thread-local state
Overloads with support for thread-local state (aggregation patterns)
Configuration options
Control over multiple aspects of a loop’s execution (incl. thread limiting)
Dynamic thread counts
Accommodating workloads that change in complexity over time
I.e. when some portions of work are more compute-bound than others

13. Using Tasks To Parallelize Operations
Tech Ed North America 2010
9/11/ :44 PM
Using Tasks To Parallelize Operations
Use Task.Factory.StartNew when there’s no need to separate the creation from scheduling
If your task returns a value, don’t try to access it too early or you will block
DEMO
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14. Patterns For Waiting & Continuations
You can also explicitly wait for a task to complete (with an optional timeout)
You can wait for more than one task to complete (all or any)
Continuations allow you to access the task result when it’s ready
A continuation is a Task, so you can chain them
DEMO

15. Accessing In-Scope Variables
Closures are the easiest way to pass data into background operations, but not without cost
Beware of unintentional sharing!
When creating tasks in a loop, create a local copy of the iteration variable inside the loop
Declare variables closest to where you’re going to use them
DEMO

16. DEMO Patterns for Loops
Process the elements of an array or a collection in parallel (but only when it makes sense!)
It’s OK to nest loops and run them in parallel
Use ParallelLoopState to Break/Stop a loop
Also use it to see whether you should voluntarily stop processing
DEMO

17. Patterns For Exception Handling
TPL correctly handles multiple exceptions for the built-in parallelism constructs
It also makes is easy to handle multiple exceptions when implementing your own parallelism
The AggregateException class enables visibility into the failure(s) that happened on other threads
DEMO

18. Patterns for Cancellations
Use a CancellationTokenSource to cancel tasks
A pending task will be cancelled by the scheduler
A running task needs to notice that it was asked to cancel (and throw a TaskCancelledException)
DEMO

19. WHEN SHOULD I PARALLELIZE?
Parallelism starts at the design stage
Analyze your application to identify common patterns

20. COMMON PATTERNS Fork-Join Pipeline Speculative Processing
Producer-Consumer
Aggregation
Map-Reduce …

21. Using TPL To Implement Speculative Processing
Tech Ed North America 2010
9/11/ :44 PM
Using TPL To Implement Speculative Processing
DEMO
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. Required Slide
Track PMs will supply the content for this slide, which will be inserted during the final scrub.
Tech Ed North America 2010
9/11/ :44 PM
Track Resources
Visual Studio –
Soma’s Blog –
MSDN Data Developer Center –
ADO.NET Team Blog –
WCF Data Services Team Blog –
EF Design Blog –
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. Resources Learning Required Slide www.microsoft.com/teched
Tech Ed North America 2010
9/11/ :44 PM
Required Slide
Resources
Learning
Sessions On-Demand & Community
Microsoft Certification & Training Resources
Resources for IT Professionals
Resources for Developers
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

24. Complete an evaluation on CommNet and enter to win!
Tech Ed North America 2010
9/11/ :44 PM
Required Slide
Complete an evaluation on CommNet and enter to win!
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25. Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st
You can also register at the North America 2011 kiosk located at registration Join us in Atlanta next year

26. Tech Ed North America 2010 9/11/2018 11:44 PM
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

27. Required Slide Tech Ed North America 2010 9/11/2018 11:44 PM
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.