Task Parallel Library: Design Principles and Best Practices Tech Ed North America 2010 9/11/2018 11:44 PM Required Slide SESSION CODE: DEV408 Task Parallel Library: Design Principles and Best Practices Shy Cohen Principal Shy Cohen Consulting © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

… ABOUT ME Shy Cohen (Ex) MSFT WCF, Cloud Tech Ed North America 2010 9/11/2018 11:44 PM ABOUT ME Shy Cohen (Ex) MSFT WCF, Cloud 20 Years in Parallel and Distributed Computing … © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

WHY SHOULD I CARE ABOUT PARALLELISM? Computers are changing Moore’s Law is taking an interesting turn SMP and Multi Core Users are changing They expect applications to be snappy and responsive CPU CPU CPU Core Core Memory Memory

OK, I’M INTERESTED – WHAT SHOULD I DO ABOUT IT? Parallelize! Thread A Thread B Thread C Quantum Expired Memory Access I/O Operation Any “wait state”

USING THREADS TO PARALLELIZE OPERATIONS Tech Ed North America 2010 9/11/2018 11:44 PM USING THREADS TO PARALLELIZE OPERATIONS Using threads on a single core machine may expedite processing when entering a wait state Using threads on multi-core machine may expedite processing by doing things in parallel DEMO © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

THE PROBLEM WITH THREADS Threads are heavyweight Each thread is allocated 1 MB of VM for its stack – this doesn’t scale Context switching is expensive Optimizing threads usage is not trivial! Not so good for “quick” parallelism Threads are hard to compose Passing data to and from threads is not simple Managing errors that happen on other threads is not trivial Coordinating the work of multiple threads is not simple Especially when you need to stop/cancel work

OK, WHAT’S THE ALTERNATIVE? PLINQ A Task-oriented Parallel Library (TPL) Parallel-Safe Data Structures Tasks Task Factories Continuations Cancellation tokens Scheduling control

TPL GOALS / THEMES Making parallelism Simpler Implementing common scenarios Passing data in & out of parallel operations Handling errors in parallel operations Interrupting work in progress Making parallelism Efficient Work stealing and variable work-chunk sizes Auto-adjusting the thread-pool size to address varying loads Reduced overhead for parallel work Giving developers Control, if they want it Creation and continuation options Scheduling tasks outside of the thread-pool

WHAT IS A TASK? Representation of an asynchronous operation A lightweight parallelism construct A way to get some value (or error ) that will exist in the future A consistent set of abstractions to program against An easy way to utilize the thread pool for parallelism Efficient for “quick” parallelism Also efficient for long-running activities

TASKS & QUEUES Global Queue Local Queues Worker Thread #1 Worker Thread #n Task #4 Task #5 Application Thread Task #1 Task #2

TASK DESIGN POINTS Quick and fine-grained creation of tasks Action and Func tasks Use of closures to pass data in and out Waiting for tasks to complete Timeouts and cancellation of tasks Nesting of Tasks Scheduling control Long running operation do not starve other operations Controlling on what work-queue a task is scheduled on Structured parallelism through parent-child relationship

OTHER TPL GOODIES Support for large collections Thread-local state Overloads for Int32 and Int64 ranges in loop operations Thread-local state Overloads with support for thread-local state (aggregation patterns) Configuration options Control over multiple aspects of a loop’s execution (incl. thread limiting) Dynamic thread counts Accommodating workloads that change in complexity over time I.e. when some portions of work are more compute-bound than others

Using Tasks To Parallelize Operations Tech Ed North America 2010 9/11/2018 11:44 PM Using Tasks To Parallelize Operations Use Task.Factory.StartNew when there’s no need to separate the creation from scheduling If your task returns a value, don’t try to access it too early or you will block DEMO © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Patterns For Waiting & Continuations You can also explicitly wait for a task to complete (with an optional timeout) You can wait for more than one task to complete (all or any) Continuations allow you to access the task result when it’s ready A continuation is a Task, so you can chain them DEMO

Accessing In-Scope Variables Closures are the easiest way to pass data into background operations, but not without cost Beware of unintentional sharing! When creating tasks in a loop, create a local copy of the iteration variable inside the loop Declare variables closest to where you’re going to use them DEMO

DEMO Patterns for Loops Process the elements of an array or a collection in parallel (but only when it makes sense!) It’s OK to nest loops and run them in parallel Use ParallelLoopState to Break/Stop a loop Also use it to see whether you should voluntarily stop processing DEMO

Patterns For Exception Handling TPL correctly handles multiple exceptions for the built-in parallelism constructs It also makes is easy to handle multiple exceptions when implementing your own parallelism The AggregateException class enables visibility into the failure(s) that happened on other threads DEMO

Patterns for Cancellations Use a CancellationTokenSource to cancel tasks A pending task will be cancelled by the scheduler A running task needs to notice that it was asked to cancel (and throw a TaskCancelledException) DEMO

WHEN SHOULD I PARALLELIZE? Parallelism starts at the design stage Analyze your application to identify common patterns

COMMON PATTERNS Fork-Join Pipeline Speculative Processing Producer-Consumer Aggregation Map-Reduce …

Using TPL To Implement Speculative Processing Tech Ed North America 2010 9/11/2018 11:44 PM Using TPL To Implement Speculative Processing DEMO © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Required Slide Track PMs will supply the content for this slide, which will be inserted during the final scrub. Tech Ed North America 2010 9/11/2018 11:44 PM Track Resources Visual Studio – http://www.microsoft.com/visualstudio/en-us/ Soma’s Blog – http://blogs.msdn.com/b/somasegar/  MSDN Data Developer Center – http://msdn.com/data ADO.NET Team Blog – http://blogs.msdn.com/adonet WCF Data Services Team Blog – http://blogs.msdn.com/astoriateam EF Design Blog – http://blogs.msdn.com/efdesign © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Resources Learning Required Slide www.microsoft.com/teched Tech Ed North America 2010 9/11/2018 11:44 PM Required Slide Resources Learning Sessions On-Demand & Community Microsoft Certification & Training Resources www.microsoft.com/teched www.microsoft.com/learning Resources for IT Professionals Resources for Developers http://microsoft.com/technet http://microsoft.com/msdn © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Complete an evaluation on CommNet and enter to win! Tech Ed North America 2010 9/11/2018 11:44 PM Required Slide Complete an evaluation on CommNet and enter to win! © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Sign up for Tech·Ed 2011 and save $500 starting June 8 – June 31st http://northamerica.msteched.com/registration   You can also register at the North America 2011 kiosk located at registration Join us in Atlanta next year

Tech Ed North America 2010 9/11/2018 11:44 PM © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Required Slide Tech Ed North America 2010 9/11/2018 11:44 PM © 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.