5-1 JMH Associates © 2004, All rights reserved Windows Application Development Chapter 5 Threading Models for Reliability

5-2 JMH Associates © 2004, All rights reserved OBJECTIVESOBJECTIVES Upon completion of this Chapter, you will be able to:  Solve more complex threading problems  Use standard threading models in application design  Combine mutexes and events reliably and usefully  Build reliable, maintainable applications  Understand and avoid many common pitfalls

5-3 JMH Associates © 2004, All rights reserved ContentsContents 1.Multithreading Models 2.COM Model Comparison 3.Events and Mutexes Together 4.The Condition Variable Model 5.SignalObjectAndWait 6.Using the Condition Variable Model 7.Some Performance Tradeoffs 8. Avoiding Incorrect Code - Hints 10.Lab Exercise 5

5-4 JMH Associates © 2004, All rights reserved 1. Multithreading Models Advantages of using thread models:  Expedite design and development  Models are well understood and well tested  Avoid potential multithreading problems  Models match structure of most programming problems Very simple cases, plus three “classic” models  Boss/worker  Pipeline  Client/server Plus, combined models “Programming in the large”

5-5 JMH Associates © 2004, All rights reserved Boss/Worker Model ProgramResourcesInput (Stream) Files Databases Workers TaskN Task2 Task1 main () Boss... Computations

5-6 JMH Associates © 2004, All rights reserved Pipeline Model Q Q Transmitter Receiver Consumers Q Q M1M2...M5M1M2...M5 P1 Producers PN Message TO FROM DATA Log C1 CN

5-7 JMH Associates © 2004, All rights reserved Client/Server Model TLS Code Shared Data Constants Thread-Thread Communication & Synchronization Arg Thread 1 Specific Data Server Thread 1 Stack 1 Client 1 Arg Server Thread N Client N Thread N Specific Data Status Broadcast and Monitor TLS Stack Arg Statistics Stack

5-8 JMH Associates © 2004, All rights reserved CombinationsCombinations Large systems rarely a pure instance of a single model  Pipeline stages often implemented as boss/ worker systems  A server may perform client requests in a pipeline sequence  Always use good implementation documenting individual modules in terms of these models  Easier to maintain, understand and troubleshoot

5-9 JMH Associates © 2004, All rights reserved 2. COM Microsoft’s Component Object Model  Object-oriented technology  Single threaded object  Only one thread can access it  Apartment model threading  Unique thread assigned to each instance of an object  Free threaded object  A thread from a thread pool is assigned, or created, when a request is made

5-10 JMH Associates © 2004, All rights reserved COM – Useful Terminology Thread pool  Collection of available threads Symmetric thread model  A group of threads performs the same task using exactly the same thread functions Asymmetric thread model  Different for separate thread functions COM only  Don’t confuse with “Classical Models”

5-11 JMH Associates © 2004, All rights reserved 3. Events and Mutexes Together Example: Pipeline producer/consumer  Producer creates a checksummed message  A mutex protects the message  Producer signals the consumer that a message is ready  The producer waits on an event  The mutex defines the critical code section for accessing the message data structure object  Assures the object’s “invariant” properties  The event is used to signal that there is a new message  Signals that the object (message queue) is in a specified state

5-12 JMH Associates © 2004, All rights reserved Events and Mutexes Together  One thread (producer) locks the data structure  Changes the object’s state by creating a new message  Sets or pulses the event – new message  One or more threads (consumers) wait on the event for the object to reach the desired state  The wait must occur outside the critical code section  A consumer thread can also lock the mutex  And test the object’s state  The state will not changer while being tested

5-13 JMH Associates © 2004, All rights reserved 4. The Condition Variable Model “Programming in the small” Concepts are taken from Pthreads (Supplem’try Section) Several key elements:  Data structure of type STATE_TYPE  Contains all the data such as messages, checksums, etc.  A mutex and one or more associated events  The mutex protects the data structure  The events signal “interesting” data structure state changes  Boolean functions evaluate “condition variable predicates”  For example, “a new message is ready”  An event is associated with each condition variable predicate  The event is signaled with the cvp becomes true

5-14 JMH Associates © 2004, All rights reserved Condition Variable Model: Data Shared Data Structure: typedef struct _state_t { HANDLE Guard; /* Mutex to protect the object */ HANDLE cvpSet; /* Autoreset Event */... other condition variables /* State structure with counts, etc. */ struct STATE_VAR_TYPE StateVar; } STATE_TYPE State;... /* Initialize State, creating mutex & event(s) */...

5-15 JMH Associates © 2004, All rights reserved Condition Variable Model: Producer /* PRODUCER thread that modifies State */ WaitForSingleObject (State.Guard, INFINITE); /* Change state so that the CV predicate holds */... State.StateVar.xyz =... ; SetEvent (State.cvpSet); ReleaseMutex (State.Guard); /* End of the interesting part of the producer */...

5-16 JMH Associates © 2004, All rights reserved Condition Variable Model: Consumer /* CONSUMER thread waits for a particular state */ WaitForSingleObject (State.Guard, INFINITE); while (!cvp(&State)) { ReleaseMutex (State.Guard); WaitForSingleObject (State.CvpSet, TimeOut); WaitForSingleObject (State.Guard, INFINITE); }... ReleaseMutex (State.Guard); /* End of the interesting part of the consumer */

5-17 JMH Associates © 2004, All rights reserved Condition Variable Model Comments You need to repeat the loop and test  The timeout is finite – a tunable parameter CVM avoids missed signals and other problems This is the “broadcast” version  Multiple consumers released by one producer SetEvent()  Subsequent behavior depends on cvp() and produce/consume semantics  Uses an AR event and SetEvent() “Signal” version uses MR event and PulseEvent()  A single consumer is released

5-18 JMH Associates © 2004, All rights reserved 5. SignalObjectAndWait Three essential steps of the loop in the consumer  Unlock the mutex  Wait on the event  Lock the mutex again The first two are not atomic  The producer can signal before the consumer waits  Risk: A missed signal – hence the timeout SignalObjectAndWait() combines the first two steps  One atomic function  Pthreads combines all three steps  >= NT 4.0 (not on Win 9x, Me)

5-19 JMH Associates © 2004, All rights reserved SignalObjectAndWaitSignalObjectAndWait DWORD SignalObjectAndWait( HANDLE hObjectToSignal, // Mutex HANDLE hObjectToWaitOn, // Event DWORD dwMilliseconds, // time-out in ms BOOL bAlertable // Use FALSE – until S7; SOAW is general purpose  Just the mutex-event usage is shown Timeout can usually be infinite Be sure to: #define _WIN32_WINNT 0x400 // WINBASE.H

5-20 JMH Associates © 2004, All rights reserved 6. Using the CV Model  Producer locks the mutex, changes state, sets the event and unlocks the mutex  Event should be set with the mutex locked  Consumer tests the CV predicate with the mutex locked  If the predicate does not hold, the consumer must unlock the mutex before waiting on the event  Event wait must have a timeout – to avoid missed signals  Unless you use SOAW!  Consumer always retests the predicate after the event wait  Consumer always owns the mutex when it leaves the loop  Whether loop body was executed or not

5-21 JMH Associates © 2004, All rights reserved CV Model Variation In producer/consumer code, multiple threads released  Auto-reset event  SetEvent Or, there may be only one message available and multiple consuming threads  Event should be manual-reset  Producer should call PulseEvent  Assure exactly one thread is released

5-22 JMH Associates © 2004, All rights reserved When Interested in the Next Event /* CONSUMER thread waits for NEXT state change */ WaitForSingleObject (State.Guard, INFINITE); do { SignalObjectAndWait (State.Guard, State.cvpSet, TimeOut, FALSE); WaitForSingleObject (State.Guard, INFINITE); } while (!cvp(&State)); /* Thread now owns the mutex and cvp(&State) holds */ /* Take appropriate action, perhaps modifying State */... ReleaseMutex (State.Guard); /* End of the interesting part of the consumer */

5-23 JMH Associates © 2004, All rights reserved 7. Some Performance Tradeoffs

5-24 JMH Associates © 2004, All rights reserved 8. Avoiding Incorrect Code - Hints  Pay attention to design, implementation, and use of familiar programming models  Best debugging technique: Don’t create bugs in the first place  Many serious defects will elude the most extensive and expensive testing  Debuggers change timing behavior  Masking the race conditions you wish to expose Most of these hints are from Programming with POSIX Threads by David Butenhof, Addison-Wesley, 1997.

5-25 JMH Associates © 2004, All rights reserved Avoiding Incorrect Code - Hints  Avoid relying on “thread inertia”  Never bet on a thread race  Scheduling is not the same as synchronization  Sequence races can occur  Even when you use mutexes to protect shared data  Cooperate to avoid deadlocks  Never share events between predicates

5-26 JMH Associates © 2004, All rights reserved Avoiding Incorrect Code - Hints  Beware of sharing stacks and related memory corrupters  Beware: No warning of thread stack overflow!  Be sure to use the volatile storage modifier  Use the condition variable model properly  Understand your invariants and condition variable predicates  Keep it simple  Test on multiple systems (single and multiprocessor)  Testing is necessary but not sufficient  Be humble  Be prepared for unpleasant surprises

5-27 JMH Associates © 2004, All rights reserved Thread Blocking Notes At any point any thread may go to sleep for an unbounded period of time No ordering exists between threads unless you cause ordering Thread scheduling behavior can vary widely  Various operating system implementations  Different models of the same system  Systems with different processor speeds Explicit synchronization is required There is no timeout from a pthread_join() call

5-28 JMH Associates © 2004, All rights reserved Word Tearing A single mutex should be used in every critical section associated with a shared resource  a.k.a. “fighting over cache lines” - SMP  Example: A processor may access quadword memory units  64 bits, since a word is 16 bits  16-bit words are not separate resources  When different threads access distinct values that lie in the same quadword, we have word tearing  The entire quadword should be protected with one mutex Demonstration program included in exercise files  Ask the instructor for more information if interested!

5-29 JMH Associates © 2004, All rights reserved Word Tearing (2 of 2) Memory Bit Words Thread B Value Wins; Next Time; Thread A Might Win Thread AThread B Writes to First Byte Terminates Writes to Last Byte Terminates Read

5-30 JMH Associates © 2004, All rights reserved Deadlock Avoidance, Lock Hierarchies Avoid deadlocks by using a lock hierarchy whereby mutexes are always locked in the same sequence  Release locks in the reverse sequence Additional deadlock notes  During development and troubleshooting  Assure that multiple locks are always acquired in the same order  Document the lock hierarchy  Lock hierarchies are provably deadlock free

5-31 JMH Associates © 2004, All rights reserved Lock (&ListB.guard) No Deadlocks with Hierarchical Locks AddSharedElement Thread DeleteSharedElement Thread Running Ready OS SCHEDULER Lock (&ListA.guard) Preempt Ready Run Running Unock (&ListB.guard) Unlock (&ListA.guard) Preempt Ready Running Blocked Run Lock (&ListA.guard) Blocked Unblock Ready Run Lock (&ListB.guard) Unock (&ListB.guard) Unlock (&ListA.guard) Running

5-32 JMH Associates © 2004, All rights reserved 9. Lab Exercise 10-1 ThreeStage.c implements the illustrated pipeline model  See the Pipeline slide  It was used to generate the performance results  It is implemented with SignalObjectAndWait Fix QueueObjX.c, and/or Determine performance of different implementations on your system(s)