1. Multi-core Programming Programming with Windows Threads

2. 2 Basics of VTune™ Performance Analyzer Topics Explore Win32 Threading API functions – Create threads – Wait for threads to terminate – Synchronize shared access between threads

3. 3 Programming with Windows Threads Windows* HANDLE type Each Windows object is referenced by HANDLE type variables – Pointer to kernel objects – Thread, process, file, event, mutex, semaphore, etc. Object creation functions return HANDLE Object controlled through its handle – Don’t manipulate objects directly

4. 4 Programming with Windows Threads Windows* Thread Creation HANDLE CreateThread( LPSECURITY_ATTRIBUTES ThreadAttributes, DWORD StackSize, LPTHREAD_START_ROUTINE StartAddress, LPVOID Parameter, DWORD CreationFlags, LPDWORD ThreadId ); // Out

5. 5 Programming with Windows Threads LPTHREAD_START_ROUTINE CreateThread() expects pointer to global function – Returns DWORD – Calling convention WINAPI – Single LPVOID (void *) parameter Thread begins execution of function DWORD WINAPI MyThreadStart(LPVOID p);

6. 6 Programming with Windows Threads Using Explicit Threads Identify portions of code to thread Encapsulate code into function – If code is already a function, a driver function may need to be written to coordinate work of multiple threads Add CreateThread call to assign thread(s) to execute function

7. 7 Programming with Windows Threads Destroying Threads Frees OS resources – Clean-up if done with thread before program completes Process exit does this for you BOOL CloseHandle(HANDLE hObject);

8. 8 Programming with Windows Threads Example: Thread Creation #include DWORD WINAPI helloFunc(LPVOID arg ) { printf(“Hello Thread\n”); return 0; } main() { HANDLE hThread = CreateThread(NULL, 0, helloFunc, NULL, 0, NULL ); } What Happens?

9. 9 Programming with Windows Threads Example Explained Main thread is process When process goes, all threads go Need some method of waiting for a thread to finish

10. 10 Programming with Windows Threads #include BOOL threadDone = FALSE ; DWORD WINAPI helloFunc(LPVOID arg ) { printf(“Hello Thread\n”); threadDone = TRUE ; return 0; } main() { HANDLE hThread = CreateThread(NULL, 0, helloFunc, NULL, 0, NULL ); while (!threadDone); } Waiting for Windows* Thread Not a good idea! // wasted cycles!

11. Wait for one object (thread) DWORD WaitForSingleObject( HANDLE hHandle, DWORD dwMilliseconds ); Calling thread waits (blocks) until – Time expires Return code used to indicate this – Thread exits (handle is signaled) Use INFINITE to wait until thread termination Does not use CPU cycles 11 Programming with Windows Threads Waiting for a Thread

12. Wait for up to 64 objects (threads) DWORD WaitForMultipleObjects( DWORD nCount, CONST HANDLE *lpHandles, // array BOOL fWaitAll, // wait for one or all DWORD dwMilliseconds) Wait for all: fWaitAll==TRUE Wait for any: fWaitAll==FALSE – Return value is first array index found 12 Programming with Windows Threads Waiting for Many Threads

13. 13 Programming with Windows Threads Notes on WaitFor* Functions Handle as parameter Used for different types of objects Kernel objects have two states – Signaled – Non-signaled Behavior is defined by object referred to by handle – Thread: signaled means terminated

14. 14 Programming with Windows Threads Example: Multiple Threads #include const int numThreads = 4; DWORD WINAPI helloFunc(LPVOID arg ) { printf(“Hello Thread\n”); return 0; } main() { HANDLE hThread[numThreads]; for (int i = 0; i < numThreads; i++) hThread[i] = CreateThread(NULL, 0, helloFunc, NULL, 0, NULL ); WaitForMultipleObjects(numThreads, hThread, TRUE, INFINITE); }

15. Modify the previous example code to print out – Appropriate “Hello Thread” message – Unique thread number Use for-loop variable of CreateThread loop Sample output: Hello from Thread #0 Hello from Thread #1 Hello from Thread #2 Hello from Thread #3 15 Programming with Windows Threads Activity 1 - “HelloThreads”

16. 16 Programming with Windows Threads What’s wrong? What is printed for myNum? DWORD WINAPI threadFunc(LPVOID ) { DWORD WINAPI threadFunc(LPVOID pArg) { int* p = (int*)pArg; int* p = (int*)pArg; int myNum = *p; printf( “Thread number %d\n”, myNum);}... // from main(): for (int i = 0; i < numThreads; i++) { hThread[i] = hThread[i] = CreateThread(NULL, 0, threadFunc, &i, 0, NULL); CreateThread(NULL, 0, threadFunc, &i, 0, NULL);}

17. 17 Programming with Windows Threads Hello Threads Timeline TimemainThread 0Thread 1 T0T0 i = 0------- T1T1 create(&i)--- T2T2 i++ (i == 1)launch--- T3T3 create(&i)p = pArg--- T4T4 i++ (i == 2)myNum = *p myNum = 2 launch T5T5 waitprint(2)p = pArg T6T6 waitexitmyNum = *p myNum = 2

18. 18 Programming with Windows Threads Race Conditions Concurrent access of same variable by multiple threads – Read/Write conflict – Write/Write conflict Most common error in concurrent programs May not be apparent at all times

19. 19 Programming with Windows Threads How to Avoid Data Races Scope variables to be local to threads – Variables declared within threaded functions – Allocate on thread’s stack – TLS (Thread Local Storage) Control shared access with critical regions – Mutual exclusion and synchronization – Lock, semaphore, event, critical section, mutex…

20. 20 Programming with Windows Threads Solution – “Local” Storage DWORD WINAPI threadFunc(LPVOID ) DWORD WINAPI threadFunc(LPVOID pArg){ (int*)pArg) int myNum = *((int*)pArg); printf( “Thread number %d\n”, myNum);}... // from main(): for (int i = 0; i < numThreads; i++) { tNum[i] = i; tNum[i] = i; hThread[i] = hThread[i] = CreateThread(NULL, 0, threadFunc, &tNum[i], CreateThread(NULL, 0, threadFunc, &tNum[i], 0, NULL); 0, NULL);}

21. 21 Programming with Windows Threads Windows* Mutexes Kernel object reference by handle “Signaled” when available Operations: – CreateMutex(…) // create new – WaitForSingleObject // wait & lock – ReleaseMutex(…) // unlock Available between processes

22. 22 Programming with Windows Threads Windows* Critical Section Lightweight, intra-process only mutex Most useful and most used New type CRITICAL_SECTION cs; Create and destroy operations InitializeCriticalSection(&cs) DeleteCriticalSection(&cs);

23. 23 Programming with Windows Threads Windows* Critical Section CRITICAL_SECTION cs ; Attempt to enter protected code EnterCriticalSection(&cs) – Blocks if another thread is in critical section – Returns when no thread is in critical section Upon exit of critical section LeaveCriticalSection(&cs) – Must be from obtaining thread

24. 24 Programming with Windows Threads Example: Critical Section #define NUMTHREADS 4 CRITICAL_SECTION g_cs; // why does this have to be global? int g_sum = 0; DWORD WINAPI threadFunc(LPVOID arg ) { int mySum = bigComputation(); EnterCriticalSection(&g_cs); g_sum += mySum; // threads access one at a time LeaveCriticalSection(&g_cs); return 0; } main() { HANDLE hThread[NUMTHREADS]; InitializeCriticalSection(&g_cs); for (int i = 0; i < NUMTHREADS; i++) hThread[i] = CreateThread(NULL,0,threadFunc,NULL,0,NULL); WaitForMultipleObjects(NUMTHREADS, hThread, TRUE, INFINITE); DeleteCriticalSection(&g_cs); }

25. 25 Programming with Windows Threads Numerical Integration Example 4.0 2.0 1.0 0.0 4.0 (1+x 2 ) f(x) =  4.0 (1+x 2 ) dx =  0 1 X static long num_steps=100000; double step, pi; void main() { int i; double x, sum = 0.0; step = 1.0/(double) num_steps; for (i=0; i< num_steps; i++){ x = (i+0.5)*step; sum = sum + 4.0/(1.0 + x*x); } pi = step * sum; printf(“Pi = %f\n”,pi); }

26. 26 Programming with Windows Threads Activity 2 - Computing Pi Parallelize the numerical integration code using Windows* Threads How can the loop iterations be divided among the threads? What variables can be local? What variables need to be visible to all threads? static long num_steps=100000; double step, pi; void main() { int i; double x, sum = 0.0; step = 1.0/(double) num_steps; for (i=0; i< num_steps; i++){ x = (i+0.5)*step; sum = sum + 4.0/(1.0 + x*x); } pi = step * sum; printf(“Pi = %f\n”,pi);}

27. 27 Programming with Windows Threads Windows* Events Used to signal other threads that some event has occurred – Data is available, message is ready Threads wait for signal with WaitFor * function Two kinds of events – Auto-reset – Manual-reset

28. 28 Programming with Windows Threads Types of Events Event stays signaled until any one thread waits and is released – If no threads waiting, state stays signaled – Once thread is released, state reset to non-signaled Event remains signaled until reset by API call – Multiple threads can wait and be released – Threads not originally waiting may start wait and be released Caution: Be careful when using WaitForMultipleObjects to wait for ALL events Manual-resetAuto-reset

29. 29 Programming with Windows Threads Windows* Event Creation HANDLE CreateEvent( LPSECURITY_ATTRIBUTES lpEventAttributes, BOOL bManualReset, // TRUE => manual reset BOOL bInitialState, // TRUE => begin signaled LPCSTR lpName); // text name for object Set bManualReset to TRUE for manual- reset event; FALSE for auto-reset event Set bInitialState to TRUE for event to begin in signaled state; FALSE to begin unsignaled

30. 30 Programming with Windows Threads Event Set and Reset Set an event to signaled state BOOL SetEvent( HANDLE event ); Reset manual-reset event BOOL ResetEvent( HANDLE event ); Pulse event BOOL PulseEvent( HANDLE event );

31. 31 Programming with Windows Threads Example: Thread Search Created thread searches for item – Signals if item is found Main thread waits for signal and thread termination – Print message if item found – Print message upon thread termination Illustrates – Using generic HANDLE type – Not waiting for every object to signal

32. 32 Programming with Windows Threads Example: Events DWORD WINAPI threadFunc(LPVOID arg) { BOOL bFound = bigFind() ; if (bFound) { SetEvent(hObj[0]); // signal data was found bigFound() ; } moreBigStuff() ; return 0; } HANDLE hObj[2]; // 0 is event, 1 is thread

33. 33 Programming with Windows Threads Example: Main function... hObj[0] = CreateEvent(NULL, FALSE, FALSE, NULL); hObj[1] = CreateThread(NULL,0,threadFunc,NULL,0,NULL); /* Do some other work while thread executes search */ DWORD waitRet = WaitForMultipleObjects(2, hObj, FALSE, INFINITE); switch(waitRet) { case WAIT_OBJECT_0: // event signaled printf("found it!\n"); WaitForSingleObject(hObj[1], INFINITE) ; // fall thru case WAIT_OBJECT_0+1:// thread signaled printf("thread done\n"); break ; default: printf("wait error: ret %u\n", waitRet); break ; }...

34. 34 Programming with Windows Threads Example: Main function... hObj[0] = CreateEvent(NULL, FALSE, FALSE, NULL); hObj[1] = CreateThread(NULL,0,threadFunc,NULL,0,NULL); /* Do some other work while thread executes search */ DWORD waitRet = WaitForMultipleObjects(2, hObj, FALSE, INFINITE); switch(waitRet) { case WAIT_OBJECT_0: // event signaled printf("found it!\n"); WaitForSingleObject(hObj[1], INFINITE) ; // fall thru case WAIT_OBJECT_0+1:// thread signaled printf("thread done\n"); break ; default: printf("wait error: ret %u\n", waitRet); break ; }...

35. 35 Programming with Windows Threads Windows* Semaphores Synchronization object that keeps a count – Represents the number of available resources – Formalized by Edsger Dijkstra (1968) Two operations on semaphores – Wait [P(s)]: Thread waits until s > 0, then s = s-1 – Post [V(s)]: s = s + 1 Semaphore is in signaled state if count > 0

36. 36 Programming with Windows Threads Win32* Semaphore Creation HANDLE CreateSemaphore( LPSECURITY_ATTRIBUTES lpEventAttributes, LONG lSemInitial, // Initial count value LONG lSemMax, // Maximum value for count LPCSTR lpSemName); // text name for object Value of lSemMax must be 1 or greater Value of lSemInitial must be – greater than or equal to zero, – less than or equal to lSemMax, and – cannot go outside of range

37. 37 Programming with Windows Threads Wait and Post Operations Use WaitForSingleObject to wait on semaphore – If count is == 0, thread waits – Decrement count by 1 when count > 0 Increment semaphore (Post operation) BOOL ReleaseSemaphore( HANDLE hSemaphore, LONG cReleaseCount, LPLONG lpPreviousCount ); – Increase semaphore count by cReleaseCount – Returns the previous count through lpPreviousCount

38. 38 Programming with Windows Threads Semaphore Uses Control access to data structures of limited size – Queues, stacks, deques – Use count to enumerate available elements Control access to finite number of resourse – File descriptors, tape drives… Throttle number of active threads within a region Binary semaphore [0,1] can act as mutex

39. 39 Programming with Windows Threads Semaphore Cautions No ownership of semaphore Any thread can release a semaphore, not just the last thread that waits – Use good programming practice to avoid this No concept of abandoned semaphore – If thread terminates before post, semaphore increment may be lost – Deadlock

40. 40 Programming with Windows Threads Example: Semaphore as Mutex Main thread opens input file, waits for thread termination Threads will – Read line from input file – Count all five-letter words in line

41. 41 Programming with Windows Threads Example: Main main() { HANDLE hThread[NUMTHREADS]; hSem1 = CreateSemaphore(NULL, 1, 1, NULL); // Binary semaphore hSem2 = CreateSemaphore(NULL, 1, 1, NULL); // Binary semaphore fd = fopen(“InFile”, “r”); // Open file for read for (int i = 0; i < NUMTHREADS; i++) hThread[i] = CreateThread(NULL,0,CountFives,NULL,0,NULL); WaitForMultipleObjects(NUMTHREADS, hThread, TRUE, INFINITE); fclose(fd); printf(“Number of five letter words is %d\n”, fiveLetterCount); } HANDLE hSem1, hSem2; FILE *fd; int fiveLetterCount = 0;

42. 42 Programming with Windows Threads Example: Semaphores DWORD WINAPI CountFives(LPVOID arg) { BOOL bDone = FALSE ; char inLine[132]; int lCount = 0; while (!bDone) { WaitForSingleObject(hSem1, INFINITE); // access to input bDone = (GetNextLine(fd, inLine) == EOF); ReleaseSemaphore(hSem1, 1, NULL); if (!bDone) if (lCount = GetFiveLetterWordCount(inLine)) { WaitForSingleObject(hSem2, INFINITE); // update global fiveLetterCount += lCount; ReleaseSemaphore(hsem2, 1, NULL); }

43. 43 Programming with Windows Threads Programming with Windows Threads What’s Been Covered Create threads to execute work encapsulated within functions Typical to wait for threads to terminate Coordinate shared access between threads to avoid race conditions – Local storage to avoid conflicts – Synchronization objects to organize use