Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners Improve Application Performance on Windows*

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners What is the world’s biggest semiconductor company doing building software products?

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 3 Intel ® Software Development Products  Intel® Compilers Best way to get application performance on Intel processors  Intel® VTune™ Analyzers Quickly identify “hot spots” and how to fix them  Intel® Performance Libraries Highly optimized, ready to use building-block functions  Intel® Threading Tools Speeds, simplifies development & maintenance of threaded apps  Intel® Cluster Tools Create, analyze, optimize and deploy cluster-based applications Intel Software Development Products for Intel® Personal Internet Client Architecture processors, Pentium® M, Pentium® 4, Intel® Xeon™ and Itanium® 2 Processors

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 4 Intel® Software Development Products  Performance –Enable developers to deliver higher performance software  Compatibility –Compatible with the leading tools and development environments already used by many software developers –Easy to incorporate into the development process  Support –Premier Customer Support –Technical training offered through Intel Software College

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners Intel Compilers

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 6 Compilers for Intel PCA, Intel® 32-bit, EM64T & Itanium® 2 Processors  Intel compilers for Intel PCA processor line support Intel® Wireless MMX™ technology  Intel 32-bit processor support: SSE3, Intel Net Burst® microarchitecture, Hyper-threading  Itanium® 2 processor support: software pipelining, improved branch prediction, branch reduction thru predication  Advanced optimization features of Intel compilers –Profile Guided Optimization, Inter-Procedural Optimization –Parallelism: Auto-parallelization, vectorization, OpenMP* support –Data prefetching –Processor dispatch on IA-32 processors  Intel® Premier Support: Compiler updates, support, expertise, customer interaction via compiler forums, architectural information, white papers and more

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 7 Intel Compilers Optimize for Specific Processors Instruction Scheduling –Schedule instructions to be optimal for specific processor –How? On Windows: /G1, /G2, /G5, /G7… Build target for specific processor –For target processor it uses processor specific opcodes & features like SSE, SSE2, Vectorization –Runs only the target processor –How? On Windows*: /QxK, /QxW, QxB… Automatic Processor Dispatch –Runs on all x86 processors –How? On Windows*: /QaxK, /QaxW, /QaxB…

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 8 Intel Compilers High-Level Optimizations High-Level Optimizer –Performs loop level optimizations, aids optimal memory access –How? On Windows: /O3 Inter-Procedural Optimization –Enables inter-procedural optimizations for single/ multiple files –How? On Windows*: /Qip, /Qipo Profile Guided Optimization –Use execution-time feedback to guide optimization –Aids paging, branch-prediction, basic block reordering –How? On Windows*: /Qprof_gen, /Qprof_use

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 9 Intel Compilers Using Parallel Programming Directives Auto-Parallelization –Automatically converts loops to use multiple processors –How? On Windows*: /Qparallel OpenMP Support –Intel Compilers supports multi-platform shared-memory parallel programming in C/C++ and FORTRAN on all platforms & OS –How? On Windows*: /Qopenmp OpenMP usage example #pragma omp parallel for for (i = 0;i < n; i++) { dy[i] = dy[i] + da*dx[i]; } dy[i] = dy[i] + da*dx[i]; }

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 10 Intel® Code Coverage Tool Example of code coverage summary for a project. The workload applied in this test exercised 34 of 143 blocks, representing 5 of 19 functions in 2 of 3 modules. In the file, SAMPLE.C, 4 of 5 functions were exercised Clicking on SAMPLE.C produces a listing that highlights the code that was exercised. In this example, the pink-highlighted code was never exercised, the yellow was run but not exercised by any of the tests set up by the developer and the beige was partially covered.

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 11 Intel® Test Prioritization Tool  Helps guide and speed software testing, –Helps produce better code more quickly –Helps improve programmer productivity  Example: –These 3 achieve 52.17% block and 50.00% function coverage –Test 3 alone covers 45.65% of basic blocks or 87.50% of total block coverage from all tests –By adding Test 2, cumulative block coverage goes to 52.17%, or 100% of the total block coverage of Test 1, Test 2, and Test 3 –Eliminating Test 1 has no negative impact on block coverage and saves time

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 12 Intel® Compilers 8.1  C++ and Fortran  IA-32, Intel® Itanium® 2, EM 64T & Intel® PCA processor-based systems  Intel® Code-Coverage & Intel® Test-Prioritization tools  Threaded application support (Hyper-Threading Technology) –OpenMP* 2.0 standard support –Auto-Parallel feature that automatically generates threaded code  Windows specific: –Integrates into MS Visual Studio.NET* IDE –Support for MSVC.NET* language features (no support for C# or managed code) –Compaq Visual Fortran* language features with Intel code generation and optimization technology

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners Intel VTune Performance Analyzer

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 14 Performance Tuning  Detecting common issues –Where to add threads, what to optimize? –Load imbalance? –Wait, blocked, or idle time? –Excessive overhead? –Processor architecture issues? –Application issues? No particular order: Address issues as needed

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 15 Intel ® VTune™ Performance Analyzer  VTune analyzer’s intimate knowledge of the processor enables it to provide extensive insights into how software utilizes CPU resources  Allows you to identify and locate performance bottlenecks in your code –Collects and displays software performance data –Features that help you identify and address performance issues:  Sampling that uses non-intrusive technologies  Call Graph that displays graphically the program’s flow of control  Analyzer that has detailed knowledge of the processor’s microarchitecture  Intel Tuning Assistant that suggests optimization techniques for your Windows code “The Intel VTune Performance Analyzer took a multi-day task and turned it into a sub-day task.” — — Randy Camp, V.P. Software Research and Development, MUSICMATCH, Inc.

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 16 Sampling – Identifying Performance Bottlenecks  “Sample” the CPU’s execution context  As program runs, gather occasional CPU context snapshots triggered by CPU’s performance monitoring registers –Interrupt based sampling using CPU registers –Low intrusion – doesn’t change performance of the software –No special builds required  Sample rate set to provide statistically meaningful data –Based on CPU clock speed or can be auto-calibrated  Can measure performance sensitive CPU events –Cache misses, branch mispredictions, etc.

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 17 How to use Intel VTune Performance Analyzer Build the application –Build the application in Release mode with compiler optimizations Find “Hotspots” using VTune –A “Hotspot” in an application or a system is a section of code where there is a significant amount of activity. –Finding “hotspots” would assist you in determining the compiler/ code optimizations required for gaining performance improvement. Symbols required for VTune Analyzer –Required Intel compiler switch (on Windows*): /Zi

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 18 Start New Project using Sampling Wizard Intel VTune Performance Analyzer Select Application Type to Profile Select Application to Launch

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 19 Understanding VTune Interface Choose Project/ Activity/ Run Choose Project/ Activity/ Run Different Views System-wide performance data Most Instructions Retired Statistics Summary Events Measured Sampling Analysis Per CPU Analysis Status Output

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 20 Hotspot Drill Down Function Statistics LINPACK performance data Symbols required for Hotspot Drill-down Events Measured Is this the Hotspot? More analysis needed. Use VTune Call Graph feature to obtain flow info!

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 21 Source Level View “Hotspot” source Efficiency (CPI) View Assembly

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 22 Using Sampling & Call Graph Together Why?  Use sampling to find which functions have hotspots.  Use call graph to find out who is calling these functions. Why?  Use sampling to find which functions have hotspots.  Use call graph to find out who is calling these functions.

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 23 What Are Users Saying “SGI develops applications for its computers that employ many levels of parallelism, demanding the highest level of performance. The VTune Performance Analyzer for Windows provided invaluable insights to the correction of performance bottlenecks in these applications at the process, thread, and basic block levels." –Arthur Raefsky, Technical Lead, SGI, Mountain View, CA

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners Intel Threading Tools

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 25 Threads Defined  OS creates process for each program loaded –Each process executes as a separate thread  Additional threads can be created within the process  All threads share code and data –Each thread has its own Stack and Instruction Pointer  OS creates process for each program loaded –Each process executes as a separate thread  Additional threads can be created within the process  All threads share code and data –Each thread has its own Stack and Instruction Pointer … Data Code thread2() Stack IP threadN() Stack IP Process thread1() Stack IP Threading Overview

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 26 Amdahl’s Law Threading Overview If only 1/2 of the code is parallel, 2X speedup is unlikely If only 1/2 of the code is parallel, 2X speedup is unlikely P = parallel portion of process N = number of processors (cores) O = parallel overhead time PP P(1-P) T Total

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 27 Correctness Bugs: Data Races Threading Overview: Challenges Unique to Threading Thread1 x = a + b Thread2 b = 42  What is value of x if: –Thread1 runs before Thread2? –Thread2 runs before Thread1?  Data race: concurrent read, modify, write of same address x = 3 x = 43  Suppose: a=1, b=2 Outcome depends on thread execution order

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 28 Solving Data Races: Synchronization Thread1 Acquire(L) a = 1 b = 2 x = a + b Release(L)  Acquisition of mutex L ensures atomic access –Only one thread can hold lock at a time  Example APIs: -EnterCriticalSection(), LeaveCriticalSection() -pthread_mutex_lock(), pthread_mutex_unlock() Thread2 Acquire(L) b = 42 Release(L) Threading Overview: Challenges Unique to Threading

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 29 Performance Penalty: Synchronization  Thread blocked waiting for Mutex –Thread not running, so no parallelism  Mutex Release, Acquire takes time –Release marks mutex free –Acquire must check for free  If free, mark as in use  If not free, thread put to sleep –Costs context switch out and in of processor Threading Overview: Challenges Unique to Threading

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 30 Problem Statement  Developing threaded applications is hard  New class of problems are caused by the interaction between concurrent threads –Correctness problems (data races, deadlocks, etc) –Performance problems (contention, imbalance, etc) Threading Overview

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 31 Software Development Cycle Introduce Threads –Intel® Performance libraries: IPP and MKL –OpenMP* (supports incremental threading) –Explicit threading (Win32*, Pthreads*) Debug for correctness –Intel® Thread Checker –Intel Debugger Tune for performance –Thread Profiler –VTune™ Performance Analyzer Scope of the Tools

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 32 Intel® Software Development Products  Intel® Thread Checker and Thread Profiler  VTune™ Performance Analyzer –Prerequisite for Intel® Threading Tools –VTune analyzer has thread support  Intel® Compilers support OpenMP* and the Threading tools –More detailed results are generated with the Intel compilers  Intel Performance Libraries are thread safe –Many functions are threaded

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 33 Common Threading Errors/Bugs  Race conditions –Unprotected concurrent access to shared variables by multiple threads –Most common error  Deadlocks –Multiple threads waiting on resources that are held by other threads  Thread stalls –Threads waiting on resources infinitely

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 34 Intel® Thread Checker Intro  Identifies threading bugs in applications threaded with: –Windows* threads on Windows* systems –OpenMP* on Windows* systems  Plugs into VTune™ environment –Windows* for IA-32 systems Intel® Thread Checker

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 35 Intel® Thread Checker Analysis  Dynamic monitoring as software runs –Data (workload) -driven execution  Includes monitoring of: –Thread and Sync APIs used –Thread execution order  Scheduler impacts results –Memory accesses between threads Only executed code path is analyzed Intel® Thread Checker

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 36 Thread Checker Usage  Dynamic Correctness tool –Dataset selection is important  Must touch all code paths –Multiple runs exercising different data paths yield best results –Use small data set for each path  Monitoring of all memory references is time consuming Intel® Thread Checker

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 37 Starting Thread Checker  Start VTune™Performance Analyzer 1 2 Intel® Thread Checker

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 38 Diagnostics List Intel® Thread Checker

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 39 Location in Source Code Each entry in the diagnostics list links to its source code line(s) Intel® Thread Checker

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 40 Common Performance Issues  Parallel Overhead –Due to thread creation, scheduling..  Synchronization –Excessive use of global data, contention for the same synchronization object –Implicit synchronization  Load balance –Improper distribution of parallel work  Granularity –No sufficient parallel work

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 41 Thread Profiler  Plugs in to the VTune™ performance environment  Identifies performance issues in OpenMP* or unstructured threaded applications using the Win32*  Pinpoints performance bottlenecks that directly affect execution time  Uses binary instrumentation technology Intel® Threading Tools: Thread Profiler

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 42 Thread Profiler  Uses critical path analysis  Provides a breakdown of execution time along the critical path –Provides insight into system utilization  Under-subscribed vs. over-subscribed –Thread state transitions  Blocked->Running, call stack information Allows comparison of multiple runs Intel® Threading Tools: Thread Profiler

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 43 Execution Flows and Critical Path  Multiple execution flows in applications  Flow splits when a thread creates new threads or signals another thread to continue  Flow ends when a thread stalls or terminates Thread 1 Thread 2 Thread 3 T0T0 T1T1 T2T2 T3T3 T4T4 T5T5 T6T6 T7T7 T8T8 T9T9 T 10 T 11 T 12 T 13 T 14 T 15 Acquire lock L Wait for Threads 2 & 3 Wait for L Release LWait for L Release L critical path Longest flow is the critical path Intel® Threading Tools: Thread Profiler

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 44 Why use Critical Path?  Goal is to shorten the execution time  Shorten the critical path and you shorten the total execution time  Events recorded are events that impact the critical path –Lock/Unlock –Thread Creation, suspension, resume, termination –Blocking calls, external events Intel® Threading Tools: Thread Profiler

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 45 Critical Path Analysis  System Utilization –Idle, serial, parallel and oversubscribed –This is relative to the system the application is running on  Time categories along critical path (CP) –Cruise, overhead, blocking and impact time  Resulting view is a combination of utilization and execution time along CP Intel® Threading Tools: Thread Profiler

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 46 System Utilization  Examines processor utilization to determine parallel activity of the application  Concurrency is the number of threads that are active Thread 1 Thread 2 Thread 3 T0T0 T1T1 T2T2 T3T3 T4T4 T5T5 T6T6 T7T7 T8T8 T9T9 T 10 T 11 T 12 T 13 T 14 T 15 Thread Profiler: Critical Path Analysis Categorization shown for a system configuration with 2 processors Acquire lock L Wait for Threads 2 & 3 Wait for L Release LWait for L Release L Idle Serial Parallel Under-subscribed Over-subscribed

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 47 Execution Time Categories  Analyze critical path by “colorizing” the time spent along it.  Associate spans of time with the objects that caused the critical path transitions Thread Profiler: Critical Path Analysis Thread 1 Thread 2 Thread 3 T0T0 T1T1 T2T2 T3T3 T4T4 T5T5 T6T6 T7T7 T8T8 T9T9 T 10 T 11 T 12 T 13 T 14 T 15 Cruise time Overhead Blocking time Impact time Acquire lock L Wait for Threads 2 & 3 Wait for L Release LWait for L Release L

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 48 Critical Path View Thread 1 Thread 2 Thread 3 T0T0 T1T1 T2T2 T3T3 T4T4 T5T5 T6T6 T7T7 T8T8 T9T9 T 10 T 11 T 12 T 13 T 14 T 15 Thread Profiler: Critical Path Analysis Critical Path View Time  Start with the critical path  Break down by system utilization  Add overhead  Further categorize by behavior Acquire lock L Wait for Threads 2 & 3 Wait for L Release LWait for L Release L Idle Serial Parallel Under-subscribed Over-subscribed Categorization shown for a system configuration with 2 processors Cruise time Overhead Blocking time Impact time

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 49 Thread Profiler Views  Critical Path View –Shows breakdown of the critical path  Profile View –Shows the breakdown of selected critical paths –Use can select other views of the selected profile –Concurrency level, threads, objects..  Timeline View –Shows thread activity and critical path transitions for the entire application  Source View –Transition source view, creation source view Intel® Threading Tools: Thread Profiler

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 50 Intel® Thread Checker  Locates threading bugs: –Data races (storage conflicts) –Deadlocks (potential and actual)  Isolates bugs to source code line  Describes possible causes of errors and suggests resolutions  Categorizes errors by severity level  Identifies threading bugs in applications threaded with: –Windows* threads on Windows* systems –OpenMP* on Windows* systems  Plugs into VTune™ environment –Windows* for IA-32 systems

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 51 Thread Profiler 2.1  Plugs in to the VTune™ performance environment  Identifies performance issues in OpenMP* or unstructured threaded applications using the Win32*  Pinpoints performance bottlenecks that directly affect execution time  Uses binary instrumentation technology

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners Intel Software College

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 53 Expert Intel ® Software College  High-quality training by expert trainers worldwide –Take advantage of the latest Intel processors, platforms, tools and technologies  Flexible training offerings –On-line, On-site, or at Intel facility  Classroom-based or online, self-paced or custom course offerings Visit the Intel Software College website: "I attended the VTune and Compiler courses at the ISC … I am able to apply what I learned at the ISC to optimizing applications that matter to my company's business. The ISC courses were probably the best that I have had as a professional in terms of delivering on what they said they would teach." — — Keith Fish - ISV Technical Consultant, Hewlett- Packard Company

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 54 “Registering for support was easy, and we value the security of knowing that Intel is there to help, even though we haven’t needed it so far.” — — Rob Hoffmann - Director of Marketing, NewTek, Inc. Intel Premier Support  Every purchase of an Intel software development product includes a year of support services  Provides access to Intel® Premier Support and all product updates during that time  Premier Support includes online access to Intel’s Premier Support Website –Primary support for all Intel Software products –Issue submission & tracking –Product updates & related downloads –FAQ’s & other proactive notices –128-bit encrypted communication protects confidentiality –Dedicated expert staff review submissions and respond within 4 Intel business hours

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 55 Intel ® Software Development Products From Supercomputers to Cell Phones, Intel Software Development Products Enable Application Development Across Intel Processors VTune™ Performance Analyzer Libraries Threading Tools Compilers Math Kernel Library Integrated Performance Primitives Thread Checker C++ C++ MS Windows* Win CE Intel Software Development Products Fortran NA ShippingFuture Performance Analyzers Cluster Tools NA Trace Analyzer / Collector NA Palm* Symbian* Nucleus* Debuggers C++ C++ NA

Copyright © 2002, Intel Corporation. All rights reserved. *Other brands and names are the property of their respective owners 56 Next Steps  Evaluate the Products –Download at:  Contact Vivek Venkatesh with questions –