Download presentation
Presentation is loading. Please wait.
Published byMadeleine Jones Modified over 9 years ago
1
Copyright © 2003, SAS Institute Inc. All rights reserved. Where's Waldo Uncovering Hard-to-Find Application Killers Claire Cates SAS Institute, Inc Claire.Cates@sas.com
2
Copyright © 2003, SAS Institute Inc. All rights reserved. 2
3
3 Always something different Just like the Waldo pictures, performance is always different and not easy to find. Performance based on the test that was run Performance based on the data that was used Performance based on the hardware used Performance based on the number of users
4
Copyright © 2003, SAS Institute Inc. All rights reserved. 4 What is a Performance Problem When jobs cannot execute within a required time period. Something that makes the system run slower than it should When a system cannot sustain expected or unexpected growth (scalability) More users More data More processors
5
Copyright © 2003, SAS Institute Inc. All rights reserved. 5 Causes of Performance Problems Hardware Software Integration of software systems Network Web
6
Copyright © 2003, SAS Institute Inc. All rights reserved. 6 How Do You Find Performance Problems Vendor profilers Looking at system variables (cache hits, IO timings,…) Profilers that analyze the code Code reviews Network sniffers Profiles that profile the entire machine Code instrumentation
7
Copyright © 2003, SAS Institute Inc. All rights reserved. 7 Common Software Performance Problems Antipatterns Excessive Memory Allocation The Ramp Unnecessary Processing One Lane Bridge How Many Times Do I have to tell you. Unbalanced Processing More is Less
8
Copyright © 2003, SAS Institute Inc. All rights reserved. 8 Excessive Memory Allocation Memory Allocations/Frees are expensive OS calls If your system is multi-threaded, locking may need to occur Often times memory is allocated temporarily within a function Often times structures are allocated and freed and not reused Memory allocations in loops. Possible Fixes Use a stack variable instead of temporary memory allocations Maintain a free chain for structures that are used often Make one large allocation outside loop – split it within the loop
9
Copyright © 2003, SAS Institute Inc. All rights reserved. 9 The Ramp When processing slows down over time Can be caused by long lists that grow and need to be searched Can be caused by too many resources being used Another kind of Ramp happens when the system doesn’t scale well. Possible fixes Use a different algorithm when you know large amounts of data will be required Make sure all resources (especially memory) are returned.
10
Copyright © 2003, SAS Institute Inc. All rights reserved. 10 Unnecessary Processing Loops that contain statements that do not need to execute on each iteration of the loop Macros that are inefficient because they are too general Zeroing of allocated memory that does not need to be zeroed. Inefficient algorithms Possible fixes Rework algorithm Don’t use macros
11
Copyright © 2003, SAS Institute Inc. All rights reserved. 11 One Lane Bridge Resource contention – every thread waiting on the same resource Eventually causes a traffic jam Possible fixes Change the algorithm to use less resources that are shared
12
Copyright © 2003, SAS Institute Inc. All rights reserved. 12 How Many Times Do I Have To Tell You Calling a routine too often with the same parameters Sometimes a routine (foo) is the hot spot yet, (foo) is fine, it is just that foo is called to often. Most common occurrence of this that I see occurs when (afoo) (bfoo) (cfoo) (foo) So cfoo is called twice and foo is called 4 times all because afoo does not know what bfoo and cfoo actually call
13
Copyright © 2003, SAS Institute Inc. All rights reserved. 13 Unbalanced Processing Occurs in a multi-threaded environment running on a multi-processor machine. When one or two threads do all the processing while other threads are idle Possible fixes Rebalance the work in the threads so that all threads process a similar amount of work
14
Copyright © 2003, SAS Institute Inc. All rights reserved. 14 More is Less or Thundering Herd Adding threads does not make a system run faster The system spends its time thrashing Thrashing can occur because of Excessive thread switching Destroying the system memory or I/O caches when each thread runs Possible fixes Don’t create more threads than there is work to be done
15
Copyright © 2003, SAS Institute Inc. All rights reserved. 15 Common Profilers in Use at SAS InTune – z/OS VTune – Intel – PCs Caliper – HP Rational Quantify – Windows, Solaris, HP, Linux Jprobe OS tools – not really profilers but gives information on −System calls −IO bottlenecks −Caching problems
16
Copyright © 2003, SAS Institute Inc. All rights reserved. 16 SAS’s use of Quantify Other Profilers we use Event / timer based Based more on the function level, not the source level Network traffic & other processes on the machine distorted the results Great results from all profilers Tips I’m using with Quantify do apply to these other tools
17
Copyright © 2003, SAS Institute Inc. All rights reserved. 17 Quantify differences… Quantify Counts instruction cycles for the user code Times the system calls (can be turned off) Does not have to run on a clean machine (a machine with no other activity) – developer machine Provides performance information down to the source line level Performance can be broken down by threads
18
Copyright © 2003, SAS Institute Inc. All rights reserved. 18 A Little about Quantify Quantify integrates itself with your system. As your system executes, Quantify will accumulate a mass of data. Function counts for every function used The amount of time spent in every function The amount of time spent in every function plus the amount of time spent in all its descendent routines. Full Call tree information
19
Copyright © 2003, SAS Institute Inc. All rights reserved. 19 Quantify Quick Overview
26
Copyright © 2003, SAS Institute Inc. All rights reserved. 26 Excessive Memory Allocation
36
Copyright © 2003, SAS Institute Inc. All rights reserved. 36 How Many Times Do I Have to Tell You?
43
Copyright © 2003, SAS Institute Inc. All rights reserved. 43 More Is Less
47
Copyright © 2003, SAS Institute Inc. All rights reserved. 47
52
Copyright © 2003, SAS Institute Inc. All rights reserved. 52 Unnecessary Processing
55
Copyright © 2003, SAS Institute Inc. All rights reserved. 55 One Lane Bridge
59
Copyright © 2003, SAS Institute Inc. All rights reserved. 59 Unbalanced Processing
64
Copyright © 2003, SAS Institute Inc. All rights reserved. 64 Ramp
70
Copyright © 2003, SAS Institute Inc. All rights reserved. 70 Another Useful Hint
72
Copyright © 2003, SAS Institute Inc. All rights reserved. 72 Summary Profilers are useful in helping to uncover performance problems in a software application By being patient and looking at the mass of data provided by the tools you can find out a tremendous amount about what your system is doing and how to optimize the performance.
73
Copyright © 2003, SAS Institute Inc. All rights reserved. 73 References AntiPatterns – Refactoring Software, Architectures, and Projects in Crisis - Brown, Malveau, McCormick & Mowbray ISBN 0-471-19713-0 New Software Performance AntiPatterns: More Ways to Shoot Yourself in the Foot. Smith & Williams CMG 2002 Paper Presentation Computer Measurement Group www.cmg.org
74
Copyright © 2003, SAS Institute Inc. All rights reserved. 74
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.