1 Make and Gprof COS 217. 2 Goals of Today’s Lecture Overview of two important programming tools  Make for compiling and linking multi-file programs.

Slides:



Advertisements
Similar presentations
Chapter 11 Introduction to Programming in C
Advertisements

CS 11 C track: lecture 7 Last week: structs, typedef, linked lists This week: hash tables more on the C preprocessor extern const.
GNU gprof Profiler Yu Kai Hong Department of Mathematics National Taiwan University July 19, 2008 GNU gprof 1/22.
Programming Types of Testing.
The Makefile utility ABC – Chapter 11, Motivation Small programs single file “Not so small” programs : –Many lines of code –Multiple components.
1 The Makefile Utility ABC – Chapter 11,
The Makefile Utility ABC – Chapter 11, Motivation Small programs single file “Not so small” programs : –Many lines of code –Multiple components.
Performance Improvement
Guide To UNIX Using Linux Third Edition
The Makefile Utility ABC – Chapter 11, Motivation Small programs single file “Not so small” programs : –Many lines of code –Multiple components.
1 Building. 2 Goals of this Lecture Help you learn about: The build process for multi-file programs Partial builds of multi-file programs make, a popular.
G++ and make Dan Wilson CS193 02/01/06. The g++ Compiler What happens when you call g++ to build your program? Phase 1, Compilation:.cpp files are compiled.
Prof. R. Willingale Department of Physics and Astronomy 2nd Year C+R 2 nd Year C and R Workshop Part of module PA2930 – 2.5 credits Venue: Computer terminal.
Spring 2014 SILICON VALLEY UNIVERSITY CONFIDENTIAL 1 Introduction to Embedded Systems Dr. Jerry Shiao, Silicon Valley University.
Chocolate Bar! luqili. Milestone 3 Speed 11% of final mark 7%: path quality and speed –Some cleverness required for full marks –Implement some A* techniques.
Data Representation, Data Structures, and Multi-file compilation.
Homework Reading Programming Assignments
Make: the good, the bad, and the ugly Dan Berger Titus Winters
CIS*2450 Seminar I Makefiles Debugging/Design Approaches Exception Handling Library Revision Control Designed by: Terry Moreland Updated by: Tom Crabtree.
Unix Makefiles COP 3330 Lecture Notes Dr. David A. Gaitros.
1 Programming Languages Tevfik Koşar Lecture - II January 19 th, 2006.
Programming Tools gcc make utility Open Source code Static and Shared Libraries gdb Memory debugging tools.
Old Chapter 10: Programming Tools A Developer’s Candy Store.
Introduction to C Programming CE Lecture 7 Compiler options and makefiles.
C Tutorial Session #2 Type conversions More on looping Common errors Control statements Pointers and Arrays C Pre-processor Makefile Debugging.
History of C 1950 – FORTRAN (Formula Translator) 1959 – COBOL (Common Business Oriented Language) 1971 – Pascal Between Ada.
Fundamental Programming: Fundamental Programming K.Chinnasarn, Ph.D.
Computer Programming I Hour 2 - Writing Your First C Program.
Application Profiling Using gprof. What is profiling? Allows you to learn:  where your program is spending its time  what functions called what other.
CSE 303 Concepts and Tools for Software Development Richard C. Davis UW CSE – 12/6/2006 Lecture 24 – Profilers.
Makefile M.A Doman. Compiling multiple objects Card.cpp -> Card.o Deck.cpp -> Deck.o main.cpp -> main.o main.o Deck.o Card.o -> Dealer.exe.
Debugging and Profiling With some help from Software Carpentry resources.
Timing Programs and Performance Analysis Tools for Analysing and Optimising advanced Simulations.
Adv. UNIX: Profile/151 Advanced UNIX v Objectives –introduce profiling based on execution times and line counts Special Topics in Comp.
Makefiles. Multiple Source Files (1) u Obviously, large programs are not going to be contained within single files. u C provides several techniques to.
1 Building Jennifer Rexford. 2 Goals of this Lecture Help you learn about: The build process for multi-file programs Partial builds of multi-file programs.
15213 Recitation Section C Introduction Unix and C Playing with Bits Practice Problems Shimin Chen Sept. 9, 2002 Outline.
Design - programming Cmpe 450 Fall Dynamic Analysis Software quality Design carefully from the start Simple and clean Fewer errors Finding errors.
Performance Improvement
Modular Programming. Introduction As programs grow larger and larger, it is more desirable to split them into sections or modules. C allows programs to.
Copyright 2014 – Noah Mendelsohn Performance Analysis Tools Noah Mendelsohn Tufts University Web:
Announcements Assignment 1 due Wednesday at 11:59PM Quiz 1 on Thursday 1.
C P ROGRAMMING T OOLS. C OMPILING AND R UNNING S INGLE M ODULE P ROGRAM.
Advanced UNIX progamming Fall 2002 Instructor: Ashok Srinivasan Lecture 2 Class web site:
Announcements Assignment 1 will be regraded for all who’s score (not percentage) is less than 6 (out of 65). If your score is 6 or higher, but you feel.
10/07/2011ecs40 fall Software Development & Object- Oriented Programming ecs40 Fall 2012: Software Development & Object- Oriented Programming #05:
Brandon Packard. Why make? So far, you have probably worked on relatively small projects Coding projects can become huge My research consists of 1600.
CSc 352 An Introduction to make Saumya Debray Dept. of Computer Science The University of Arizona, Tucson
Winter 2016CISC101 - Prof. McLeod1 CISC101 Reminders Assignment 5 is posted. Exercise 8 is very similar to what you will be doing with assignment 5. Exam.
Introduction to Computer Programming Concepts M. Uyguroğlu R. Uyguroğlu.
Makefiles Manolis Koubarakis Data Structures and Programming Techniques 1.
July 10, 2016ISA's, Compilers, and Assembly1 CS232 roadmap In the first 3 quarters of the class, we have covered 1.Understanding the relationship between.
Makefiles CSSE 332 Operating Systems
Profiling with GNU GProf
Precept 10 : Building and Performance
Performance Analysis Tools
Winter 2009 Tutorial #6 Arrays Part 2, Structures, Debugger
Prof: Dr. Shu-Ching Chen TA: Hsin-Yu Ha Fall 2015
Brief Intro to Make CST494/ Gannod.
Editor, Compiler, Linker, Debugger, Makefiles
Makefile Tutorial CIS5027 Prof: Dr. Shu-Ching Chen
Chapter 11 Introduction to Programming in C
Makefiles and Notes on Programming Assignment PA2
CMPSC 60: Week 4 Discussion
Performance Improvement Revisited
Min Heap Update E.g. remove smallest item 1. Pop off top (smallest) 3
Appendix F C Programming Environment on UNIX Systems
Preparation for Assignment 2
Compiler vs linker The compiler translates one .c file into a .o file
Makefiles, GDB, Valgrind
Presentation transcript:

1 Make and Gprof COS 217

2 Goals of Today’s Lecture Overview of two important programming tools  Make for compiling and linking multi-file programs  Gprof for profiling to identify slow parts of the code Make  Overview of compilation process  Motivation for using Makefiles  Example Makefile, refined in five steps Gprof  Timing, instrumenting, and profiling  GNU Performance Profiler (Gprof)  Running gprof and understanding the output

3 Example of a Three-File Program Program divided into three files  mymath.h : interface, included in mymath.c and testmath.c  mymath.c : implementation of math functions  testmath.c : implementation of tests of the math functions Creating the testmath binary executable mymath.hmymath.ctestmath.c testmath gcc –Wall –ansi –pedantic –o testmath testmath.c mymath.c

4 Many Steps, Under the Hood Preprocessing ( gcc –E mymath.c > mymath.i )  Removes preprocessor directives  Produces mymath.i and testmath.i Compiling ( gcc –S mymath.i )  Converts to assembly language  Produces mymath.s and testmath.s Assembling ( gcc –c mymath.s )  Converts to machine language with unresolved directives  Produces the mymath.o and testmath.o binaries Linking ( gcc –o testmath testmath.o mymath.o –lc )  Creates machine language exectutable  Produces the testmath binary

5 Motivation for Makefiles Typing at command-line gets tedious  Long command with compiler, flags, and file names  Easy to make a mistake Compiling everything from scratch is time-consuming  Repeating preprocessing, compiling, assembling, and linking  Repeating these steps for every file, even if just one has changed UNIX Makefile tool  Makefile: file containing information necessary to build a program –Lists the files as well as the dependencies  Recompile or relink only as necessary –When a dependent file has changed since command was run –E.g. if mymath.c changes, recompile mymath.c but not testmath.c  Simply type “make”, or “make –f ”

6 Main Ingredients of a Makefile Group of lines  Target: the file you want to create  Dependencies: the files on which this file depends  Command: what to execute to create the file (after a TAB) Examples testmath: testmath.o mymath.o gcc –o testmath testmath.o mymath.o mymath.o: mymath.c mymath.h gcc -Wall -ansi -pedantic -c -o mymath.o mymath.c

7 Complete Makefile #1 testmath: testmath.o mymath.o gcc –o testmath testmath.o mymath.o testmath.o: testmath.c mymath.h gcc -Wall -ansi -pedantic -c -o testmath.o testmath.c mymath.o: mymath.c mymath.h gcc -Wall -ansi -pedantic -c -o mymath.o mymath.c Three groups  testmath: link testmath.o and mymath.o  testmath.o: compile testmath.c, which depends on mymath.h  mymath.o: compile mymath.c, which depends on mymath.h

8 Adding Non-File Targets Adding useful shortcuts for the programmer  “ make all ”: create the final binary  “ make clobber ”: delete all temp files, core files, binaries, etc.  “ make clean ”: delete all binaries Commands in the example  “ rm –f ”: remove files without querying the user  Files ending in ‘ ~ ’ and starting/ending in ‘ # ”’ are temporary files  “ core ” is a file produced when a program “dumps core” all: testmath clobber: clean rm -f *~ \#*\# core clean: rm -f testmath *.o

9 Complete Makefile #2 # Build rules for non-file targets all: testmath clobber: clean rm -f *~ \#*\# core clean: rm -f testmath *.o # Build rules for file targets testmath: testmath.o mymath.o gcc –o testmath testmath.o mymath.o testmath.o: testmath.c mymath.h gcc -Wall -ansi -pedantic -c -o testmath.o testmath.c mymath.o: mymath.c mymath.h gcc -Wall -ansi -pedantic -c -o mymath.o mymath.c

10 Useful Abbreviations Abbreviations  Target file:  First item in the dependency list: $< Example testmath: testmath.o mymath.o gcc –o testmath testmath.o mymath.o testmath: testmath.o mymath.o gcc –o $< mymath.o

11 Complete Makefile #3 # Build rules for non-file targets all: testmath clobber: clean rm -f *~ \#*\# core clean: rm -f testmath *.o # Build rules for file targets testmath: testmath.o mymath.o gcc –o $< mymath.o testmath.o: testmath.c mymath.h gcc -Wall -ansi -pedantic -c -o $< mymath.o: mymath.c mymath.h gcc -Wall -ansi -pedantic -c -o $<

12 Useful Pattern Rules: Wildcard % Can define a default behavior  Build rule: gcc -Wall -ansi -pedantic -c -o $<  Applied when target ends in “.o” and dependency in “.c” Can omit command clause in build rules %.o: %.c gcc -Wall -ansi -pedantic -c -o $< testmath: testmath.o mymath.o gcc –o $< mymath.o testmath.o: testmath.c mymath.h mymath.o: mymath.c mymath.h

13 Macros for Compiling and Linking Make it easy to change which compiler is used  Macro: CC = gcc  Usage: $(CC) -o $< mymath.o Make it easy to change the compiler flags  Macro: CFLAGS = -Wall -ansi –pedantic  Usage: $(CC) $(CFLAGS) -c -o $< CC = gcc # CC = gccmemstat CFLAGS = -Wall -ansi -pedantic # CFLAGS = -Wall -ansi -pedantic -g # CFLAGS = -Wall -ansi -pedantic -DNDEBUG # CFLAGS = -Wall -ansi -pedantic -DNDEBUG -O3

14 Sequence of Makefiles (see Web)  Initial Makefile with file targets testmath, testmath.o, mymath.o  Adding non-file targets all, clobber, and clean  Adding abbreviations and $<  Adding pattern rules %.o: %.c 5.Adding macros CC and CFLAGS

15 References on Makefiles Brief discussion in the King book  Section 15.4 (pp ) GNU make  Cautionary notes  Don’t forget to use a TAB character, rather than blanks  Be careful with how you use the “ rm –f ” command

16 Timing, Instrumenting, Profiling How slow is the code?  How long does it take for certain types of inputs? Where is the code slow?  Which code is being executed most? Why is the code running out of memory?  Where is the memory going?  Are there leaks? Why is the code slow?  How imbalanced is my hash table or binary tree? Program Output Input

17 Timing Most shells provide tool to time program execution  E.g., bash “ time ” command Breakdown of time  Real: elapsed time between invocation and termination  User: time spent executing the program  System: time spent within the OS on the program’s behalf But, which parts of the code are the most time consuming? bash> time sort output.txt real 0m12.977s user 0m12.860s sys 0m0.010s

18 Instrumenting Most operating systems provide a way to get the time  e.g., UNIX “ gettimeofday ” command #include struct timeval start_time, end_time; gettimeofday(&start_time, NULL); gettimeofday(&end_time, NULL); float seconds = end_time.tv_sec - start_time.tv_sec + 1.0E-6F * (end_time.tv_usec - start_time.tv_usec);

19 Profiling Gather statistics about your program’s execution  e.g., how much time did execution of a function take?  e.g., how many times was a particular function called?  e.g., how many times was a particular line of code executed?  e.g., which lines of code used the most time? Most compilers come with profilers  e.g., pixie and gprof Gprof (GNU Performance Profiler)  gcc –Wall –ansi –pedantic –pg –o mymath.o mymath.c

20 Gprof (GNU Performance Profiler) Instrumenting the code  gcc –Wall –ansi –pedantic –pg –o mymath.o mymath.c Running the code (e.g., testmath )  Produces output file gmon.out containing statistics Printing a human-readable report from gmon.out  gprof testmath > gprofreport

21 Two Main Outputs of Gprof Call graph profile: detailed information per function  Which functions called it, and how much time was consumed?  Which functions it calls, how many times, and for how long?  We won’t look at this output in any detail… Flat profile: one line per function  name: name of the function  %time: percentage of time spent executing this function  cumulative seconds: [skipping, as this isn’t all that useful]  self seconds: time spent executing this function  calls: number of times function was called (excluding recursive)  self ms/call: average time per execution (excluding descendents)  total ms/call: average time per execution (including descendents)

22 Call Graph Output called/total parents index %time self descendents called+self name index called/total children [1] internal_mcount [1] /3 atexit [35] [2] _start [2] /1 main [3] /3 atexit [35] /1 _start [2] [3] main [3] /1 getBestMove [4] /1 clock [20] / GameState_expandMove [6] /1 exit [33] /1 Move_read [36] /1 GameState_new [37] / GameState_getStatus [31] / GameState_applyDeltas [25] /1 GameState_write [44] /1 sscanf [54] / GameState_getPlayer [30] /1 _fprintf [58] /1 Move_write [59] /3 GameState_playerToStr [63] /2 strcmp [66] /1 GameState_playerFromStr [68] /1 Move_isValid [69] /1 GameState_getSearchDepth [67] /1 main [3] [4] getBestMove [4] /6 minimax [5] / GameState_expandMove [6] / Delta_free [10] / GameState_genMoves [17] / Move_free [23] / GameState_applyDeltas [25] / GameState_unApplyDeltas [27] / GameState_getPlayer [30] minimax [5] /6 getBestMove [4] [5] minimax [5] / GameState_expandMove [6] / Delta_free [10] / GameState_genMoves [17] / Move_free [23] / GameState_applyDeltas [25] / GameState_unApplyDeltas [27] / GameState_getPlayer [30] / GameState_getStatus [31] / GameState_getValue [32] minimax [5] / main [3] / getBestMove [4] / minimax [5] [6] GameState_expandMove [6] / calloc [7] / rem [28] / Move_read [36] / GameState_new [37] / GameState_genMoves [17] / GameState_expandMove [6] [7] calloc [7] / malloc [8] / umul [18] / _memset [22] / udiv [29] / _findbuf [41] / calloc [7] [8] malloc [8] / _malloc_unlocked [13] / _mutex_unlock [14] / mutex_lock [15] Complex format at the beginning… let’s skip for now.

23 Flat Profile % cumulative self self total time seconds seconds calls ms/call ms/call name internal_mcount [1] _free_unlocked [12] _mcount (693) _return_zero [16] umul [18] GameState_expandMove [6] calloc [7] _mutex_unlock [14] mutex_lock [15] _memset [22] div [21] cleanfree [19] _malloc_unlocked[13] malloc [8] _smalloc[24] minimax [5] Delta_free [10] free [9] GameState_applyDeltas [25] realfree [26] GameState_unApplyDeltas [27] rem [28] udiv [29] GameState_getPlayer [30] GameState_getStatus [31] GameState_genMoves [17] Move_free [23] GameState_getValue [32] _ferror_unlocked [357] _realbufend [358] nvmatch [60] _doprnt [42] memchr [61] printf [43] _write [359] _xflsbuf [360] _memcpy [361] mul [62] ___errno [362] _fflush_u [363] GameState_playerToStr [63] _findbuf [41] Second part of profile looks like this; it’s the simple (i.e.,useful) part; corresponds to the “prof” tool

24 Overhead of Profiling % cumulative self self total time seconds seconds calls ms/call ms/call name internal_mcount _free_unlocked _mcount (693) _return_zero umul [18] GameState_expa calloc [7] _mutex_unlock mutex_lock _memset [22] div [21] cleanfree [19] _malloc_unlo malloc [8] _smalloc minimax [5] Delta_free [10] free [9] GameState_appl realfree [26] GameState_unAp rem [28] udiv [29] GameState_getPl GameState_getSt

25 Malloc/calloc/free/... % cumulative self self total time seconds seconds calls ms/call ms/call name internal_mcount [1] _free_unlocked [12] _mcount (693) _return_zero [16] umul [18] GameState_expandMove calloc [7] _mutex_unlock [14] mutex_lock [15] _memset [22] div [21] cleanfree [19] _malloc_unlocked [13] malloc [8] _smalloc[24] minimax [5] Delta_free [10] free [9] GameState_applyDeltas realfree [26] GameState_unApplyDeltas rem [28] udiv [29] GameState_getPlayer GameState_getStatus GameState_genMoves [17]

26 expandMove % cumulative self self total time seconds seconds calls ms/call ms/call name internal_mcount [1] _free_unlocked [12] _mcount (693) _return_zero [16] umul [18] GameState_expandMove calloc [7] _mutex_unlock [14] mutex_lock [15] _memset [22] div [21] cleanfree [19] _malloc_unlocked [13] malloc [8] _smalloc[24] minimax [5] Delta_free [10] free [9] GameState_applyDeltas realfree [26] May be worthwhile to optimize this routine

27 Don’t Even Think of Optimizing These % cumulative self self total time seconds seconds calls ms/call ms/call name internal_mcount [1] _free_unlocked [12] _mcount (693) _return_zero [16] umul [18] GameState_expandMove [6] calloc [7] _mutex_unlock [14] mutex_lock [15] _memset [22] div [21] cleanfree [19] _malloc_unlocked [13] malloc [8] _smalloc [24] minimax [5] Delta_free [10] free [9] GameState_applyDeltas [25] realfree [26] GameState_unApplyDeltas [27] rem [28] udiv [29] GameState_getPlayer [30] GameState_getStatus [31] GameState_genMoves [17] Move_free [23] GameState_getValue [32] _ferror_unlocked [357] _thr_main [367] GameState_playerToStr [63] strcmp [66] GameState_getSearchDepth [67] GameState_new [37] GameState_playerFromStr [68] GameState_write [44] Move_isValid [69] Move_read [36] Move_write [59] check_nlspath_env [46] clock [20] exit [33] getBestMove [4] getenv [47] main [3] mem_init [70] number [71] scanf [53]

28 Using a Profiler Test your code as you write it  It is very hard to debug a lot of code all at once  Isolate modules and test them independently  Design your tests to cover boundary conditions Instrument your code as you write it  Include asserts and verify data structure sanity often  Include debugging statements (e.g., #ifdef DEBUG and #endif)  You’ll be surprised what your program is really doing!!! Time and profile your code only when you are done  Don’t optimize code unless you have to (you almost never will)  Fixing your algorithm is almost always the solution  Otherwise, running optimizing compiler is usually enough

29 Summary Two valuable UNIX tools  Make: building large program in pieces  Gprof: profiling a program to see where the time goes How does gprof work?  Good question!  Essentially, by randomly sampling the code as it runs –And seeing what line is running, and what function its in  More on this for the last programming assignment of the course! Rest of this week  Exam on Thursday 10:00-10:50am in this room  Open books and open notes, but not open laptop/PDA  No precept on Wednesday/Thursday in honor of midterms