Build Tools Software Engineering SS 2007

Agenda for today Build Tools 1. Motivation 2. Key Concepts 3. Tools Available 4. Presentation 5. Discussion Objectives - Use modern build systems for software Software Engineering, lecture #: Topic 2

Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise 5. Discussion Software Engineering, lecture #: Topic 3

Software complexity Software moved from a monolithic implementation to a highly dynamic and modular implementation Software grew in size and complexity Different languages Different stages of file transformation Many dependencies to keep track Portability of the software became a problem Building of the software also got complex Shell scripts got extremely fragile Set up the situation that would call for constructing a tool such as Make Comment: content good, but to much text. Use colors or figure Software Engineering, lecture #: Topic 4

A tool was needed Quality Assurance became a necessity Such that every build was reproducible on different platforms Portability between different platforms Let the tool master the environment, instead of the developer Incremental builds to speed up the development time Ensuring the build produces the same results as a full clean build Comment: Use free space to separate text Software Engineering, lecture #: Topic 5

Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise 5. Discussion Software Engineering, lecture #: Topic 6

File transformation The building of software consists of transforming files from one format to another hello_world.c  hello_world Must instruct the appropriate tool to perform the transformation cc –c –o hello_world.o hello_world.c Must check that all dependencies of the target have been also successfully transformed cc –o hello_world hello_world.o Comment: suggest a diagram for this and for the dependency tracking. Comment: Use figure to explain Software Engineering, lecture #: Topic 7

Dependency tracking Files may depend on other files For a successful transformation, the dependencies are required to be available and transformed Usually done by a time stamp on the file Before we compile hello_world, we must first check that hello_world.o is newer than our source file, hello_world.c If it is not, we must recompile hello_world.o Comment: Use figure to explain Software Engineering, lecture #: Topic 8

External instrumentation It is the purpose of the build tool to instruct external tools in the appropriate order to perform the file transformations However, build tools may not know all the possible tools it may be used with Most build tools use either a shell scripting language or allow for a plug-in framework Tools contain some predefined rules for common tasks, like compilation Ant and Java Make and Fortran, C Software Engineering, lecture #: Topic 9

Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise 5. Discussion Software Engineering, lecture #: Topic 10

General idea Allow the developer to define targets, dependencies, and rules. Dependencies may either be files, or other targets When a target is performed, then the dependencies are checked to ensure a correct transformation Some tools support incremental builds If the dependencies have not changed since the last transformation, then we do not need to re-transform them Software Engineering, lecture #: Topic 11

Make Created by Stuart Feldman in 1977 at Bell Labs 2003, he received the ACM Software System Award for the Make tool Arguably, the single most important step in the direction of modern build environments Still the de-facto standard for software builds on Unix systems Easy to compile and install Unix based applications make make install A bit of history of Make Comment: Move 14 and 15 after this slide Software Engineering, lecture #: Topic 12

Modern versions BSD Make Derived from Adam de Boor’s work on a version of make capable of building targets in parallel. Includes conditionals and iterative loops applied to parsing stage. Generation of targets at runtime GNU Make Used in GNU/Linux installations Allows for pattern-matching in dependency graphs and build targets Heavy use of external macros Just to express that there isn’t just one Make in the world. Software Engineering, lecture #: Topic 13

Make example helloworld: helloworld.o $(CC) $(CPPFLAGS) $(LDFLAGS) –o $@ $< helloworld.o: helloword.c $(CC) –c $(CPPFLAGS) $(CFLAGS) –o $@ $< .PHONY: clean clean: $(RM) helloworld helloworld.o Don’t Forget about the Tab! All actions must be tabbed (\t) ! Make file format for this. Target: dependency command Where $@ represents the text of the target And $< represents the text of the dependency $(CC) is the default c compiler $(CPPFLAGS) is the c pre processor flags $(CFLAGS) is the c compile flags $(LDFLAGS) are the link loader flags for libraries $(RM) is the standard rm for make (overrides any user based aliases) .PHONY declares clean to not produce any files. Software Engineering, lecture #: Topic 14

Extensions of make Make files also have machine dependencies Compiler options, alternate command names This became extremely hard for developers to support various platforms IMake Generates Make files from templates and macro functions GNU Automake Successor of IMake Generates Make files from a higher level language Dynamic dependency tracking Automake also takes care of automatically generating the dependency information, so that when a source file is modified, the next invocation of the make command will know which source files need to be recompiled. If the compiler allows it, Automake tries to make the dependency system dynamic: whenever a source file is compiled, that file's dependencies are updated by asking the compiler to regenerate the file's dependency list. In other words, dependency tracking is a side effect of the compilation process. This attempts to avoid the problem with some static dependency systems, where the dependencies are detected only once when the programmer starts working on the project. In such a case, if a source file gains a new dependency (e.g., if the programmer adds a new #include directive in a C source file), then a discrepancy is introduced between the real dependencies and those that are used by the compilation system. The programmer should then regenerate the dependencies, but runs the risk of forgetting to do so. Software Engineering, lecture #: Topic 15

Apache ant Similar as Make, but Java language specific Uses an XML file for build description Primary goal was to solve Make’s portability problem Make targets depend on a Unix shell Ant’s built-in functionality will usually behave identical on all platforms Transition from Make to Ant. Comment: Use free space to separate text Software Engineering, lecture #: Topic 16

Apache ant Created by James Duncan Davidson from Sun Microsystems Officially released as a stand alone tool July 2000 Easy to integrate JUnit tests and other external processes However, Ant has limitations XML format does not allow for complex build tasks, requires a Java plug-in that encodes the task Limited fault handling rules Developer must make explicit incremental builds History and shortcomings of Ant Software Engineering, lecture #: Topic 17

Ant example <project default=“compile”> <target name=“compile”> <mkdir dir=“build” /> <javac srcdir=“source” destdir=“build”/> </target> <target name=“package” depends=“compile”> <jar jarfile=“helloworld.jar” basedir=“build”/> <target name = “clean”> <delete dir=“build”/> </project> A small ant example for our hello world class. We have to use the directories for simplicity. i.e. source/hello_world.java The rest should be pretty straight forward. Should make some pretty colors. Comment: Use colors to highlight keywords or most interesting elements Software Engineering, lecture #: Topic 18

Apache maven Dependencies are hard to maintain in Java programs Apache ant could not address the problem Maven was created with dependency control as the prominent feature Uses concept of a Project Object Model Description of software project and external dependencies External dependencies not available, will be downloaded automatically Transition from Ant to Maven Software Engineering, lecture #: Topic 19

Maven example Maven 2 will be recommended in this course A simple hello world example mvn archetype:create \ -DgroupId=ch.ethz.HelloWorld \ -DartifactId=hello-world And Maven creates a skeleton for our project Project Object Model file Main source directory Test source directory Adheres to best practices A simple Maven example Comment: Maven commands with a different font Software Engineering, lecture #: Topic 20

Extensions of Apache maven Maven has a rich plug-in framework Default installation includes many powerful plug-ins Build statistics exported to a website Integration with continuous testing Deployment of web applications through .war files SCM integration for deployment of systems Software metrics TODO list generation Export to Netbeans or Eclipse And many, many more The TODO list generation comes from the @todo or TODO that programmer’s often put in source code I thought this was a bit of a clever one. Software Engineering, lecture #: Topic 21

Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise 5. Discussion Software Engineering, lecture #: Topic 22

Ant exercise Still the de facto for industrial builds After the initial set up of Ant we will Set some standard project properties Configure the appropriate class path Write the targets to make and deploy our system Produce javadoc from our source code This will be the first part to the exercises. First we have them do the build system in Ant, then in Maven. This will give them the basic required for the course, and allow them to see the differences between the two systems. Software Engineering, lecture #: Topic 23

Maven exercise For this course Maven is a good choice building our project From the initial setup of Maven we will Set some standard tags in the POM file Add our dependencies to our project Make and deploy our project And if time permits, use our maven build system within the Netbeans platform We suggest to use maven for the build system of their projects. Start by creating a new project, moving their source code into the project, and how to configure it for deployment and external dependencies. See the included file (CVS) on the full presentation. Software Engineering, lecture #: Topic 24

Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise 5. Discussion Software Engineering, lecture #: Topic 25

Concluding remarks Necessary to include build tools in Configuration Management Automake 1.4 does not produce the same results as 1.9 Change of machine dependencies will usually not trigger a re-transformation May lead to deployment of software with the wrong compile flags Change of machine dependencies may also produce different results Compilation on Java 1.5 may produce different run-time behaviors than Java 1.4 One important point of build tools is that you have a build process encoded in say Make, but if you change the compiler, you may still not have the same resulting output. Various version of virtual machines depend on which you use, may produce different results and run-time behaviors . A CPP flag change will not trigger a re-compile in most versions of Make. Some versions however, do take this into account. Software Engineering, lecture #: Topic 26

References Make http://www.gnu.org/ http://www.bsd.org Ant http://ant.apache.org Maven http://maven.apache.org http://codehaus.org Software Engineering, lecture #: Topic 27