1. Build Tools Software Engineering SS 2007

2. Agenda for today Build Tools 1. Motivation 2. Key Concepts
3. Tools Available
4. Presentation
5. Discussion
Objectives
- Use modern build systems for software
Software Engineering, lecture #: Topic 2

3. Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise
5. Discussion
Software Engineering, lecture #: Topic 3

4. Software complexity
Software moved from a monolithic implementation to a highly dynamic and modular implementation
Software grew in size and complexity
Different languages
Different stages of file transformation
Many dependencies to keep track
Portability of the software became a problem
Building of the software also got complex
Shell scripts got extremely fragile
Set up the situation that would call for constructing a tool such as Make
Comment: content good, but to much text. Use colors or figure
Software Engineering, lecture #: Topic 4

5. A tool was needed Quality Assurance became a necessity
Such that every build was reproducible on different platforms
Portability between different platforms
Let the tool master the environment, instead of the developer
Incremental builds to speed up the development time
Ensuring the build produces the same results as a full clean build
Comment: Use free space to separate text
Software Engineering, lecture #: Topic 5

6. Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise
5. Discussion
Software Engineering, lecture #: Topic 6

7. File transformation
The building of software consists of transforming files from one format to another
hello_world.c  hello_world
Must instruct the appropriate tool to perform the transformation
cc –c –o hello_world.o hello_world.c
Must check that all dependencies of the target have been also successfully transformed
cc –o hello_world hello_world.o
Comment: suggest a diagram for this and for the dependency tracking.
Comment: Use figure to explain
Software Engineering, lecture #: Topic 7

8. Dependency tracking Files may depend on other files
For a successful transformation, the dependencies are required to be available and transformed
Usually done by a time stamp on the file
Before we compile hello_world, we must first check that hello_world.o is newer than our source file, hello_world.c
If it is not, we must recompile hello_world.o
Comment: Use figure to explain
Software Engineering, lecture #: Topic 8

9. External instrumentation
It is the purpose of the build tool to instruct external tools in the appropriate order to perform the file transformations
However, build tools may not know all the possible tools it may be used with
Most build tools use either a shell scripting language or allow for a plug-in framework
Tools contain some predefined rules for common tasks, like compilation
Ant and Java
Make and Fortran, C
Software Engineering, lecture #: Topic 9

10. Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise
5. Discussion
Software Engineering, lecture #: Topic 10

11. General idea
Allow the developer to define targets, dependencies, and rules.
Dependencies may either be files, or other targets
When a target is performed, then the dependencies are checked to ensure a correct transformation
Some tools support incremental builds
If the dependencies have not changed since the last transformation, then we do not need to re-transform them
Software Engineering, lecture #: Topic 11

12. Make Created by Stuart Feldman in 1977 at Bell Labs
2003, he received the ACM Software System Award for the Make tool
Arguably, the single most important step in the direction of modern build environments
Still the de-facto standard for software builds on Unix systems
Easy to compile and install Unix based applications
make
make install
A bit of history of Make
Comment: Move 14 and 15 after this slide
Software Engineering, lecture #: Topic 12

13. Modern versions BSD Make
Derived from Adam de Boor’s work on a version of make capable of building targets in parallel.
Includes conditionals and iterative loops applied to parsing stage.
Generation of targets at runtime
GNU Make
Used in GNU/Linux installations
Allows for pattern-matching in dependency graphs and build targets
Heavy use of external macros
Just to express that there isn’t just one Make in the world.
Software Engineering, lecture #: Topic 13

14. Make example helloworld: helloworld.o
$(CC) $(CPPFLAGS) $(LDFLAGS) –o $<
helloworld.o: helloword.c
$(CC) –c $(CPPFLAGS) $(CFLAGS) –o $<
.PHONY: clean
clean:
$(RM) helloworld helloworld.o
Don’t Forget about the Tab!
All actions must be tabbed (\t) !
Make file format for this.
Target: dependency
command
Where represents the text of the target
And $< represents the text of the dependency
$(CC) is the default c compiler
$(CPPFLAGS) is the c pre processor flags
$(CFLAGS) is the c compile flags
$(LDFLAGS) are the link loader flags for libraries
$(RM) is the standard rm for make (overrides any user based aliases)
.PHONY declares clean to not produce any files.
Software Engineering, lecture #: Topic 14

15. Extensions of make Make files also have machine dependencies
Compiler options, alternate command names
This became extremely hard for developers to support various platforms
IMake
Generates Make files from templates and macro functions
GNU Automake
Successor of IMake
Generates Make files from a higher level language
Dynamic dependency tracking
Automake also takes care of automatically generating the dependency information, so that when a source file is modified, the next invocation of the make command will know which source files need to be recompiled. If the compiler allows it, Automake tries to make the dependency system dynamic: whenever a source file is compiled, that file's dependencies are updated by asking the compiler to regenerate the file's dependency list. In other words, dependency tracking is a side effect of the compilation process.
This attempts to avoid the problem with some static dependency systems, where the dependencies are detected only once when the programmer starts working on the project. In such a case, if a source file gains a new dependency (e.g., if the programmer adds a new #include directive in a C source file), then a discrepancy is introduced between the real dependencies and those that are used by the compilation system. The programmer should then regenerate the dependencies, but runs the risk of forgetting to do so.
Software Engineering, lecture #: Topic 15

16. Apache ant Similar as Make, but Java language specific
Uses an XML file for build description
Primary goal was to solve Make’s portability problem
Make targets depend on a Unix shell
Ant’s built-in functionality will usually behave identical on all platforms
Transition from Make to Ant.
Comment: Use free space to separate text
Software Engineering, lecture #: Topic 16

17. Apache ant Created by James Duncan Davidson from Sun Microsystems
Officially released as a stand alone tool July 2000
Easy to integrate JUnit tests and other external processes
However, Ant has limitations
XML format does not allow for complex build tasks, requires a Java plug-in that encodes the task
Limited fault handling rules
Developer must make explicit incremental builds
History and shortcomings of Ant
Software Engineering, lecture #: Topic 17

18. Ant example <project default=“compile”>
<target name=“compile”>
<mkdir dir=“build” />
<javac srcdir=“source” destdir=“build”/>
</target>
<target name=“package” depends=“compile”>
<jar jarfile=“helloworld.jar” basedir=“build”/>
<target name = “clean”>
<delete dir=“build”/>
</project>
A small ant example for our hello world class.
We have to use the directories for simplicity.
i.e.
source/hello_world.java
The rest should be pretty straight forward.
Should make some pretty colors.
Comment: Use colors to highlight keywords or most interesting elements
Software Engineering, lecture #: Topic 18

19. Apache maven Dependencies are hard to maintain in Java programs
Apache ant could not address the problem
Maven was created with dependency control as the prominent feature
Uses concept of a Project Object Model
Description of software project and external dependencies
External dependencies not available, will be downloaded automatically
Transition from Ant to Maven
Software Engineering, lecture #: Topic 19

20. Maven example Maven 2 will be recommended in this course
A simple hello world example
mvn archetype:create \
-DgroupId=ch.ethz.HelloWorld \
-DartifactId=hello-world
And Maven creates a skeleton for our project
Project Object Model file
Main source directory
Test source directory
Adheres to best practices
A simple Maven example
Comment: Maven commands with a different font
Software Engineering, lecture #: Topic 20

21. Extensions of Apache maven
Maven has a rich plug-in framework
Default installation includes many powerful plug-ins
Build statistics exported to a website
Integration with continuous testing
Deployment of web applications through .war files
SCM integration for deployment of systems
Software metrics
TODO list generation
Export to Netbeans or Eclipse
And many, many more
The TODO list generation comes from the
@todo or TODO that programmer’s often put in source code
I thought this was a bit of a clever one.
Software Engineering, lecture #: Topic 21

22. Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise
5. Discussion
Software Engineering, lecture #: Topic 22

23. Ant exercise Still the de facto for industrial builds
After the initial set up of Ant we will
Set some standard project properties
Configure the appropriate class path
Write the targets to make and deploy our system
Produce javadoc from our source code
This will be the first part to the exercises. First we have them do the build system in Ant, then in Maven. This will give them the basic required for the course, and allow them to see the differences between the two systems.
Software Engineering, lecture #: Topic 23

24. Maven exercise
For this course Maven is a good choice building our project
From the initial setup of Maven we will
Set some standard tags in the POM file
Add our dependencies to our project
Make and deploy our project
And if time permits, use our maven build system within the Netbeans platform
We suggest to use maven for the build system of their projects.
Start by creating a new project, moving their source code into the project, and how to configure it for deployment and external dependencies.
See the included file (CVS) on the full presentation.
Software Engineering, lecture #: Topic 24

25. Build tools 1. Motivation 2. Key Concepts 3. Tools 4. Exercise
5. Discussion
Software Engineering, lecture #: Topic 25

26. Concluding remarks
Necessary to include build tools in Configuration Management
Automake 1.4 does not produce the same results as 1.9
Change of machine dependencies will usually not trigger a re-transformation
May lead to deployment of software with the wrong compile flags
Change of machine dependencies may also produce different results
Compilation on Java 1.5 may produce different run-time behaviors than Java 1.4
One important point of build tools is that you have a build process encoded in say Make, but if you change the compiler, you may still not have the same resulting output.
Various version of virtual machines depend on which you use, may produce different results and run-time behaviors .
A CPP flag change will not trigger a re-compile in most versions of Make. Some versions however, do take this into account.
Software Engineering, lecture #: Topic 26

27. References Make http://www.gnu.org/ http://www.bsd.org Ant
Maven
Software Engineering, lecture #: Topic 27