1. Build Systems Margolin Alex 2/4/09 Includes part of ”Using GNU Autotools” by Alexandre Duret-Lutz

2. Outline Intro to Build systems Make GNU Autotools CMake Scons Ant

3. Purpose of build systems Main goals: o Compile source sode into a package o Deploy package at the target machine Secondary goals: o Automatic testing o Customization on target platform o Automatic documentation o Standartization o Cross-compilation

4. Goals vs. Real life The main deal is compiling the sources and preparing the package Many packages are pre-compiled and ready to install using other software (apt, rpm, etc.)‏ Automatic testing and docmentation are achived by running external, dedicated, tools Complete portability is hard to achieve...

5. Perspective In the beginning there was make (BSD Unix)‏ o GNU make (widespread)‏ o Nmake (windows)‏ Same idea, different syntax: cook, jam Then the Build and Configuration systems Bootstrapping, or: Who builds the builder?

7. Make Basic building utility Building software since 1977 Works by targets/dependencies Portable*: BSD, GNU, Windows (nmake)‏ o Only basic makefiles work on all platforms o Most makefiles work in an OS-dependant fashion On its own – impractical for a serious project

8. Example Makefile CC=g++ CFLAGS=-c -Wall LDFLAGS= SOURCES=main.cpp hello.cpp factorial.cpp OBJECTS=$(SOURCES:.cpp=.o)‏ EXECUTABLE=hello all: $(SOURCES) $(EXECUTABLE)‏ $(EXECUTABLE): $(OBJECTS)‏ $(CC) $(LDFLAGS) $(OBJECTS) -o $@.cpp.o: $(CC) $(CFLAGS) $< -o $@

9. So... Why not Make? Not that portable (depends on shell and FS)‏ Scalability issues (large projects/makefiles)‏ User-friendly? (not too human-readable)‏ Reliability (uses timestamps and not content)‏ Implicit dependency engine (no such thing)‏ Hard syntax (whitespaces matter, no and/or)‏ No debugging (typos silently ignored...)‏

10. GNU Build System (GBS) - Autotools

11. Why Autotools? Consider C functions... o that do not exist everywhere (e.g., strtod()) o that have different names (e.g., strchr() vs. index()) o that have varying prototypes (e.g., int setpgrp(void); vs. int setpgrp(int, int);) o that can behave differently (e.g., malloc(0);) o that might require other libraries (is pow() in libm.so or in libc.so?) o that can be defined in different headers (string.h vs. strings.h vs. memory.h) How should a package deal with those?

12. Why Autotools? Possible solutions: o Slice the code with lots of #if/#else o Create substitution macros o Create substitution functions strdup.c (from the GNU C library): char ∗ s t r d u p ( const char ∗ s ) { s i z e t len = s t r l e n ( s ) + 1; v o i d ∗ new = m a l l o c ( l e n ) ; i f ( new == NULL) r e t u r n NULL ; r e t u r n ( char ∗ ) memcpy ( new, s, l e n ) ; } Excerpt of coreutils-5.2.1’s system.h: #i f ! HAVE FSEEKO && ! d e f i n e d f s e e k o # d e f i n e f s e e k o ( s, o, w) ( ( o ) == ( long ) ( o ) \ ? f s e e k ( s, o, w) \ : ( e r r n o = EOVERFLOW, −1)) #e n d i f

13. Why Autotools? The answer (by autotools): o Scan the source code for potential problems o Scan target to get exact configuration (regarding these problems) o Build the source code for local configuration

14. Autotools structure Autotools (GBS) is composed of: o autoscan o autoconf o autoheader o automake o aclocal o libtool o pkg-config o gettext o make...

15. Autotools Demo In 90% of the cases, as a user: o Download the software o Unpack it (unzip, untar, etc.)‏ o $./configure o $ make o $ make install What about the other 10%?

16. Autotools Demo printdate.h #ifndef PRINTDATE_H #define PRINTDATE_H #include void printdate(); #endif yesterday.c #include "yesterday.h" time_t yesterday() { time_t today = time(NULL); return today – (time_t)(24*60*60); } yesterday.h #ifndef YESTERDAY_H #define YESTERDAY_H #include time_t yesterday(); #endif main.c #include "yesterday.h" #include "printdate.h" int main() { time_t y = yesterday(); printdate(y); return 0; } printdate.c #include "printdate.h" void printdate(time_t d) { struct tm* t; char buf[100]; t=localtime(&d); strftime(buf, sizeof(buf), "Yesterday was: %Y- %m-%d\n", t); printf(buf); }

17. configure.ac $ touch configure.ac & autoscan Found 2 portability issues: o Strftime()‏ o Struct tm $ cp configure.scan configure.ac configure.ac: warning: missing AC_FUNC_STRFTIME wanted by: printdate.c:9 configure.ac: warning: missing AC_PREREQ wanted by: autoscan configure.ac: warning: missing AC_PROG_CC wanted by: main.c configure.ac: warning: missing AC_STRUCT_TM wanted by: printdate.c:4

18. configure.ac # -*- Autoconf -*- # Process this file with autoconf to produce a configure script. AC_PREREQ(2.59)‏ AC_INIT(yesterday, 2.3, system@cs)‏ AM_INIT_AUTOMAKE AC_CONFIG_SRCDIR([main.c])‏ AC_CONFIG_HEADER([config.h])‏ # Checks for programs. AC_PROG_CC # Checks for libraries. # Checks for header files. # Checks for typedefs, structures, and compiler characteristics. AC_STRUCT_TM # Checks for library functions. AC_FUNC_STRFTIME AC_CONFIG_FILES([Makefile])‏ AC_OUTPUT

19. M4 and aclocal M4- script language for text-maniplation macros: POSIX standard, today mostly used in autotools aclocal builds aclocal.m4 from configure.ac, later used for creating "configure" with these m4-s o $ aclocal define(`H2_COUNT', 0)dnl define(`H2', `define(`H2_COUNT', incr(H2_COUNT))'dnl ` H2_COUNT. $1 ')dnl dnl H2(First Section) H2(Second Section) H2(Conclusion)

20. Autoheader Autoheader scans configure.ac and creates config.h.in Used for adding parameters relevent for c/c++ header file, such as special "#define"-s The result is later used as an include by the "configure" script o $autoheader

21. Makefile.am Create a short Makefile.am file: Surves as input to the Automake program Causes the ”configure” script (later on) to create a Makefile for Make bin_PROGRAMS = yesterday yesterday_SOURCES = main.c yesterday.c yesterday.h printdate.c printdate.h

22. Automake Takes Makefile.am, creates Makefile.in Requires a list of files in order to run: o $ touch NEWS README AUTHORS ChangeLog o $ Automake -a configure.ac: installing `./install-sh' configure.ac: installing `./missing' Makefile.am: installing `./INSTALL' Makefile.am: installing `./COPYING' Makefile.am: installing `./depcomp'

23. Autoconf Creates the ”configure” script, using configure.ac(.in)‏ This script will customize the code on the target The configure script uses other generated files: o Makefile.in o config.h.in o... $ autoconf

24. ./configure checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... gawk checking whether make sets $(MAKE)... yes checking for gcc... gcc checking for C compiler default output file name... a.exe checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables....exe checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ANSI C... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking whether struct tm is in sys/time.h or time.h... time.h checking for strftime... yes configure: creating./config.status config.status: creating Makefile config.status: creating config.h config.status: executing depfiles commands

25. Deliverables Now we have a Makefile! We can create a deliverabe package: o $ make dist-gzip o Creates yesterday-2.3.tar.gz (using AC_INIT)‏ What does a user do with this package? o Unpack o./configure o Make o Make install

26. A few words about GCC DStands for "GNU Compiler Collection", an official part of the GBS (and used by many others) Started at 1985, to become a key GNU component The Makefile is mostly, in fact, a list of directives to compile and link the code, to build the software GCC builds the software according to parameters, specified command-line arguments. For example: o $ gcc -O3 test.c (optimiztion of run time) o $ gcc -Wall -g test.c (warnings and debug info) o $ gcc -La -Ib test.c (link folder a and include bs) o $ gcc -fPIC -DSHARED_OBJECT -c test.c (force position-independent code, i.e. shared object)

27. pkg-config How do we know what flags to compile with? How do we check if we need new flags? This is what "pkg-config" is for: o gcc test.c `pkg-config --libs --cflags glib-2.0`

28. libtool Intended for building shared libraries We need a portable solution o unix has.so (elf), windows has.dll (PE)... Libtool does just that.

30. CMake Short for ”cross-platform make” Takes ”CmakeLists.txt” Produces Makefile (unix), Project/workspace for visual c++ (windows), and other formats Generally faster and easier to use (syntacticly) Building creates a cache, presentable by GUI Integration with other tools o Ctest, CPack Started at 2001 (at kitware), KDE4 moved by 2oo6

31. Cmake Demo Basic usage for our yesterday-2.3: CMake can use variables to effect the build: PROJECT(yesterday-2.3)‏ SET(SRCS yesterday.c main.c printdate.c)‏ ADD_EXECUTABLE(myproject ${SRCS})‏ $ cmake –DWITH_USB=ON /path/to/source/files PROJECT(myproject)‏ OPTION(WITH_USB "Include our USB component" OFF)‏ SET(SRCS file1.c file2.c file3.c)‏ ADD_EXECUTABLE(myproject ${SRCS})‏ IF(WITH_USB)‏ ADD_DIRECTORY(usblib)‏ TARGET_LINK_LIBRARIES(myproject usblib)‏ ENDIF(WITH_USB)‏

32. CTest, CPack Supplements for CMake CTest makes sure the build passes predefined tests o Integrated with DART to create "Dashboards" o supports major coverage tools o supports memory testing (valgrind, purify) CPack creates custom packages for builds o supports popular archive types o supports package formats (.deb/.rpm/.exe)

34. Scons intro Started at 2000, as ScCons Based on a script language – python Supports C/C++,java,latex... (easy to expand)‏ Own regexp-based dependency scanner MD5 signatures instead of timestamps

35. Scons Demo Remember yesterday-2.3? Python knowledge not required o - but is certainly a plus! SConstruct env = Envronment()‏ env.Program(target='yesterday', source=['yesterday.c','main.c',printdate.c])‏

36. Elaborate SCONS Demo SConstruct: env = Environment(LIBPATH = ['#libA', '#libB'])‏ Export('env')‏ SConscript('libA/SConscript')‏ SConscript('libB/SConscript')‏ SConscript('Main/SConscript')‏ libA/SConscript: Import('env')‏ env = Environment(CCFLAGS = '-g')‏ env.Library('a', Split('a1.c a2.c a3.c'))‏ libB/SConscript: Import('env')‏ env.Library('b', Split('b1.c b2.c b3.c'))‏ Main/SConscript: Import('env')‏ e = env.Copy(LIBS = ['a', 'b'])‏ e.Program('foo', Split('m1.c m2.c m3.c'))‏

37. Adding new builders bld = Builder(action = 'pdftex $TARGET' suffix = '.pdf', src_suffix = '.tex')‏ env = Environment()‏ env.Append(BUILDERS = {'PDFBuilder' : bld})‏ env.PDFBuilder(target = 'foo.pdf', source = 'foo.tex')‏ env.Program(target = 'bar', source = 'bar.c')‏ One can add new builders: It is even possible to write a custom function computing the dependencies for files...

38. Timing experiment An experiment was preformed, to test speed: o Building a complete project o Rebuilding without any changes o Rebuilding a library without any changes

40. Ant Analogous to Make – for Java projects Uses an XML format for its description files Like java, relays on native implementations for portability (, not rm -rf a)‏ Consider a simple ”hello world” demo: public class HelloWorld { public static void main ( final String[] args ) { System.out.println ( "Hello World." ) ; }

41. Ant Demo

42. More systems Many systems are left unmentioned: o Jam, Cook (Alternatives to Make)‏ o Rake, Rant (written in Ruby)‏ o Maven (newer Java builder)‏ o MinGW (Minimalist GNU for Windows) o qmake (not to confuse with Qmake...)‏