Merijn de Jonge IPA Spring Days 2005 April 1, 2005 Build-level Components

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Overview Introduction/motivation Build-level components Build-level composition Decoupling into build-level components Concluding remarks

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Overview

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Mozilla

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Mozilla Directory Structure One very large directory tree 26,000 files 2,500 subdirectories 240 file types One very large directory tree 26,000 files 2,500 subdirectories 240 file types

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Mozilla Directory References 1,521 directory references Avg. fan-in/fan-out 12 Max fan-in=100 Max fan-out=42 Median fan-in=5 Median fan-out=12 1,521 directory references Avg. fan-in/fan-out 12 Max fan-in=100 Max fan-out=42 Median fan-in=5 Median fan-out=12

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Build-level File (atomic entity) –Source –Documentation Directory tree (container) = source tree Build process –To build/install software –Driven by make/ANT/… Configuration process –To control build process –Driven by configure/configuration files/… Build-level interfaces –Build interface –Configuration interface

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Mozilla Build Level 1,350 Makefiles 40,000 LOC build instructions 16,000 LOC configuration Cyclic dependencies Two-phase build process Centralized build/configuration knowledge Code duplication Component implementations scattered 1,350 Makefiles 40,000 LOC build instructions 16,000 LOC configuration Cyclic dependencies Two-phase build process Centralized build/configuration knowledge Code duplication Component implementations scattered

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Wishes Independent deployment (of build-level artifacts) –No circular dependencies –Individual build processes –Individual configuration processes Automated composition –Standardized build interface –Standardized configuration interface –Standardized binding mechanism for build-level variability and dependencies

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Idea Apply CBSE principles to the build level Make build-level artefacts available for reuse by decomposing software systems into build-level components Make build-level component usable by assembling them into software systems using automated composition techniques

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Overview

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Component Definition (Szyperski) A software component is A unit of composition … with contractually specified interfaces … and explicit context dependencies only. It is subject to independent deployment … and to third-party composition.

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Standardized Build Interface Different build systems exist, e.g., –make, ANT, shell scripts, IDE Different software systems require different build actions, e.g., –make –make bootstrap, make, make install make clean make all make check make (un)install make dist make clean make all make check make (un)install make dist Standardized build interface to enable that software construction can be controlled via a standard set and sequence of build actions:

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Standardized Configuration Interface Different mechanisms exist to control software construction, e.g., –configuration files, configuration tools, Makefile editing Standardized configuration interface to enable uniform compile-time configuration: configure --help configure --prefix=/usr configure --with-aterm=/usr/lib configure --with-optimization=true configure --with-debug=false configure --help configure --prefix=/usr configure --with-aterm=/usr/lib configure --with-optimization=true configure --with-debug=false

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Explicit Context Dependencies Dependencies on build-level components are declared in configuration interfaces > configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR … > configure --help … --with-aterm=DIR use ATerm Library at DIR --with-sglr=DIR use SGLR Parser at DIR …

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Independent Deployment Build-level components are deployed as packages A package is a versioned release of a build-level component, e.g., firefox-1.0.tar.gz Packages are published on web/ftp sites

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Third-party Composition A configuration interface enables late-binding of dependencies Compositions are thus not predefined and can be defined by a third party > configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…= > configure --with-aterm=/usr/local/aterm --with-sglr=/usr/local/sglr --with-…=

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Overview

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Architectural Description Language Package Definition Language (PDL) –Domain-specific language designed specially for build- level component composition –Used for initial implementations –No explicit provides interfaces Koala –Re-use of existing composition language –Implementation is work in progress –Multiple provides interfaces –No version resolution

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Koala Concepts for Build-level Composition Interface definitions –To group build-level artefacts into types Explicit provides interfaces –To declare the type(s) of a component Diversity interfaces –To declare build-level variation points Module as atomic entity –To declare the implementation of a component

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Koala Definition for Build-level Component component Sglr { provides ISglr p; requires IATerm aterm; IToolbusLib libtb; IErrorSupport error_support; IPtSupport pt_support; contains module m url=" connects p = m; m = aterm; m = libtb; m = error_support; m = pt_support; } component Sglr { provides ISglr p; requires IATerm aterm; IToolbusLib libtb; IErrorSupport error_support; IPtSupport pt_support; contains module m url=" connects p = m; m = aterm; m = libtb; m = error_support; m = pt_support; }

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Mapping Koala to Build-level Concepts component Sglr { provides ISglr p; requires IATerm aterm; IToolbusLib libtb; IErrorSupport error_support; IPtSupport pt_support; contains module m url=" connects p = m; m = aterm; m = libtb; m = error_support; m = pt_support; } component Sglr { provides ISglr p; requires IATerm aterm; IToolbusLib libtb; IErrorSupport error_support; IPtSupport pt_support; contains module m url=" connects p = m; m = aterm; m = libtb; m = error_support; m = pt_support; } > configure --help … --with-aterm=DIR use ATerm component at DIR --with-libtb=DIR use LIBTB component at DIR --with-error_support=DIR use ERROR_SUPPORT component at DIR --with-pt_support=DIR use PT_SUPPORT component at DIR … > configure --help … --with-aterm=DIR use ATerm component at DIR --with-libtb=DIR use LIBTB component at DIR --with-error_support=DIR use ERROR_SUPPORT component at DIR --with-pt_support=DIR use PT_SUPPORT component at DIR … > curl | tar xfz - > configure --with-aterm= \ --with-libtb= \ --with-error_support= \ --with-pt_support= > configure --with-aterm= \ --with-libtb= \ --with-error_support= \ --with-pt_support=

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Koala source tree composition Checkout Implementation Parsing Realizatio n Normalization generic Component definition Package Component definition

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Unbound Diversity/Requires Interfaces Propagate upwards to top-level configuration interface r interface IConf { string foo=“some_value”; string bar; } interface IConf { string foo=“some_value”; string bar; } > configure --help --with-bar=VAL bind bar to VAL --with-r=DIR use R component at DIR > configure --help --with-bar=VAL bind bar to VAL --with-r=DIR use R component at DIR > configure --with-foo=“some_value” > configure --with-foo=“some_value” \ --with-bar=bar_binding() \ --with-r=r_binding() > configure --with-foo=“some_value” \ --with-bar=bar_binding() \ --with-r=r_binding() rr

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Online Package Base

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Overview

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Three Phases of Source Tree Decoupling 1.Source tree analysis 2.Source tree transformation 3.Online package base creation Performed successfully for graphviz ( )

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Source Tree Analysis Find components Find component references Fine tune –Add missing arrows –Remove cycles –Combine nodes with strong cohesion

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Create components Create component definitions Fine tune Source Tree Transformation sglr-3.13.tar.gz

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Decoupling Mozilla Cyclic dependencies Centralized build/configuration knowledge Cyclic dependencies Centralized build/configuration knowledge

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Dealing with Cyclic Dependencies 108 potential build-level components 4 cyclic-dependent clusters 37 non-cyclic components 108 potential build-level components 4 cyclic-dependent clusters 37 non-cyclic components

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Dealing with Cyclic Dependencies For each cluster create a composite component and a Koala component definition One provides interface for each sub- component On-demand refactoring to remove cyclic dependencies If all cyclic dependencies for a sub- component are removed, the component can be separated

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Dealing with centralized build/configuration Build knowledge –Duplicate build system for each component, or … –Generate (Automake) build process, or … –Dedicated component with build knowledge Configuration knowledge –Duplicate for each component, –Prune to remove irrelevant code

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Overview

April 1, 2005IPA Spring DaysMerijn de JongeBuild-level Components‹#› Concluding Remarks Reusable functionality is often not independently deployable due to build-level deficiencies CBSE principles help to decouple source trees into build-level components Composition of build-level components can be automated Introducing CBSE principles can be (partly) automated