NA-MIC, 2008 June Workshop, IHK Akademie Westerham Engineering Methodology Sebastien BARRE, Ph.D. - Kitware, Inc.

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham2 Outline Software Process, Extreme Programming The components of the NA-MIC kit software engineering tools: CMake CPack CTest CDash KWStyle June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham2

NA-MIC, 2008 June Workshop, IHK Akademie Westerham Software Process, Extreme Programming Sebastien BARRE, Ph.D. - Kitware, Inc.

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham4 Challenges Software is larger and more complex than ever Complex software requires teams; multi-site, multi-disciplinary, multi-layered, academic and commercial Complex software demands long-term maintenance Testing is vital to insure quality Documentation needs are greater Managing change to the system is critical to capture bug fixes Abundance of computer configurations (hard/software, OS) means addressing cross-platform issues June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham4

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham5 Our solution A methodology developed over several years in support of large-scale open-source projects A methodology based on the principles of agile-programming or extreme-programming (late 90´s) Ultimate Goal: A production software release every day June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham5

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham6 Extreme Programming The idea is that the standard development tasks: requirements generation, software design, managing source code versions and updates, configuring projects for specific platforms, compilation and linking, testing the code at run time, verifying the validity of output, documenting the code, tracking and repairing bugs are performed continuously rather than in a waterfall fashion.

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham7 Extreme Programming (cont.)

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham8 Heart and Soul of the Process Testing is the mantra Every night Every time changes are made Every time new computing platforms are introduced “If is isn’t tested, it doesn’t work” The results are posted to a web page for everyone to monitor All developers have the right and encouragement to repair errors A feedback loop is created between the community and the automated processes that measure software quality

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham9 Heart and Soul of the Process (cont.) CVS/ SVN Results posted on web (the dashboard) Source code repository Testing client compiles source code, runs tests Developers review results Developers check-in code

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham10 Extreme Programming (cont.) The community owns the code Although the identity of the original author is kept, other developers are free to correct defects and enhance each other's code In the end, all of the software should appear as though one author wrote it

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham11 Extreme Programming (cont.) Release early, release often Although developers are tempted to keep their code under wraps until it is perfect, the process encourages them to release their code as soon as it passes some minimum tests The longer the code is visible to the community, the better integrated it will be

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham12 Extreme Programming (cont.) Continuous integration There is no scheduled porting to computer platforms All new software builds on supported platforms every evening

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham13 Extreme Programming (cont.) All developers agree to keep the software defect free Although everyone is encouraged to submit their code early, the code must compile and pass tests nightly A continuous build process sends s to developers who check in code that does not compile More effectively, the community enforces the commitment though peer pressure

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham14 Software Process Tools Communication and Documentation Revision Control Build Management Testing Release process

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham15 Communication and Documentation Mailing lists Wiki Online documentation (Doxygen) Issue/Bug Tracker IM, TCON, VCON

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham16 Mailing Lists Mailman list manager Two lists per project developer – discussion about implementation details user – discussion by users about how to use it Provides a searchable archive Dynamic, quick exchange of information

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham17 Wiki Powerful collaborative websites or intranets Easy to setup Easy to edit Don´t forget to backup

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham18 Doxygen Literate programming concepts combine source and documentation Run every night Simple markup for C++/C code /** *... text... */ /*! *... text... */ /*! \brief Brief description. * Brief description continued. * Detailed description starts here. */

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham19 Doxygen (cont.) From vtkActor.h: // Description: // This causes the actor to be rendered. It in turn will render the actor's // property, texture map and then mapper. If a property hasn't been // assigned, then the actor will create one automatically. Note that a side // effect of this method is that the pipeline will be updated. virtual void Render(vtkRenderer *, vtkMapper *) {}

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham20 Issue Tracker Web based issue tracker is required to keep track of bugs and fixes Tracker of choice: Mantis Requires periodic “triage” (usually before a major release)

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham21 Revision Control Projects evolve over time Who changed what? When? Why? SCM tools like CVS or SVN maintain a central repository of source code in which developers retrieve, modify and commit files and changes to files. Hello, World! sample.html Simple Example Hello, World! sample.html

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham22 Revision Control (cont.) Commit Date/time of change Log message describing change Person making the change Versions before and after change Commits can conflict, but the repository will not lock Commits can be referenced in bug trackers

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham23 Revision Control (cont.) Traditionally, history is always a tree Each development path is a branch trunk release-1_2 release-1_0 = Version/Revision/State = Commit (new state and pointer to old) = Series of Commits

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham24 Revision Control (cont.) Repository Working Tree = flow of changes W.T. Repo Repository W.T. Repo W.T. Repo W.T. Repo W.T. Repo

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham25 Build Management Often overlooked CMake is the (Kitware) answer, more about it later…

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham26 Testing Smoke test does it compile does it run and not crash Regression testing If something fails track, find the change and fix it harder to implement CTest/Cdash/DART are used; more about it later...

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham27 Release Process One person freezes and maintains the release branch Two types of release Binary Source CPack – tool to create release packages; more about it later

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham28 Software Process Tools Summary CMake – cross platform building Dashboards, CTest, CDash, DART – continuous testing Mailman Mailing List – Developer communication Wiki – Store developer communication Doxygen – Online documentation Mantis Bug Tracker – keep track of bugs and feature requests Revision control – cvs, svn CPack – release management

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham29 Software Process Tools Summary

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham30 Conclusion A set of tools, a set of rules and practices that creates a feedback loop resulting in higher quality code But it does require some discipline Paying attention to the dashboard Picking enforcers Creating tests June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham30

NA-MIC, 2008 June Workshop, IHK Akademie Westerham CMake Sebastien BARRE, Ph.D. - Kitware, Inc.

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham Start of the Insight Segmentation and Registration Toolkit (ITK) funded by NLM as part of the Visible Human Project Kitware tasked with build environment VTK current build system autoconf on UNIX pcmaker on Windows Very specific to VTK and hard to maintain CMake born out of necessity 2006 KDE desktop adopted CMake Why CMake

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham33 CMake Features Single input format for all platforms Generate native build files Visual Studio 6/7/8/9 Nmake, borland make, unix make Xcode KDevelop Eclipse Platform inspection tool can search for Programs Libraries Include paths Determine hardware specifics like byte order

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham34 CMake Features (cont.) Linux, SunOS, HPUX, IRIX, Windows, Mac OSX, QNX, etc. Simple, easy to use Support complex, large build environments (KDE, ParaView, SecondLife) Do not require any software other than CMake to be installed. Need only a C++ compiler (same as ITK itself) Full cross platform install system Comes with a GUI layer for easy edition of input variables.

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham35 Windows GUI Unix GUI CMakeLists.txt MSVC Project.NET Solution Makefile User selects CMake Example Usage

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham36 CMake GUI Process Flow Configure Write CMakeCache.txt Did Cache Change? Yes No Generate Makefile or Project

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham37 A very simple project (see how easy it is!) Files needed: Hello.c CMakeLists.txt The CMakeLists.txt file would contain two lines: PROJECT(Hello) ADD_EXECUTABLE(Hello Hello.c)

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham38 Basic CMake Syntax COMMAND_NAME(space separated arguments) command can be upper or lower case SCREAM_COMMAND or quiet_command Variables SET(VAR value) Semi-colon separated lists SET(VAR “a b c”) - VAR holds one thing SET(VAR a;b;c) or SET(VAR a b c) – VAR holds three things LIST or FOREACH will get access to each thing

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham39 Flow control (IF) IF(VAR) code ENDIF(VAR) IF(NOT VAR) IF(VAR AND VAR2) IF(VAR OR VAR2) IF(VAR MATCHES regular_expression) IF(COMMAND command) IF(EXISTS file) IF(VAR LESS VAR2)

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham40 Flow control (FOREACH WHILE) foreach(F a b c) message(${F}) endforeach(F) while(VAR) message(${VAR}) set(VAR FALSE) endwhile(VAR)

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham41 FIND_PACKAGE Easy for smaller projects to use larger projects project(SampleProject) find_package(VTK REQUIRED) include(${VTK_USE_FILE}) set(SOURCES mySource1.cxx mySource2.cxx) add_executable(myExecutable ${SOURCES}) target_link_libraries(myExecutable vtkRendering)

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham42 CMake Advanced Features Shared library and DLL building Support code generation at build time (VTK wrappers, Qt’s moc) Create configured.h files Automatic dependency generation Parallel builds Out of source builds Command line support Create OSX library frameworks and application bundles Cross compiling Integrated testing system called Ctest Integrated packaging system called Cpack Easy integration with CDash and Dart dashboard servers And more...

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham43June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham43 Find more about CMake… Online documentation Wiki FAQ Mailing list Books Mastering Cmake

NA-MIC, 2008 June Workshop, IHK Akademie Westerham CPack Sebastien BARRE, Ph.D. - Kitware, Inc.

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham45 CPack Cross platform packaging and distribution tool Package binary into native package format Distributed with CMake

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham46 CPack Features Support both CMake and Non-CMake projects Unix: TGZ, TZ, TBZ2, STGZ (Self extract TGZ), RPM DEB Windows: NullSoft Scriptable Install System (NSIS), ZIP Mac OSX: PackageMaker OSXX11 (Mac OSX X11 Bundle)

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham47 Using CPack On Windows install command line ZIP program, and NSIS Setup your project to work with CPack Get make install to work install(…) make sure you executables work with relative paths and can work from any directory set CPack option variables if needed (license, version, etc.) include(CPack) Run CPack make package (create all packages) make package_source (create source package) cpack -C CPackConfig.cmake -G NSIS cpack -C CPackConfig.cmake -G ZIP cpack -C CPackSourceConfig.cmake -G ZIP

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham48 Using CPack Simple Example project(CoolStuff) add_executable(coolstuff coolstuff.cxx) install(TARGETS coolstuff RUNTIME DESTINATION bin) SET(CPACK_PACKAGE_EXECUTABLES "coolstuff" "The Cool Stuff") # used by NSIS include(CPack) $ make package [100%] Built target coolstuff Run CPack packaging tool... CPack: Create package using NSIS CPack: Install projects CPack: - Run preinstall target for: CoolStuff CPack: - Install project: CoolStuff CPack: Compress package CPack: Finalize package CPack: Package C:/CoolStuffExample/CoolStuff win32.exe generated. CPack: Create package using ZIP CPack: Install projects CPack: - Run preinstall target for: CoolStuff CPack: - Install project: CoolStuff CPack: Compress package CPack: Finalize package CPack: Package C:/CoolStuffExample/CoolStuff win32.zip generated.

June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham49June 16-17, 2008NA-MIC, 2008 June Workshop, IHK Akademie Westerham49 Find more about CPack… Online documentation Wiki Books Mastering CMake