National Alliance for Medical Image Computing Testing with CTest and CDash Julien Jomier Kitware Inc.

National Alliance for Medical Image Computing Testing Overview CVS/SVN CDash Web Browser CTest Developers

National Alliance for Medical Image Computing Testing Terminology Sites dash5.kitware iris.elmtech Linux-g++ Builds Linux-gcc3 Linux-gcc4 Dashboard

National Alliance for Medical Image Computing Why Test Daily? Large code base too large for any single developer to understand Developers distributed around the world Identify problems as they occur Insure that object API remains unchanged Provide feedback to developers as they experiment with new implementations

National Alliance for Medical Image Computing “If it’s not tested it doesn’t work” Dasboards and Software Process –Roots in GE Six Sigma initiative Cross platform testing Nightly regression testing Memory (Purify/ValGrind) testing Coverage testing Style checking Documentation evaluation

National Alliance for Medical Image Computing How to write a good test? Use empirical values (5/2=2.5) Test most of the use cases (even the boring/most difficult ones) Make your tests as efficient as possible Have someone else write the test for your algorithm

National Alliance for Medical Image Computing Regression Testing Compare generated image against standard “correct” image –pixel-by-pixel comparison –can use a threshold metric –adjusted for effects like dithering

National Alliance for Medical Image Computing CTest Testing client Distributed with CMake Submit test results to the dashboard XML submission Independent of CMake

National Alliance for Medical Image Computing CTest with CMake CMakeLists.txt INCLUDE(CTest) ENABLE_TESTING() # Add a simple test ADD_TEST(CDashTest1 CDashTest CDashTest) ADD_EXECUTABLE(CDashTest main.cxx) CTestConfig.txt set(CTEST_PROJECT_NAME "MyProject") set(CTEST_NIGHTLY_START_TIME "21:00:00 EDT") set(CTEST_DROP_METHOD "http") set(CTEST_DROP_SITE "localhost") set(CTEST_DROP_LOCATION "/CDash/submit.php?project=MyProject") set(CTEST_DROP_SITE_CDASH TRUE)

National Alliance for Medical Image Computing Testing Submission XML Submission XML Parsing HTTP PUT CTest CDash Backup XML File Fill Database Send Update RSS Feed Update.xml Configure.xml Build.xml Test.xml Coverage.xml DynamicAnalysis.xml Compiler/Tester

National Alliance for Medical Image Computing CDash New generation of Dart Written in Php/Javascript MySQL Open Source Run on top of a web server XSL implementation Web 2.0

National Alliance for Medical Image Computing CDash Features Ajax requests Graphs: build, test times Builds group User management notifications Timing defects

National Alliance for Medical Image Computing Coverage Support Low Coverage Bullseye support

National Alliance for Medical Image Computing Build Notes Share with other developers Keep it simple

National Alliance for Medical Image Computing My CDash Personal Page Subscribe to project View sites preferences

National Alliance for Medical Image Computing Project Roles Normal user Site maintainer Project administrator

National Alliance for Medical Image Computing Claiming Sites Site information Automatic (CTest 2.6) Information history Manual editing Personal view/

National Alliance for Medical Image Computing CDash Misc. Automatic Geolocation from IP addresses Support for iPhone

National Alliance for Medical Image Computing Installing CDash (L/W)AMP : Apache, MySQL, PHP - XAMPP, EasyPHP Download CDash ( in your Apache web directorywww.cdash.org Edit config.php Point Web Browser to CDash - Setup the database - Create first project

National Alliance for Medical Image Computing Style Checking KWStyle C++ Style checker Configuration via XML HTML Generator (TBI in CDash) Open Source Written in C++

National Alliance for Medical Image Computing Contribute Website Bug Tracker - Mailing List - bin/mailman/listinfo/cdash

National Alliance for Medical Image Computing Happy Testing!