1. National Alliance for Medical Image Computing http://na-mic.org Grid Computing with BatchMake Julien Jomier Kitware Inc.

2. National Alliance for Medical Image Computing http://na-mic.org Motivation Algorithm Design & Implementation Data Processing Reports & Analysis

3. National Alliance for Medical Image Computing http://na-mic.org Outline Introduction to BatchMake Grid Computing with BatchMake Reporting with the BatchBoards BatchMake with Slicer

4. National Alliance for Medical Image Computing http://na-mic.org Introduction to BatchMake Batch script generator Open-Source BSD license Common Language (CMake-like) Generation for: - Shell script - Condor - KWGrid Run locally or on grid Cross platform command line and GUI tool C++ with ITK and FTLK

5. National Alliance for Medical Image Computing http://na-mic.org BatchMake Workflow CommandLine Executable ApplicationWrapper BatchMake Script Local Execution Condor Script KWGrid Script Shell Script

6. National Alliance for Medical Image Computing http://na-mic.org Application Wrapper Describe command line arguments - Manually (GUI) - Automatically using itkMetaCommand Internal XML format.bmm files (BatchMake Modules) Compatible with Slicer execution modules

7. National Alliance for Medical Image Computing http://na-mic.org MetaCommand C++ class for command line parsing Output recognized by - BatchMake (-vxml) - Slicer (--xml) command.SetOption("compileScript","c",false, "Compile a script"); command.AddOptionField("compileScript","filename", MetaCommand::STRING,true); http://www.itk.org/Wiki/MetaIO Insight/Utilities/MetaIO

8. National Alliance for Medical Image Computing http://na-mic.org Example Script 1 - SetApp(imSeg @ImageSegmentation) 2 - SetAppOption(imSeg.threshold 10) 3 - 4 - Set(workdir '/BatchMake/Demo/Ellipse_Images') 5 - ListFileInDir(files ${workdir} '*.mha') 6 - 7 - Foreach(file ${files}) 8- Set(filename '${workdir}/${file}') 9- Set(outfile '${workdir}/out_${file}') 10- SetAppOption(imSeg.inputFilename ${filename}) 11- SetAppOption(imSeg.outputFilename ${outfile}) 12- Run(output ${imSeg}) 13- EndForeach(file)

9. National Alliance for Medical Image Computing http://na-mic.org Condor Specialized workload management system for compute-intensive jobs Created by UW-Madison Free for use Provides job queuing mechanism, scheduling policy, priority scheme, resource monitoring, and resource management. Unix and Windows

10. National Alliance for Medical Image Computing http://na-mic.org Condor Example Script Universe = vanilla Executable = EMSegmenter Arguments = ‘brain.mha segmentation.mha --brain’ Output = emsegmentation.out Error = emsegmentation.error Log = emsegmentation.log Requirements = (OpSys =="LINUX" && Arch=="INTEL") Should_transfer_files = YES When_to_transfer_output = ON_EXIT_OR_EVICT Transfer_input_files = brain.mha Queue

11. National Alliance for Medical Image Computing http://na-mic.org Condor Workflow (Vanilla) Submit Machine Central Manager Execute Machines Condor Pool Submit job Issue job Idle Workstation Busy Workstation Available Execute Machine Push out job to machine Central Manager Unavailable Execute Machine Dispatch to Central manager = Available Dispatch to Central manager = Unavailable

12. National Alliance for Medical Image Computing http://na-mic.org BatchMake Condor Generation

13. National Alliance for Medical Image Computing http://na-mic.org BatchMake Condor Generation Application/Module description - Inputs/Outputs - Data to transfer to nodes Loop unrolling Scope definition Directed Acyclic Graph (DAG) Ensure order of execution of jobs

14. National Alliance for Medical Image Computing http://na-mic.org Monitoring Condor Jobs Flock Monitor (Condor) Condor Watcher (FLTK) Condor Web Watcher (Ajax/PHP)

15. National Alliance for Medical Image Computing http://na-mic.org Reporting with BatchMake CVS/ SVN Results posted on web (i.e., the batchboard) CTest/DART compiles source code Developers review results Developers check-in code MIDAS XNat BatchMake validates on datasets +

16. National Alliance for Medical Image Computing http://na-mic.org Reporting with BatchMake

17. National Alliance for Medical Image Computing http://na-mic.org BatchMake Server PHP Scripts with MySQL database Simple C++ HTTP post Statistical Analysis (Factor analysis: Taguchi). Dashboard creation capability User management

18. National Alliance for Medical Image Computing http://na-mic.org BatchBoard Script DashboardHost(http://www.insight-journal.org/batchmake) DashboardUser(‘Julien Jomier') DashboardKey('FKH3BPM1y11VXsMf') CreateExperiment(exp ‘Liver Project‘ ‘Segmentation Experiment' '') CreateMethod(Meth exp 'Registration' 'Register a T1 with noise to a translated T1') AddMethodInput(noiseImageName Meth 'Fixed Image') Set(fixedSlice '${fixedImage}Slice.png') ExtractSlice('${fixedImage}' '${fixedSlice}' 2 1) AddMethodInput(fixedSlice Meth 'Fixed Slice with noise' png) DashboardSend(Meth)

19. National Alliance for Medical Image Computing http://na-mic.org BatchMake and Slicer BatchMake distributed with Slicer Some modules can be run on a collection of data - EMSegment - GaussianBlur Generic way to run modules on a collection Support for XNat

20. National Alliance for Medical Image Computing http://na-mic.org GridWizard Open-Source Written in Java BIRN and Na-Mic Support for Sun Grid Engine and Condor Shell-like scripts

21. National Alliance for Medical Image Computing http://na-mic.org GridWizard Example bin/gwiz-run prog -t 1:3 prog -t 1 prog -t 2 prog -t 3 bin/gwiz-run prog -i sftp://host.com/opt/mri/subjects/*.img prog -i sftp://host.com/opt/mri/subjects/s1.img prog -i sftp://host.com/opt/mri/subjects/s2.img prog -i sftp://host.com/opt/mri/subjects/s3.img

22. National Alliance for Medical Image Computing http://na-mic.org References www.batchmake.org www.insight-journal.org/batchmake http://forums.gridsphere.org/index.shtml

23. National Alliance for Medical Image Computing http://na-mic.org Condor Job Submission Workflow Collector Submit Machine Scheduler Starter Negociator Central ManagerExecute Machine Master ClassAd Master MatchClassAdJob 1 2 3 4 5