Alain Roy Computer Sciences Department University of Wisconsin-Madison Packaging & Testing: NMI & VDT

In Brief: What is NMI? › NSF Middleware Initiative › Funding for middleware infrastructure › Build, package, and test grid software

In Brief: What is VDT? › Virtual Data Toolkit—GriPhyN project › Grid middleware + tools for virtual data › Supports physicists in GriPhyN, usable by anyone  Currently in active use in CMS, Atlas, EDG, and more › Very easy to install

Why do you care? › Condor participates in grid activities › Condor team members are part of NMI & VDT › More testing means better Condor and Condor-G › NMI & VDT make it easy for you to start with grid software

NMI at Condor › Three full-time Condor team members work on NMI › Build & package grid software  Only quality software is included.  Ensure that versions that work together (not always easy to do!) › Test infrastructure

NMI Testing: Verification › NMI does basic verification testing  Can we submit a Globus single job?  Can we submit a Condor-G job?  Can we transfer a single file?

NMI Testing: Local Grids › NMI has seven computers that can be organized in a local-area grid › Will expand with more computers, more architectures

NMI Testing: Local Grids › Stress testing with DAGMan  Coordinated tests of thousands of jobs  Jobs distributed across grid sites  Tests Condor-G, Condor, Globus › Real-life testing with CMS application  Tests everything: Globus, Condor-G, Condor, file transfers…  Real-life workload, high stress

NMI Testing: Larger Grids › We can attach more resources to local grids for stress testing  UW Condor pool with hundreds of nodes  Remote resources › Distributed testing across NMI partner sites

VDT at Condor › Two full-time Condor team members › Packaging and installation  NMI packages software  VDT installs it beautifully › Packaging and installation are not glamorous, but are essential › NMI & VDT work together and make each other better

VDT installation › VDT installation goal:  You hit a button, the software is correctly installed and configured.  (You can customize it afterwards.) › What does this depend on?  Excellent packaging  Excellent installation  Excellent testing

VDT example installation › Download Pacman  › pacman –get VDT-Server  Answer a few questions › pacman –get VDT-Client › ls condor/ ftsh/ Pacman.db replica/ vdt/ doc/ globus/ perl/ setup.csh vdt-install.log edg/ gpt/ post-install/ setup.sh

VDT Testing › Wouldn’t it be nice if VDT could share with NMI?  VDT and NMI people are both on the Condor staff  Collaboration to share: Test harness Tests Test infrastructure

Where can you learn more? › NMI: Bill Taylor: › VDT: Alain Roy:

Take-home message › NMI & VDT are pushing us towards:  Heavily tested grid middleware  Supported software infrastructure Better support, better software