1. Alain Roy Computer Sciences Department University of Wisconsin-Madison roy@cs.wisc.edu http://www.cs.wisc.edu/condor Packaging & Testing: NMI & VDT

2. www.cs.wisc.edu/condor In Brief: What is NMI? › NSF Middleware Initiative › Funding for middleware infrastructure › Build, package, and test grid software

3. www.cs.wisc.edu/condor In Brief: What is VDT? › Virtual Data Toolkit—GriPhyN project › Grid middleware + tools for virtual data › Supports physicists in GriPhyN, usable by anyone  Currently in active use in CMS, Atlas, EDG, and more › Very easy to install

4. www.cs.wisc.edu/condor Why do you care? › Condor participates in grid activities › Condor team members are part of NMI & VDT › More testing means better Condor and Condor-G › NMI & VDT make it easy for you to start with grid software

5. www.cs.wisc.edu/condor NMI at Condor › Three full-time Condor team members work on NMI › Build & package grid software  Only quality software is included.  Ensure that versions that work together (not always easy to do!) › Test infrastructure

6. www.cs.wisc.edu/condor NMI Testing: Verification › NMI does basic verification testing  Can we submit a Globus single job?  Can we submit a Condor-G job?  Can we transfer a single file?

7. www.cs.wisc.edu/condor NMI Testing: Local Grids › NMI has seven computers that can be organized in a local-area grid › Will expand with more computers, more architectures

8. www.cs.wisc.edu/condor NMI Testing: Local Grids › Stress testing with DAGMan  Coordinated tests of thousands of jobs  Jobs distributed across grid sites  Tests Condor-G, Condor, Globus › Real-life testing with CMS application  Tests everything: Globus, Condor-G, Condor, file transfers…  Real-life workload, high stress

9. www.cs.wisc.edu/condor NMI Testing: Larger Grids › We can attach more resources to local grids for stress testing  UW Condor pool with hundreds of nodes  Remote resources › Distributed testing across NMI partner sites

10. www.cs.wisc.edu/condor VDT at Condor › Two full-time Condor team members › Packaging and installation  NMI packages software  VDT installs it beautifully › Packaging and installation are not glamorous, but are essential › NMI & VDT work together and make each other better

11. www.cs.wisc.edu/condor VDT installation › VDT installation goal:  You hit a button, the software is correctly installed and configured.  (You can customize it afterwards.) › What does this depend on?  Excellent packaging  Excellent installation  Excellent testing

12. www.cs.wisc.edu/condor VDT example installation › Download Pacman  http://physics.bu.edu/~youssef/pacman › pacman –get VDT-Server  Answer a few questions › pacman –get VDT-Client › ls condor/ ftsh/ Pacman.db replica/ vdt/ doc/ globus/ perl/ setup.csh vdt-install.log edg/ gpt/ post-install/ setup.sh

13. www.cs.wisc.edu/condor VDT Testing › Wouldn’t it be nice if VDT could share with NMI?  VDT and NMI people are both on the Condor staff  Collaboration to share: Test harness Tests Test infrastructure

14. www.cs.wisc.edu/condor Where can you learn more? › NMI: www.nsf-middleware.org Bill Taylor: bt@cs.wisc.edu › VDT: www.griphyn.org/vdt Alain Roy: roy@cs.wisc.edu

15. www.cs.wisc.edu/condor Take-home message › NMI & VDT are pushing us towards:  Heavily tested grid middleware  Supported software infrastructure Better support, better software