Management & Coordination Paul Avery, Rick Cavanaugh University of Florida Ian Foster, Mike Wilde University of Chicago, Argonne GriPhyN NSF Project Review January 2003 Chicago

229 Jan 2003 Mike Wilde, University of Chicago GriPhyN Management Paul Avery (Florida) –co-Director Ian Foster (Chicago) –co-Director Mike Wilde (Argonne) –Project Coordinator Rick Cavanaugh (Florida) –Deputy Coordinator

329 Jan 2003 Mike Wilde, University of Chicago Management and Coordination Challenges Management structure External advisory committee How we work How we are coordinated Meetings Managing and tracking progress External collaborations Summary

429 Jan 2003 Mike Wilde, University of Chicago GriPhyN Project Challenges We balance and coordinate –CS research with “goals, milestones & deliverables” –GriPhyN schedule/priorities/risks with those of the 4 experiments –General tools developed by GriPhyN with specific tools developed by 4 experiments –Data Grid design, architecture & deliverables with those of other Grid projects Appropriate balance requires –Tight management, close coordination, trust We have (so far) met these challenges –But requires constant attention, good will

529 Jan 2003 Mike Wilde, University of Chicago External Advisory Committee Physics Experiments Project Directors Paul Avery Ian Foster Internet 2DOE Science NSF PACIs Project Coordination Mike Wilde Rick Cavanaugh Outreach/Education Manuela Campanelli Industrial Connections Ian Foster / Paul Avery EDG, LCG, Other Grid Projects Architecture Carl Kesselman VDT Development Coord.: M. Livny Requirements, Definition & Scheduling (Miron Livny) Integration, Testing, Documentation, Support (Alain Roy) Globus Project & NMI Integration (Carl Kesselman) CS Research Coord.: I. Foster Virtual Data (Mike Wilde) Request Planning & Scheduling (Ewa Deelman) Execution Management (Miron Livny) Measurement, Monitoring & Prediction (Valerie Taylor) Applications Coord.: R. Cavanaugh ATLAS (Rob Gardner) CMS (Rick Cavanaugh) LIGO (Albert Lazzarini) SDSS (Alexander Szalay) Inter-Project Coordination: R. Pordes HICB (Larry Price) HIJTB (Carl Kesselman) PPDG (Ruth Pordes) TeraGrid, NMI, etc. (Carl Kesselman) International (EDG, etc) (Ruth Pordes) GriPhyN Management iVDGL iVDGL Rob Gardner

629 Jan 2003 Mike Wilde, University of Chicago External Advisory Committee Members –Fran Berman (SDSC Director) –Dan Reed (NCSA Director) –Joel Butler (former head, FNAL Computing Division) –Jim Gray (Microsoft) –Bill Johnston (LBNL, DOE Science Grid) –Fabrizio Gagliardi (CERN, EDG Director) –David Williams (former head, CERN IT) –Paul Messina (former CACR Director) –Roscoe Giles (Boston U, NPACI-EOT) Met with us 3 times: 4/2001, 1/2002, 1/2003 –Extremely useful guidance on project scope & goals

729 Jan 2003 Mike Wilde, University of Chicago How We Work (1) System architecture & virtual data toolkit as two overarching organizational mechanisms Project activities all defined in relationship to these organizing principles: –Research: Explore new techniques to guide evolution of the system architecture and VDT –Development: Construct VDT software –Experimentation: Deploy, apply, & evaluate VDT software and/or new techniques in context of application challenges Intimate coordination with experiments for requirements, evaluation, dissemination

829 Jan 2003 Mike Wilde, University of Chicago How We Work (2) Computer Science Research Virtual Data Toolkit Partner Physics Projects Larger Science Community Globus, Condor, NMI, EU DataGrid, PPDG Communities Production Deployment Tech Transfer Techniques & software Requirements Prototyping & experiments Other linkages: - Work force - CS researchers - Industry GriPhyN

discovery Science Review Production Manager Researcher discovery sharing instrument Applications Virtual Data storage element Grid Grid Fabric storage element storage element composition planning data Planning Execution Virtual Data Toolkit Services Chimera virtual data system Pegasus planner DAGman Globus Toolkit Condor Ganglia, etc. GriPhyN Architecture Performance ProductionAnalysis params exec. data

1029 Jan 2003 Mike Wilde, University of Chicago How We Are Coordinated The activities of this large, multidisciplinary group are coordinated by frequent and multivalent communications –Face-to-face meetings in large & small groups –Formal and informal documents defining requirements, challenge problems, testbeds – lists, phone calls, web sites, videoconferences –Cooperation on challenge problems and technology and application demonstrations –Cooperation on software releases Explicit, long-term pairings with experiments to ensure communication & collaboration

1129 Jan 2003 Mike Wilde, University of Chicago Meetings in GriPhyN/iVDGL meetings –Oct. 2000All-handsChicago –Dec. 2000ArchitectureChicago –Apr. 2001All-hands, EACUSC/ISI –Aug. 2001PlanningChicago –Oct. 2001All-hands, iVDGLUSC/ISI Numerous smaller meetings –CS-experiment –CS research –Liaisons with PPDG and EU DataGrid –US-CMS and US-ATLAS computing reviews –Experiment meetings at CERN

1229 Jan 2003 Mike Wilde, University of Chicago Meetings in 2002 GriPhyN/iVDGL meetings –Jan. 2002EAC, Planning, iVDGLFlorida –Mar. 2002Outreach WorkshopBrownsville –Apr. 2002All-handsArgonne –Jul. 2002Reliability WorkshopISI –Oct. 2002Provenance WorkshopArgonne –Dec. 2002Troubleshooting WorkshopChicago –Dec. 2002All-hands technicalISI + Caltech –Jan. 2003EACSDSC Numerous other 2002 meetings –iVDGL facilities workshop (BNL) –Grid activities at CMS, ATLAS meetings –Several computing reviews for US-CMS, US-ATLAS –Demos at IST2002, SC2002 –Meetings with LCG (LHC Computing Grid) project –HEP coordination meetings (HICB)

1329 Jan 2003 Mike Wilde, University of Chicago Managing and Tracking Progress Project milestones –Initially prepared for NSF at time of award –Yearly “annual workplan” maps from these milestones to more detailed subgroup goals >Experiment subgroups: ATLAS, CMS, LIGO, SDSS >Technology subgroups: CS areas & VDT –Tracked regularly by management team –Reports to project at all-hands meetings Plus –VDT releases as technology coordination point –Application data challenges & demonstrations as cross-cutting coordination mechanisms

Progress Towards Project Goals

1529 Jan 2003 Mike Wilde, University of Chicago Global Context: Grid Projects U.S. Infrastructure Projects –GriPhyN (NSF) –iVDGL (NSF) –Particle Physics Data Grid (DOE) –PACIs and TeraGrid (NSF) –DOE Science Grid (DOE) –NSF Middleware Infrastructure (NSF) –National Virtual Observatory (NSF) EU, Asia major projects –European Data Grid (EDG) (EU, EC) –EDG-related national Projects (UK, Italy, France, …) –CrossGrid (EU, EC) –DataTAG (EU, EC) –VIRGO, GEO, European Virtual Observatory projects –LHC Computing Grid (LCG) (CERN) –Japanese Grid Projects –Korea Grid project

1629 Jan 2003 Mike Wilde, University of Chicago U.S. Project Coordination: Trillium Trillium = GriPhyN + iVDGL + PPDG –Large overlap in leadership, people, experiments Benefit of coordination –Common S/W base + packaging: VDT + PACMAN –Low overhead for collaborative or joint projects –Wide deployment of new technologies –Stronger, more extensive outreach effort Forum for US Grid projects –Joint view, strategies, meetings and work –Unified entity to deal with EU & other Grid projects

1729 Jan 2003 Mike Wilde, University of Chicago International Coordination EU DataGrid & DataTAG –Advisory boards and formal collaboration HICB: HEP Inter-Grid Coordination Board –HICB-JTB: Joint Technical Board –GLUE Participation in LHC Computing Grid (LCG) International networks –Standing Committee on Inter-regional Connectivity –Digital Divide projects, IEEAF

1829 Jan 2003 Mike Wilde, University of Chicago Summary GriPhyN has succeeded in creating a team from 60+ people at 12 institutions, linking also with major appln & CS projects –Multidisciplinary teams that actually interact on a daily basis—and even like each other! –Real and substantial progress on computer science, technology, and science infrastructure We take this for granted, but we believe that in fact it is a remarkable achievement We’re not exactly sure what makes it work, but work it does –One major reason must surely be the caliber and leadership roles of the participants