DOE/NSF Project Experience Key Ingredients to Success Cosmology with CMB-S4 Workshop DOE/NSF Project Experience Key Ingredients to Success SLAC National Accelerator Laboratory February 27, 2017 Jim Yeck

Outline Personal Experience DOE Office of Science (SC) Experience Projects after the Superconducting Super Collider DOE SC management perspectives NSF Large Project Experience IceCube Large Hadron Collider Experiments Satisfying Needs of Project Stakeholders Next Steps

My projects Infrastructure Project Purpose Cost/Circa for CD-3 Funding Role Compact Ignition Tokamak (CIT) at Princeton Plasma Physics Lab Fusion Energy Science $330M 1988 DOE DOE Acting Project Manager Relativistic Heavy Ion Collider (RHIC) at Brookhaven Lab (BNL) Nuclear Physics $600M 1991 DOE + NSF + Int DOE Project Manager US Large Hadron Collider (USLHC) In-kind delivered to CERN High Energy Physics $530M 1998 DOE & NSF DOE/NSF Project Director IceCube Neutrino Observatory at South Pole Particle Astrophysics $300M 2005 NSF + Int U of Wisconsin -– Project Director National Synchrotron Light Source II at BNL Photon Source $900M 2008 DOE + Other BNL - Deputy Project Director Deep Underground Science and Engineering Laboratory (DUSEL) Physics, Biology, and Engineering $750M 2010 NSF + Private U of Cal – Associate Project Director European Spallation Source (ESS) in Sweden Neutron Source $2,500M 2014 European States ESS ERIC – Director General & CEO Time frame for each of the projects being presented.

Key Ingredients to success Facility is a priority of the science community! Strong funding agency commitments and host role Project leaders viewed as enabling success of others Establish realistic goals – “Experience over hope” Credibility through openness and transparency Collective ownership of problems and solutions Populate organization with critical experience Success requires energy and enthusiasm! Font color to white (dependent on how lighting is in room) Bullets in front of each Project leaders who prioritize on schedule performance and exhibit behaviour that is consistent with a “project culture” are likely to be successful!

Projects evaluated against the ingredients #1 Priority of Science Community #2 Strong Agency Commitment & Host Role #3 Leaders Enable the Success of Others #4 Experience Over Hope #5 Openness and Trans-parency #6 Collective Owner-ship #7 & #8 Experience, Energy & Enthusiasm Outcome CIT   ✖ RHIC ✔ USLHC IceCube NSLS II DUSEL ✖ ? ESS ? -Build presentation with transition/animation row by row? -Then go through projects one by one

DOE Office of Science Project Performance After SSC Failure of SSC resulted in major changes Current performance is excellent and SC has high credibility Dan Lehman, PLI Jan 2017

Some Unique Features of SC Projects SC Laboratories are Not-for-Profit M&O contracts Projects are typically ‘build to cost’ with a goal of maximizing science capability Project designs consider future upgrades—programs and projects take a long view Mostly non-nuclear projects Laboratory’s viability/future is dependent of the success of new advanced facilities or projects to perform state-of-the-art R&D and to attract the best and brightest Highly technical HQ Program personnel (with extensive laboratory experience) allows evaluation of feasibility and complexity of proposed technical approaches by the labs. Dan Lehman, PLI Jan 2017

Successful Projects – Dan Lehman (DOE retired) Primary Factors for Successful Project Completion Clear Ownership, Accountability, and Responsibilities Effective Front-End Planning Appropriate Project Contingencies Sufficient and Stable Funding Regular Independent Oversight Jeff to give short bio of Dan Dan Lehman, PLI Jan 2017

Pat Dehmer, DOE PLI 01/17

DOE Office of Science – 2003 priorities vs 2013 status Some priorities become projects later (and earlier) than originally planned Project scope is often different than earliest proposals Not all priorities are realized Pat Dehmer, DOE PLI 01/17

NSF Large Project Experience NSF is not a “mission” agency like DOE and NASA Major Research Equipment and Facilities Construction (MREFC) Program funds NSF large projects Many successful MREFC projects but some problems A few early off-ramps, e.g., RSVP, DUSEL Increased expectations and requirements for projects Stronger emphasis on front end planning No cost overrun philosophy embraced NSF experienced and successful in delivering projects that enable scientific collaborations

The IceCube Collaboration includes > 300 researchers from 47 institutes in 12 countries. IceCube is one of the NSF’s large facilities (LIGO, LSST, … 2 dozen others). The Operations and Management of the facility is handled by WIPAC at UW-Madison

Official birth of the ATLAS The birth of ATLAS March 1992 – Summer 1992 Merging of EAGLE and ASCOT September 1992: Decision on the name 1st round 2nd round ATLAS 31 ATLAS 40 ALICE 12 ALICE 13 ACE 5 ALEX 5 LHD 0 October 1992 ATLAS LoI submitted to the LHCC Official birth of the ATLAS Collaboration ThyssenKrupp, 8-11-2010 Peter Jenni (CERN) ATLAS Project at CERN's LHC

ATLAS Project at CERN's LHC The ATLAS organization has been defined in a lean document (less than 10 pages) in 1994, approved by the Collaboration Board (CB) and the RRB (Funding Agencies) Some key elements: - Each Institution has one vote (independent on the the number of people and the resources) in the CB - The CB meets 3-4 a year, and has an elected Chair - The CB elects the Spokesperson (SP, the ‘CEO’) of the Collaboration, by ballot, renewable with a 2/3 majority (initially renewable 3-year terms of office, now 2-year and only once renewable) - Based on explicit consultation, the SP proposes the other management positions, to be endorsed by ballot by the CB (CERN Director General has to agree on Technical and Resources Coordinators) - Sub-system Project Leaders (PLs) elected by the Institutes forming a sub-system, the SP has a right to ‘influence constructively’ this process - The SP finally propose the PLs to the CB for endorsement (by ballot) to form the Executive Board (EB) - The composition of the EB evolves according to the ATLAS Project phases ATLAS Management 2004-2009 ThyssenKrupp, 8-11-2010 Peter Jenni (CERN) ATLAS Project at CERN's LHC

FH Wiener Neustadt, Wisconsin, Wuppertal, Würzburg, Yale, Yerevan ATLAS Collaboration (Status August 2010) 38 Countries 174 Institutions 3000 Scientific participants total (1000 Students) Albany, Alberta, NIKHEF Amsterdam, Ankara, LAPP Annecy, Argonne NL, Arizona, UT Arlington, Athens, NTU Athens, Baku, IFAE Barcelona, Belgrade, Bergen, Berkeley LBL and UC, HU Berlin, Bern, Birmingham, UAN Bogota, Bologna, Bonn, Boston, Brandeis, Brasil Cluster, Bratislava/SAS Kosice, Brookhaven NL, Buenos Aires, Bucharest, Cambridge, Carleton, CERN, Chinese Cluster, Chicago, Chile, Clermont-Ferrand, Columbia, NBI Copenhagen, Cosenza, AGH UST Cracow, IFJ PAN Cracow, SMU Dallas, UT Dallas, DESY, Dortmund, TU Dresden, JINR Dubna, Duke, Edinburgh, Frascati, Freiburg, Geneva, Genoa, Giessen, Glasgow, Göttingen, LPSC Grenoble, Technion Haifa, Hampton, Harvard, Heidelberg, Hiroshima IT, Indiana, Innsbruck, Iowa SU, Iowa, UC Irvine, Istanbul Bogazici, KEK, Kobe, Kyoto, Kyoto UE, Lancaster, UN La Plata, Lecce, Lisbon LIP, Liverpool, Ljubljana, QMW London, RHBNC London, UC London, Lund, UA Madrid, Mainz, Manchester, CPPM Marseille, Massachusetts, MIT, Melbourne, Michigan, Michigan SU, Milano, Minsk NAS, Minsk NCPHEP, Montreal, McGill Montreal, RUPHE Morocco, FIAN Moscow, ITEP Moscow, MEPhI Moscow, MSU Moscow, LMU Munich, MPI Munich, Nagasaki IAS, Nagoya, Naples, New Mexico, New York, Nijmegen, Northern Illinois, BINP Novosibirsk, Ohio SU, Okayama, Oklahoma, Oklahoma SU, Olomouc, Oregon, LAL Orsay, Osaka, Oslo, Oxford, Paris VI and VII, Pavia, Pennsylvania, NPI Petersburg, Pisa, Pittsburgh, CAS Prague, CU Prague, TU Prague, IHEP Protvino, Regina, Rome I, Rome II, Rome III, Rutherford Appleton Laboratory, DAPNIA Saclay, Santa Cruz UC, Sheffield, Shinshu, Siegen, Simon Fraser Burnaby, SLAC, South Africa, Stockholm, KTH Stockholm, Stony Brook, Sydney, Sussex, AS Taipei, Tbilisi, Tel Aviv, Thessaloniki, Tokyo ICEPP, Tokyo MU, Tokyo Tech, Toronto, TRIUMF, Tsukuba, Tufts, Udine/ICTP, Uppsala, UI Urbana, Valencia, UBC Vancouver, Victoria, Waseda, Washington, Weizmann Rehovot, FH Wiener Neustadt, Wisconsin, Wuppertal, Würzburg, Yale, Yerevan

Project Management Technical Management Board - TMB (Chaired by the Technical Coordinator) Spokesperson ATLAS Mgt. Executive Board TC TMB technical activities technical aspects computing physics aspects political aspects global ATLAS policy aspects (from CB) resources aspects TMB systems, sub-systems, WGs, tasks, etc. dedicated task forces CERN tech. services Project Management ThyssenKrupp, 8-11-2010 Peter Jenni (CERN) ATLAS Project at CERN's LHC

DOE and NSF Joint Oversight Group Model DOE/NSF Joint Oversight Group Established for LHC in 1998 – standard practice today for joint projects Clear lines of institutional and individual accountability

DOE/NSF US LHC Funding Agreement – circa 1998 DOE/NSF Approved USLHC Project Execution Plan (DOE MIEs and NSF MREFC) Partnering between DOE and NSF (plus DOE/NSF w/ CERN) resulted in unwavering commitment by the funding agencies

Satisfying Needs of Project Stakeholders Funding Agency Organized scientific collaboration w/ defined governance Clear lines of project accountability and institutional authority, strong oversight and advisory bodies Collaborating Institutions Responsibilities (inst./people) understood and recognized Firm commitment by funding agencies and collaborators Collaborators Strong role shaping science requirements Formal roles and continuing influence in project planning and execution

Next Steps Collaborators Continue collaborative work to define scientific and technical requirements and to grow the collaboration Smaller group to review collab models and to define preferred options for consideration the collaboration Respect the need to eventually establish a strong project planning and delivery organization and collaborate on ways to influence and strengthen the project organization Collaborating Institutions (Universities, Labs, …) Start discussing possible project management schemes and ways to coordinate interfaces with funding agencies

Questions? Thank You!

Backup Slides from Dan Lehman’s Presentation at the DOE Project Leadership Institute in Jan 2017

Clear Ownership, Accountability and Responsibility The Owner: Is usually the Acquisition Executive (Project Management Executive) Is accountable and responsible for project success Prioritizes projects and needs based on available funding Requests, defends, and provides the project funding Seeks advice and recommendations from Subject Matter Experts on project through project peer reviews, ESAAB processes, regular communication with the project team, etc. Approves critical decisions, which allows a project to proceed to the next phase Approves major changes to the project Makes difficult and timely decisions Dan Lehman, PLI Jan 2017

Include facility or system users in the planning process Front–End Planning Ensure that scope and technical specifications are well defined and documented Include facility or system users in the planning process Design should be sufficiently mature prior to baselining Ensure competent, proactive, experienced, and knowledgeable project team is assembled Identify and plan for all applicable requirements Identify, evaluate, and plan for internal and external risks Dan Lehman, PLI Jan 2017

Project Contingencies Cost contingency is included in the TPC regardless of who owns the risk (external or internal) Schedule contingency and the cost associated with potential schedule delays are included in the baseline Scope contingency (the difference between the Threshold and Objective KPPs) is identified in the PEP Contingency is based on project status, complexity, and risks. Analysis of contingency is performed continuously throughout the life of the project Contingencies (cost, schedule, and scope) are held by the Federal Project Director and released to the project using a formal change control process identified in the PEP Dan Lehman, PLI Jan 2017

Project funding profile needs to be realistic Project funding needs to be affordable in the context of the total Program Project funding profile needs to be realistic Funding must include adequate project contingency (regardless of who owns the risk) Funding must be stable and should not be changed after CD-2 is approved Dan Lehman, PLI Jan 2017

Independent Oversight Projects have a tendency to be too optimistic Independent project or peer reviews are necessary to ensure there are check and balances To be effective, the Independent Project Reviews need: Implementation of review recommendations To be performed regularly Diverse and knowledgeable review committee with relevant expertise/experience Communication of findings and issues with Senior Management Senior Management support and backing Develop and communicate policies, and ensure they are consistently and appropriately implemented Dan Lehman, PLI Jan 2017