1. OFFICE OF SCIENCE Review Committee for the NuMi Off-Axis Neutrino Appearance (NO A) Experiment at the Fermi National Accelerator Laboratory August 9, 2011 Kurt W. Fisher Review Committee Chair Office of Science, U.S. Department of Energy http://www.science.doe.gov/opa/

2. 2 DOE Organizational Chart OFFICE OF SCIENCE Advance Research Projects Agency-Energy

3. 3 SC Organizational Chart OFFICE OF SCIENCE Chicago Office Roxanne Purucker Office of the Director (SC-1) William F. Brinkman Advanced Scientific Comp. Research (SC-21) Daniel Hitchcock (A) Workforce Development for Teachers/ Scientists (SC-27) Wm. Valdez Basic Energy Sciences (SC-22) Harriet Kung Fusion Energy Sciences (SC-24) Edmund Synakowski High Energy Physics (SC-25) Michael Procario (A) Biological & Environ. Research (SC-23) Sharlene Weatherwax (A) Nuclear Physics (SC-26) Timothy Hallman (A) Acting 7/2011 Deputy Director for Science Programs (SC-2) Patricia Dehmer Deputy Director for Resource Management (SC-4) Jeffrey Salmon Deputy Director for Field Operations (SC-3) Joseph McBrearty (A) Office of Project Assessment (SC-28) Daniel Lehman Office of Budget (SC-41) Kathleen Klausing Office of Scientific and Tech. Info. (SC-44) Walt Warnick Office of SC Project Direction (SC-46) Rebecca Kelley Office of Grants/ Cont. Support (SC-43) Linda Shariati Office of Business Policy and Ops (SC-45) Thomas Phan Business Mgmt & Planning Div (SC-45.1) Vasilios Kountouris SC Systems & Ops Div (SC-45.2) Steven Demore Ames SO Cynthia Baebler Thomas Jeff. SO Joe Arango Stanford SO Paul Golan Pacific NWest SO Julie Erickson (A) Princeton SO Maria Dikeakos Oak Ridge SO Johnny Moore Fermi SO Michael Weis Brookhaven SO Michael Holland Berkeley SO Aundra Richards Argonne SO Joanna Livengood Oak Ridge Office Paul Golan (A) SC Integrated Support Center Office of Lab Policy & Evaluat. (SC-32) D. Streit Office of Safety, Security and Infra. (SC-31) M. Jones Human Capital Resources Div. (SC-45.3) Cynthia Mays Small Business Innovation Research (SC-29) Manny Oliver Office of Science

4. Review Committee Participants OFFICE OF SCIENCE Department of EnergyReview Committee Kurt W. Fisher, DOE, ChairpersonSubcommittee 1: Accelerator and Beamlines * John Quintana, ANL Subcommittee 2: Detector * Bill Wiisniewski, SLAC Richard Loveless, U of Wisc Peter Denes, LBNL Subcommittee 3: Cost, Schedule and Management *Gil Gilchriese, LBNL Ethan Merrill, DOE/SC [Peter Denes, LBNL] Observers Ted Lavine, DOE/SC Eli Rosenberg, DOE/SC Alan Stone, DOE/SC Pepin Carolan, DOE/FSO

5. 5 Charge Questions 1.Baseline Cost and Schedule: Is project’s plan and performance consistent with the approved baseline? Are remaining cost contingency and schedule float adequate for the risks? 2.Technical: Are accomplishments to-date and remaining activities planned sufficient to meet baseline scope objectives? 3.Management: Are the management resources adequate to deliver the project within specifications, budget and schedule, including management and mitigation of remaining technical, cost and schedule risks? 4.Has the project responded satisfactorily to the recommendations from the previous independent project review? OFFICE OF SCIENCE

6. 6 Report Outline/Writing Assignments OFFICE OF SCIENCE Executive SummaryFisher 1. IntroductionLavine 2. Technical (Charge Questions 2, 4) 2.1Accelerator and BeamlinesQuintana*/SC1 2.1.1Findings 2.1.2Comments 2.1.3Recommendations 2.2DetectorWisniewski*/SC2 3. Cost and Schedule (Charge Questions 1, 4) Merrill*/SC3 4. Management (Charge Questions 3, 4) Gilchriese*/SC3 *Lead SC - Subcommittee

7. 7 Expectations  Present closeout reports in PowerPoint.  Forward your sections for each review report (in MSWord format) to Casey Clark, casey.clark@science.doe.gov, casey.clark@science.doe.gov by August 15, 8:00 a.m. (EDT). OFFICE OF SCIENCE

8. 2.1 Accelerator and Beamlines Quintana, ANL 8 OFFICE OF SCIENCE 2.Technical: Are accomplishments to-date and remaining activities planned sufficient to meet baseline scope objectives? Yes 4.Has the project responded satisfactorily to the recommendations from the previous independent project review? Yes, however availability of ceramic beam tubes continues to be an issue.

9.  Findings –The NOvA project is beginning to develop an installation plan for upgrading the Recycler to a Proton ring while upgrading the power to the NuMI target station to 700 kW. –The NOvA project let a PO for target fabrication in March 2011 for expected delivery in Fall 2011. –The NOvA installation coordinator is preparing detailed schedules and plans for the accelerator work in the March 1, 2012 – February 1, 2013 shutdown that includes strategies for limiting radiation dose to workers. –Procurement of ceramic beam tubes continue to be an issue. –The ANU portion of the NOvA project has potential personnel challenges in some technical areas. 9

10.  Comments –While a PO has been issued for the target to RAL, NOvA needs to continue to monitor construction progress to insure delivery to meet project schedule. –The NOvA project requires continued Lab management support with respect to personnel  Recommendations –The NOvA project should insure that the development of the installation schedule includes contingency planning if ceramic beam tubes are delayed. 10

11. 11 2.2 Detector Wisniewski, SLAC/Denes, LBNL Loveless, U of Wisc OFFICE OF SCIENCE 2.Technical: Are accomplishments to-date and remaining activities planned sufficient to meet baseline scope objectives? The detector construction will begin in earnest in the next few months. Extruded parts will arrive before the end of the year. This will allow comparison of task time estimates with reality. Additional problems may arise. The next year will be critical to determination of the course of the project. At this time it appears that the project will be able to meet its baseline scope objectives. 4. Has the project responded satisfactorily to the recommendations from the previous independent project review? The recommendations concerning the detector have been addressed and, for the most part, satisfactorily closed.

12. 12 2.2 Detector Wisniewski, SLAC/Denes, LBNL Loveless, U of Wisc OFFICE OF SCIENCE ■ Findings Near detector has been assembled and read out. This provides good experience for future Far Detector assembly. Manifold cracking observed last year has been understood, and new designs implemented to decrease the likelihood of cracking. Furthermore, procedures for repairing existing cracks have been developed. The design of the block pivoter has proven to be more difficult than anticipated. Procedures planned for block assembly have been reworked with an eye to improvement. 12,000 Avalanche Photodiode (APD) arrays, which are a variant of a commercial device, are required. Failures have been observed, and are thought to be related to contamination. The devices are un- passivated or sealed, and coating them is considered to be a likely solution. Production quantities are delivered 7 months after receipt of order, and an initial lot of 250 devices could be delivered 5.5 – 6 months after order. An order for the full quantity of APDs, with a silicone coating has been placed.

13. 13 2.2 Detector Wisniewski, SLAC/Denes, LBNL Loveless, U of Wisc OFFICE OF SCIENCE ■ Comments Redesign of the extruded manifolds provides very little time for testing before beginning production. Testing of the block pivoter with FHEP will occur in the next few months. Success is critical to beginning block production in time. Infrastructure at Ash River is a very large job and the assigned manpower seems small for building an experiment at a remote site. Delays in infrastructure construction can impact the assembly schedule. Assembly planning calls for 2 10-hour shifts for 4 days, a grueling schedule, which allows little room for adding extra shift as a contingency. Significant value engineering has occurred in the last year. Included in this effort is the use of uniform PVC extrusions as well as adjustment of module layers in the blocks. The consequence is significantly enhanced performance against buckling. The project team should review all procedures for points of failure. They should develop possible alternatives for the most critical of these.

14. 14 2.2 Detector Wisniewski, SLAC/Denes, LBNL Loveless, U of Wisc OFFICE OF SCIENCE ■ Recommendations Conduct an APD review by October 31, 2011, with international experts, to ensure fallback plans and planned testing are thorough and complete. Speed up delivery of an initial lot of coated devices in order to verify the solution.

15. 15 3. Cost and Schedule Merrill, DOE/SC 1.Baseline Cost and Schedule: Is project’s plan and performance consistent with the approved baseline? YES Are remaining cost contingency and schedule float adequate for the risks? YES 4.Has the project responded satisfactorily to the recommendations from the previous independent project review? YES ■ Findings Project is currently 49% complete; Project is 72% obligated; 270 of 500 project milestones complete; SPI = 0.95, CPI = 0.97; Project has updated contingency use plan; Cost contingency is allocated based on risk to discrete activities; Schedule contingency is backend loaded; OFFICE OF SCIENCE

16. 16 3. Cost and Schedule Merrill, DOE/SC ■ Comments Project is actively managing project risks (252 risks/126 retired); Project has recovered 3 of 6 months lost of schedule contingency; Funding vs. Obligation curve is satisfactory; Block Pivoter completion is currently critical path and there are several significant “near critical path” activities; Schedule has been revised to reflect “new” detector production process but project is in the process of restarting production Schedule milestones have been added to track detector assembly; Project is currently in a transition period and uncertainty exists; expect “steady state” for detector assembly by August 2012; High Risk activities currently being managed by project include APDs, detector production, and ANU activities; continue active management to mitigate risks to critical path Consider adding cost and schedule impacts to Risk Register OFFICE OF SCIENCE

17. 17 3. Cost and Schedule Merrill, DOE/SC ■ Recommendations None OFFICE OF SCIENCE

18. 18 NOvA Project Status Merrill, DOE/SC OFFICE OF SCIENCE PROJECT STATUS as of August 9, 2011 Project TypeMIE / Cooperative Agreement CD-1Planned: 5/2007Actual: 5/2007 CD-2Planned: 10/2008Actual: 9/2008 CD-3 Planned: 3a – 2/2009 3b – 10/2009 Actual: 3a – 10/2008 3b – 10/2009 CD-4Planned: 11/2014Actual: TPC Percent Complete Planned: _51.6%Actual: _49% TPC Cost to Date $123.9M TPC Committed to Date $174.5M TPC $244.1M TEC $278M Contingency Cost (w/Mgmt Reserve)$33.9M___27% to go Contingency Schedule on CD-4b____10 months_____33% CPI Cumulative 0.97 SPI Cumulative 0.95

19. 19 4. Management 4. Management *Gilchriese, Denes, LBNL/ Merrill, DOE/SC OFFICE OF SCIENCE 3.Management: Are the management resources adequate to deliver the project within specifications, budget and schedule, including management and mitigation of remaining technical, cost and schedule risks? YES. However, continued vigorous support from the Laboratory and the Collaboration will be necessary to provide personnel resources to the project in order to deliver the project on budget and schedule, and to meet specifications. 4. Has the project responded satisfactorily to the recommendations from the previous independent project review? YES

20. 20 4. Management 4. Management *Gilchriese, Denes, LBNL/ Merrill, DOE/SC OFFICE OF SCIENCE Findings There is 24% contingency (about $30M compared to about $124M to go) on remaining work. Of this, most is assigned at a low level (according to risk) with about $2.7M considered available and not associated with current risk. The management is tracking float to CD-4. There is currently about 500 days of float for the accelerator aspects and about 200 days for the detector. The remaining duration to CD-4 is about 40 months The project has added milestones and plans to add additional milestones to assess float related to critical start dates and to track steady state assembly. There are 252 documented risks of which 126 have been retired. There are seven remaining top-priority risks. Elements of quality assurance are distributed within the project. A top-level QA plan (last updated in July 2009) exists. The project has responded to the recommendations from the previous IPR (August 2010) in NO A-doc-3079 and references therein One open recommendation remains - #5 (Commodities: Complete the vertical slice test by December 15, 2010 to ensure that there are no surprises in light production and collection.)

21. 21 4. Management 4. Management *Gilchriese, Denes, LBNL/ Merrill, DOE/SC OFFICE OF SCIENCE Comments The project should deliver information in a more timely way for the next IPR or other reviews The project is on track to meet the specifications in the threshold KPPs. The project has a clear process and timescale in place to assess if contingency can by utilized to meet the objective KPPs The enhanced near detector would have to be added to the objective KPPs and the process for potentially doing this is understood The management plans for possible utilization of contingency to meet the objective KPPs recognizes the need to maintain contingency for a robust start to production of the detector components and for the accelerator. Most decision dates for use of contingency are in 2012 after steady-state production & assembly has been reached. The remaining contingency is adequate to deliver the project on schedule, but it also important to maintain contingency for commissioning and transition to operations The risk evaluation process should continue to be exercised on a regular basis, and major risks updated. Of the sixteen recommendations from the August 2010 IPR, 15 have been closed. The remaining open recommendation (#5) will be closed by September. 888

22. 22 4. Management 4. Management *Gilchriese, Denes, LBNL/ Merrill, DOE/SC OFFICE OF SCIENCE Recommendations Update the QA plan and organization across the project by December 2011 to prepare for full-scale production, assembly and outfitting.