1. US Belle II Project Overview Jim Fast Pacific Northwest National Laboratory

2. 19-20 March 2014US Belle II Independent Project Review2 Outline Project Scope Key Performance Parameters Project Team Project Schedule Project Cost Project Risks and Contingency Analysis Contracting, ES&H and Quality Communications and Reporting Project Reviews and Responses CD-2/3 Requirements Checklist Summary

3. 19-20 March 2014US Belle II Independent Project Review3 US Belle II project scope: delivering key systems to KEK US Belle II Project 1.01 Project Integration and Support 1.02 iTOP Optics 1.03 Readout Systems 1.04 KLM Systems 1.05 Commissioning Detectors KLM Systems (WBS 1.04) iTOP Optics (WBS 1.02) Readout Systems (WBS 1.03) to record signals from WBS 1.02 and 1.04 components Commissioning Detectors (WBS 1.05) 3

4. 19-20 March 2014US Belle II Independent Project Review4 WBS 1.02: iTOP Optics Zygo Bar ITT Mirror Zygo Prism

5. 19-20 March 2014US Belle II Independent Project Review5 WBS 1.03: Readout Systems

6. 19-20 March 2014US Belle II Independent Project Review6 WBS 1.04: KLM Modules

7. 19-20 March 2014US Belle II Independent Project Review7 WBS 1.05: Commissioning Detectors

8. 19-20 March 2014US Belle II Independent Project Review8 CD- 4 Definition The US Belle II construction project will deliver parts or sub-assemblies for Belle II and the Key Performance Parameters (KPPs) are defined around pre-installation acceptance/testing of these, not final in situ performance Project Completion Criteria – WBS 1.02: 16 sets of Optical Components – WBS 1.03: Readout Systems Supporting iTOP and KLM – WBS 1.04: 32 KLM Modules - COMPLETE – WBS 1.05: 64 PIN diodes and 8 Time Projection Chambers KPPs and acceptance criteria are unaltered since CD-1 – KPPs are document in the Project Execution Plan – Acceptance Criteria have been approved by KEK and BPAC

9. 19-20 March 2014US Belle II Independent Project Review9 US Belle II Requirements/Deliverables WBS 1.02 – Quartz optics (used to distinguish two types of particles) – KPP is 16 sets (2 bars+1mirror+1prism) – Technical requirements are captured in engineering drawings and specifications documents used to establish contracts with vendors – Project completion is marked by acceptance tests done in Japan WBS 1.03 – Readout Systems (record signals from WBS 1.02/1.04 systems) – KPP is full set of readout for iTOP and KLM detectors – Technical requirements are captured in engineering drawings – Elements are fabricated at U. Hawaii based on proven design and standard circuit boards - current ASICs meets Completion Criterion – Project complete after acceptance tests and delivery to Japan Acceptance Criteria documents (available on review web page) define specifications that must be met to meet KPPs

10. 19-20 March 2014US Belle II Independent Project Review10 US Belle II Requirements/Deliverables WBS 1.04 – KLM Modules (used to measure muons and K-long mesons) – KPP is 32 modules delivered to Japan and passing acceptance tests – Modules are ~2mx2m panels of plastic scintillator with photodiodes – Project complete after acceptance tests and delivery to Japan – Completed in 2013 WBS 1.05 – Commissioning Detectors (used during accelerator startup) – KPP is 64 PIN diodes and 8 micro-Time-Projection-Chambers (micro-TPCs) – Technical requirements for diodes are easy; micro-TPCs demonstrated already – Project complete when delivered to Japan and pass acceptance tests Acceptance Criteria documents (available on review web page) define specifications that must be met to meet KPPs

11. 19-20 March 2014US Belle II Independent Project Review11 Strong management team in place and working together US Belle II Project IPT Members Federal Program Manager (Helmut Marsiske OHEP) Federal Project Director (Jeff Day PNSO); Contracting Officer (Ryan Kilbury PNSO) Contractor Project Manager (Jim Fast PNNL) 11

12. 19-20 March 2014US Belle II Independent Project Review12 Project team draws on talent from across multiple organizations 12

13. 19-20 March 2014US Belle II Independent Project Review13 US Belle II project schedule 13

14. 19-20 March 2014US Belle II Independent Project Review14 US Belle II schedule meets SuperKEKB ‘first science run start’ objective KLM module installation summer 2013 Commissioning detector for SuperKEKB runs in 2015 and 2016 iTOP module installations spring 2015 and summer 2017

15. 19-20 March 2014US Belle II Independent Project Review15 Quartz Fabrication Schedule

16. 19-20 March 2014US Belle II Independent Project Review16 US Belle II project funding Cost estimate in at-year M$ $6.13M OPC $6.82M TEC 30% contingency on TEC 16

17. 19-20 March 2014US Belle II Independent Project Review17 Cost Drivers iTOP Optics are about 50% of TPC – Estimates based on fixed price contracts – Existing contracts include options for production – Largest cost uncertainty is split of bars between two vendors (AOS and Zygo; baseline 10:17) Second largest cost is readout electronics – Well-established cost basis for ASICS (MOSIS), PCBs and board components – FPGAs are significant fraction of board cost

18. 19-20 March 2014US Belle II Independent Project Review18 Resource loaded cost estimate and commitments complies with OHEP funding 18

19. 19-20 March 2014US Belle II Independent Project Review19 Pre-Baseline Change Request Log Project is utilizing baseline change request process to track changes 19

20. 19-20 March 2014US Belle II Independent Project Review20 Major risks identified and mitigation strategies being executed The major risks have been identified and mitigation strategies developed Handling actions and due dates have been identified and are monitored Risks and handling actions are reviewed and reassessed ~quarterly Key risks: – Quartz vendors have production problems and delivery of quartz is delayed or vendors must be added. Mitigation strategy (schedule): Prototyping, use of multiple vendors, understanding implications of non-ideal articles Mitigation strategy (cost): Multiple vendors have been identified and developed; contracts include clauses allowing quantities to be reduced to allow flexibility to move production to alternate vendors in case of quality or schedule issue; differential cost for bars is understood and has been used to develop baseline cost and contingency – ASICs (TARGET or IRS) require an additional design cycle, delaying production. Mitigation strategy: Advanced TARGET testing with off-project collaboration interested in chip for other uses; following lower-risk development of IRS chip with discrete RF amplifiers rather than BLAB with integrated RF amplifiers Multiple versions of IRS chip produced in preproduction run reducing risk exposure to single design failure – Grant-funded tasks do not produce required project-related outcomes and funding must be supplemented by project. Mitigation strategy: Working closely with grant monitor so that required funding levels are understood. Project contingency in these areas is high to account for possible need to move to project-funded resources 20

21. 19-20 March 2014US Belle II Independent Project Review21 Project Contingency Analysis Monte Carlo risk analysis is in-line with Management assessment of risk Baseline contingency of $2.1M (30% of TEC) is supported by the quantitative analysis Monte Carlo analysis gives 8 months float to CD-4 date at 80% confidence 21

22. 19-20 March 2014US Belle II Independent Project Review22 Contracts Project contracts lead is head of directorate contracts group – Two contracts specialists assigned to manage key contracts – One experienced in University subcontracts – Second is managing all quartz contracts for the project Contracts managed using standard PNNL processes – E.g. Acquisition Review Board for large contracts (quartz bars) EH&S, Quality and Risk are integrated with contracting process – Standard and project-specific contract clauses

23. 19-20 March 2014US Belle II Independent Project Review23 Environment, Safety and Health Utilizing robust Integrated Safety Management systems at PNNL Project-specific Hazard Analysis Report catalogues potential ES&H risks ES&H expectations flowed down from PNNL to collaborating institutions through subcontracts or Memoranda of Understandings National Environmental Policy Act (NEPA) – NEPA met by a categorical exclusion B3.6 (signed off by NCO at PNSO).

24. 19-20 March 2014US Belle II Independent Project Review24 Quality Assurance Project-Specific Quality Assurance Program (PQAP) Configuration of the project baseline documents is being maintained using the formal change control process US Belle II Project Office periodically assesses engineering and technology readiness (management and independent assessments) Project is using the PNNL TRACS System as the central repository to track any project QA issues, deficiencies, non- conformances and related corrective actions Acceptance testing plans include both vendor testing and in-house testing programs

25. 19-20 March 2014US Belle II Independent Project Review25 Communications and Reporting Official reporting monthly through PARS II Monthly reports prepared by project office based on monthly schedule status from Level 2 managers as well as financial data from PNNL finance system – Submitted to FPD and FPM Monthly Integrated Project Team calls (OHEP/PNNL/PNSO) – More frequently as required Numerous weekly telecons within Belle II to track technical progress and address interfaces Three Belle II Collaboration meetings annually at KEK Annual Belle Physics Advisory Committee reviews – Additional reviews at behest of Belle II Project Manager Black: US Belle II Project Blue: Belle II Project

26. 19-20 March 2014US Belle II Independent Project Review26 Belle II and US Belle II have been reviewed thoroughly 8 th BPAC 9-11 February 2014 Directors Review 16-17 January 2014 Technical Design Review (KLM/Com. Det.) 13 December 2013 Technical Design Review (iTOP)/BPAC iTOP Mini Review 6-7 December 2013 Belle II Focused Review (BPAC) 9-10 September 2013 7 th BPAC 10-11 March 2013 DOE Mini Review 17 December 2012 Belle II Focused Review (BPAC) 1-2 October 2012 CD-1 Independent Project Review 26 June 2012 Peer Review 29-30 March 2012 Conceptual Design Review 15-16 March 2012 6 th BPAC 26-27 February 2012 Directors Review 15-16 December 2011 Belle II Focused Review (BPAC) 11-12 November 2011 5 th BPAC 14-15 February 2011 Black: US Belle II Project Reviews Blue: Belle II Project Reviews BPAC: Belle Physics Advisory Committee 26

27. 19-20 March 2014US Belle II Independent Project Review27 Project has responded to all prior review recommendations See backup slides for detailed responses Most important technical recommendations have been beam test of iTOP system and demonstration of integration of iTOP module – Beam test completed June 2013 – Integration demonstrated Jan 2014

28. 19-20 March 2014US Belle II Independent Project Review28 CD-2/3 requirements met RequirementDocumentationStatus Approve Updated Acquisition StrategyBelle-II Acquisition Strategy Approval at CD-2/3 Establish a Performance Baseline Belle-II Project Execution Plan Approval at CD-2/3 Approve Updated Project Execution Plan (PEP) Belle-II Project Execution Plan Approval at CD-2/3 Complete a Final Design Belle-II Technical Design Report ✔ Conduct a Final Design Review Held 6-7 December 2013 at U. Tokyo and 13 December 2013 at SLAC ✔ Complete a Final Design Report Technical Design Report Completed December 2013 ✔ Employ a certified EVMS systemN/A RequirementDocumentationStatus Perform Baseline Validation (CD- 2) and Execution Readiness (CD-3) ReviewThis review ✔ Prepare (CD-2) and Update (CD-3) Hazard Analysis Report Belle-II Hazards Analysis Report ✔ Continue with (CD-2) and Update (CD-3) Quality Assurance Program (QAP) Belle-II Project-Specific Quality Assurance Program ✔ Conduct Preliminary Security Vulnerability Assessment, if necessary (CD-2) and finalize SVA report (CD-3) Belle-II Preliminary Project Execution Plan, Section 8.8N/A Complete National Environmental Policy Act Strategy by issuing a Final Determination (i.e., EA). Categorical Exclusion (B3.6) for the US Belle-II Project ✔ 28

29. 19-20 March 2014US Belle II Independent Project Review29 Summary The project management team is organized, capable and ready to proceed Project is ready to establish the Performance Baseline The recommendations from previous reviews have all been addressed All of the CD-2/3 requirements have been met We are excited to execute the project and access the science that it will enable! 29

30. 19-20 March 2014US Belle II Independent Project Review30 BACKUP

31. 19-20 March 2014US Belle II Independent Project Review31 DOE Mini Review Recommendations Technical: 1.(Unchanged from June 26, 2012 DOE/SC CD-1 Review) See subsequent slides 2.Explore with Belle II management the possibility of shifting the iTOP installation date later by up to six months. This should be done before the next CD review. After September, 2013 BPAC Focused Review, Belle II management decided on partial iTOP installation on original schedule (spring 2015), followed by complete installation in future shutdown (summer 2017). Cost and Schedule: None Management: 3.Optimize the number of prototype quartz bars required and determine the overall vendor production rate for these materials in support of the CD-2/3 decision. Revised prototyping plan to 4 full sets of optics to evaluate vendor quality and production rates Prototype bars ordered from two vendors to compare cost/quality/production 4.Re-evaluate the current plan of requesting CD-3b in April 2013 and focus on a CD- 2/3 review request in July 2013. Eliminated CD-3b in light of technical readiness and continuing resolution 31

32. 19-20 March 2014US Belle II Independent Project Review32 CD-1 IPR Recommendations Technical: 1.WBS 1.02 (iTOP) – Further work is needed to demonstrate the performance of the counter, both in Monte Carlo and prototypes. We recommend: a.The existing prototype should be tested with a particle beam and/or Cosmic Rays over the entire phase space (azimuth, track position, and polar angle) in the presence of backgrounds similar to what is expected in Belle II. The MC should also explore performance across the phase space of tracks in the bars of Belle II. b.The prototype results should be compared to a detailed Monte Carlo simulation to ensure that all features of the counter’s performance are adequate and well simulated. 2.WBS 1.05 (Commissioning Detectors) – Redundant neutron detector systems should be considered to back-up the “Micro TPCs” See subsequent slides Cost and Schedule: None Management: 3.Evaluate the potential of increasing the upper limit of the TPC TPC raised from $14M to $15M 4.Prepare for another mini-review in September Mini Review conducted December 2012

33. 19-20 March 2014US Belle II Independent Project Review33 Mini Review and CD-1 Recommendation 1a. The existing prototype should be tested with a particle beam and/or Cosmic Rays over the entire phase space (azimuth, track position, and polar angle) in the presence of backgrounds similar to what is expected in Belle II. The MC should also explore performance across the phase space of tracks in the bars of Belle II. Beam test was conducted the first week of June 2013 Full-size bars, spherical mirror, prism, IRS and CFD readouts Data taken at [cos  =0, x=0], [cos  =0.4, x=0], [cos  =0.4, x=200mm] Cosmic Ray Test stand established at KEK Fuji hall Prototype from beam test with CFD readout installed 1b. The prototype results should be compared to a detailed Monte Carlo simulation to ensure that all features of the counter’s performance are adequate and well simulated Significant simulation progress reported at BPAC Beam test conditions simulated for comparison with data

34. 19-20 March 2014US Belle II Independent Project Review34 CD-1 IPR Recommendation WBS 1.05 (Commissioning Detectors) – Redundant neutron detector systems should be considered to back-up the “Micro TPCs” Alternatives reviewed and documented – most systems detect thermal neutrons Proton recoil only direct fast neutron technique Liquid scintillator counters are best alternative (gamma discrimination) Source direction possible using multiple cells (neutron scatter camera) however, there is not sufficient space in Belle to accommodate this He-3 thermal neutron detectors added to suite of instruments for commissioning System will be provided by new Canadian collaborators

35. 19-20 March 2014US Belle II Independent Project Review35 BPAC Review Recommendations - Quartz The committee encourages the completion of a validated bar-box design as soon as possible so that the final dimensions of the bars can be fixed. Design has matured rapidly. “Camera” mock-up assembled with all 8 PMT modules, HV and readout board stack with cable plant. Some fine tuning required, but everything fits. The committee encourages to rapidly complete the preproduction optical components so that one can better validate the component qualities and production rates, and can exercise in-house QA procedures. Bars are progressing at AOS/OOW and Zygo. Have received one from each vendor. Expect 2-4 from AOS/OOW and 4 from Zygo by end of mid-March IPR. Quality is good from AOS/OOW and extremely good from Zygo. All 4 prisms delivered by Zygo to specs and close to planned schedule. Three of 4 mirrors will be completed at ITT by end of February. Metrology from 2 of 3 complete and meet specifications. Mirror #4 is a yield loss and delivery will be made with next lot of mirrors (~5 months production time). The committee continues to encourage practicing the gluing and other assembly procedures using blanks. Similarly, the committee encouraged further testing and validation of all non-optical components used in the bar-boxes, including scratch tests of the PEEK buttons, and direct pollution tests on the quartz surfaces for the proposed adhesives during their cure cycle. Second round of glue test practice with technicians/engineers took place last week. Outgassing tests ongoing at Toray in Japan – only Araldite 2011 has GC-MS signal.

36. 19-20 March 2014US Belle II Independent Project Review36 BPAC Review Recommendations - Mechanics The committee encourages that careful prototyping and testing of all the critical components for the module assembly should be intensified, and tests made of the mechanical integrity of the assembled module, including the gas circulation and tightness, and the cooling of electronics. Mock-up of readout region (“camera mockup”) is assembled. Remainder of QBB has been tested, including use in test beam. Gas tightness test is underway at KEK – looks good. The committee encourages further study of the integration of the complete system in Belle II. The intervention procedures should be established and evaluated in terms of risk for safety and system integrity. Initial testing has been done in situ in the Belle system. Tests were mostly successful. Tooling is being modified and testing will be done again. Full-size ¼-arc test stand available for testing when Belle detector is not accessible. The committee suggests that in studying the force required to achieve good optical coupling between the wavelength filter and the prism, care should be taken to match the direction and magnitude of force applied by the spring-loaded PMT mounting in the final module. Furthermore, the committee suggests a quantitative study is made to see if a solution exists where optical coupling with small but tolerable imperfections can be produced using a substantially lower force, to obtain an additional safety margin in the design. Force required has been found to be lower than initial tests (by factor of 2-3) with new lubricating optical oil. Engineering calculations show that there is factor of 10 safety factor in strength of glue joint (prism to prism frame) that must react this load. FEA to be done. Master Bond EP37-3FLFAO selected.

37. 19-20 March 2014US Belle II Independent Project Review37 BPAC Review Recommendations - MCPPMTs As the reason for the gain variations within an MCP-PMT is not yet understood it is recommended that a fine 2D-scan of the gain across each MCP-PMT is added to the QA procedure, at least for a sub-set of units. Done. Both “standard” and “ALD” type MCPPMTs were studied. 95% show gain variation <2.5; rest will go back to HPK. The ageing behavior of the recently produced MCP-PMTs should be studied with data points added at smaller values of accumulated charge to obtain information about the reason for the shorter lifetime. Fine 2D scans of the QE and gain at different wavelengths would be helpful to monitor the rate of ageing. Underway. Aging studies take time so results will not be available until summer. Currently at ~1 C/cm 2. Will reach 4 C/cm 2 in 2 months.

38. 19-20 March 2014US Belle II Independent Project Review38 BPAC Review Recommendations - Readout Calibration of sampling times was essential for achieving the required timing resolution. Any circuit changes in IRSX related to the timing precision must be carefully verified as a priority. It should be understood if these require changes to the calibration procedure. Design changes in chip should reduce calibration complexity. Chip validation was thorough prior to submission. IRS-3D and IRS-3C as fallbacks. The issue of gain variations must be rationalized making use of the data available from MCP-PMT testing. A strategy should be developed for ensuring the correct matching of the gains of the different stages of the signal chain and the feature- extraction (timing) from saturated signals must be tested. Most channels on tested PMTs have variability less than 2.5 which is manageable with single gain setting for amplifiers. Will try to replace ~5% with large variation. Have demonstrated good timing resolution from analysis of saturated pulses. The schedule requires fast, efficient testing of the production electronics by a group of institutes. Although it was stated that groups have been identified to participate, it was not clear how much knowledge has been already transferred to them. A strong network must be created to allow good communication amongst groups, helped by appropriate training and documentation. Institutional grant funding just became clear and is very positive. Communications is good between core groups (IU, UH, PNNL, SLAC) and is expanding to new groups (e.g. S. Carolina). Hardware (board stacks) at Pitt and USC – tech transfer in process.

39. 19-20 March 2014US Belle II Independent Project Review39 BPAC Review Recommendations - Testing It is recommended that the reason for failure of some photon detector tubes in the beam test should be understood, in case there are lessons for future operation or handling of the tubes. A further beam test of the system with the final optical coupling and final electronics would be welcome to provide confidence in the solutions chosen (although this should not delay the procurement of quartz bars and other optical components). Additional beam test planned for July 2014 at SLAC. MCPPMTs likely did not fail, but rather HV arcing issues experienced around potting at front board (due to mishandling) required HV to be turned down to prevent discharge. It would be good to further improve the analytical PDF description, and show results from the other beam orientations. Studies of the effect of misalignments, or imperfect optical coupling at wide angles, would be welcome, along with the investigation of robustness against increased background. Agreed. This is a longer-term activity. Many studies were already done on misalignments (documented in summary in the Technical Design Report).