1 Report to Basic Energy Sciences Advisory Committee Committee of Visitors for Basic Energy Sciences Scientific User Facilities Division April 15-17, 2007

2 Background Second COV Review of Facilities Division Present panel assesses operations of Division’s programs during FY 2004, 2005, Examines any files from this period. Division components for review: - Synchrotron-Light and Electron-Beam Light Sources - Neutron Sources - Electron-Microscopy Sources - Nanoscale Science Research Centers - Accelerator and Detector Research At BES/DOE Germantown Reporting: Presentation to BESAC at Summer meeting. Following report acceptance by full BESAC committee, COV report to be presented to Director of Office of Science.

3 Recommendations - First COV Jacket format: Timelines of the Review history for each Facility or Center. Each Timeline should take the form: –Review–>Recommendations–>Results (including written response to COV/BESAC)[–>Re-Review and its Results, when necessary] –History should be In front of jacket of most recent review for each facility and a brief overall review history for the facility. Cross- references to the full jacket for previous reviews are also useful. Several elements should be contained in the report and file of every BES facility review; these were noted in the report. Review process: Evaluation of the success of facilities should be done on the basis of quantifiable metrics – these may vary with type of facility.

4 Recommendations -First COV (Cont’d) A revision of the review process, e.g. more executive sessions, more time to hear about related lab issues. Strongly recommended broad users’ input at all stages of construction of five centers, since they are designated as national user facilities. The committee felt strongly that it was crucial to have a clear and current definition of who exactly is a user. Consensus report may be best. Facilities Division operation Careful attention to coordination between the two major science program divisions and the Scientific User Facilities Division; strongly recommend that science program managers participate in facility reviews. Careful thought to integration of nanocenter science across centers and with core programs.

5 DOE/BES Committee of Visitors (COV) Review Panel Overall Chair Richard Osgood, Columbia* BESAC Chair John Hemminger attends 1 st day of review. Neutrons (4) Pat Gallagher, NIST John Tranquada, BNL Sunny Sinha, San Diego James Rhyne, LANL Nanoscience (4) Dave Litster, MIT Miquel Salmeron, LBL Reginald Penner, UC - Irvine Franz Himpsel, Wisconsin X-rays/ Machines (4-5) Science Tai-Chang Chiang, UICC Gabrielle Long, Argonne* Brent Fultz, Caltech Z-X Shen, Stanford/LBL* Machine Persis Drell, SLAC Don Bilderback, Cornell Janos Kirz, LBL Sam Krinsky, BNL Microscopy (3) John Silcox, Cornell Cev Noyan, Columbia Miquel Salmeron, LBL *Present BESAC Member, * Former BESAC Member

6 Charge to the Committee Panel will consider and provide evaluation of following four major elements: 1. Assess efficacy and quality of processes used to: (a) solicit, review, recommend, and document proposal actions (b) monitor active projects, programs and facilities. 2. How has the award process affected: (a) breadth and depth of portfolio elements (b) national and international standing of portfolio elements. Also Provide input for OMB evaluation of Basic Energy Sciences progress toward long- term goals. Each of the components of the Scientific User Facilities Division should be evaluated against each of the four-part long-term goals. If not applicable, please indicate so. Note OMB guideline ratings of (1) excellent, (2) good, (3) fair, (4) poor, (5) not applicable. Also, comment on observed strengths or deficiencies in any component or sub-component of the Division’s portfolio, and suggestions for improvement.

7 Introduction Charge Committee Composition Response to Prior Review Review  COV Review Process  Facility Review Process  Reports on Specific Classes of Facilities (Neutrons, X-rays, Electron Microscopy, Nanocenters, and Detectors and Accelerators) – Response to Prior Review – Documentation of Facility Review – Users of Facilities- Definition of and Uniqueness for Each Facility Type – Metrics – Comments Facility Review Process Comments on Emerging Facilities Metrics General Comments Managing the SUFD Vision PART Evaluation Conclusions Report

8 Summary COV concludes that the newly constituted Scientific User Facilities Division is well launched and is operating extremely well. Facility reviews are fair and even-handed and had significant and clear beneficial impact on several facilities, even though many of the facilities are just now reaching the point of operational review. COV finds that review process has served existing facilities well. In some cases reviews have promoted changes in management and operations and improved the scientific impact of these facilities. The reviews have added clarity and focus to wide spectrum of concerns from user community, facility personnel, and the BES. COV made specific recommendations for improvements and changes in review process, both in general and in terms of specific facility types. The Committee was satisfied that the Division is operating well and expects further definition and refining of the review process as SUFD matures. The Committee continues to urge very careful attention to the coordination of the two major science-program divisions (Materials Sciences and Engineering and Chemical Sciences, Geosciences and Biosciences) with the Scientific User Facilities Division; healthy growth of the BES organization will necessitate balance between these two organization units. The committee gave PART ratings of 1) Materials Research - Excellent, 2) Chemistry - Excellent, 3) Energy Research - N/A, and 4) Instrumentation - Excellent.

9 Recommendation: BES should adopt a practice of having a separate document summarizing the reviewers’ comments (Executive Summary) and a letter detailing actions requested by BES following the review. Recommendation: A timeline of actions and reviews of each facility should be attached to the cover of each review jacket. Recommendation: The previous COV report and BES response should be distributed to the next COV prior to their meeting. COV-Review Recommendations

10 Recommendation: The overall basic review system works exceptionally well, do not change it. But perhaps tweak it! Recommendation: The Facilities Division staff should begin to plan for an improved strategy for the review process of the largest facilities. Recommendation: To the extent possible, the results of the review should be provided in a timely fashion. In addition, the comments of the reviewers should be summarized separately from the letter containing requested actions by the SUF Division Director. Recommendation: The Committee recommends that the planned increase by 5 in the SUFD staff proceed promptly; it is needed for a well managed facilities program. In addition, allowance for increased travel, i.e. funds and time, to facilities to encourage informal evaluation of facilities should be made. Recommendation: The Committee recommends that each SUFD review explicitly discuss collaborations between core-research programs and SUFD operations. Facilities-Division-Review Recommendations

11 Recommendation: Institute a uniform, integrated, and transparent proposal system for all five NSRCs. Recommendation: Include explicit time for facility research and instrumentation review in each SUFD review. Recommendation: The committee does not recommend the adoption of a single cost metric; such a metric would not be an effective management tool and its use would lead to poor management behavior. Recommendation: SUFD should plan to discuss in more detail its strategy for developing theory at the full complement of BES facilities during the next COV. General Recommendations

12 Examples of Facility-Specific Comments: Nanocenters Users: Definition and Uniqueness for Each Facility “There may be categories of users who are not always physically present when they use the facility resources. With increased operating experience the NSRCs should develop methods to measure appropriately the services they provide to users. Especially in the NSRCs, a high-quality internal- research program is vital to ensure that the center offers state–of-the-art facilities. While this internal program can create some tension between the user demands and the internal program needs of the center, this tension should be treated as a part of the cost of operation and thus managed to optimize both goals.” Metrics “ The distinctive character of the NSRCs makes careful consideration of their performance metrics essential.” Response to Prior Review “Centers created so far have all been developed with broad users’ input via user workshops, as was recommended by the previous COV.” “The integration of the NSRCs with the core-research programs of the DOE and of the host National Lab. …. should form an element of reviews of the internal research at the centers.” Documentation of Facility Review “For the operational NSRC, the reviewers wrote detailed, constructive and frank reviews that were summarized very well by the program officer.”

13 Other Facility-Specific Comments X-rays: Include metric on remote users; impressed by future-light-source planning Electron Microscopy: Challenge is to convert from support to user center mode; simulation and theory vital Accelerators and Detectors: Vital need Neutrons: Informal feedback to BES important; accelerator program vital Accelerator and detector program discussed in “Managing SUFD Vision”.

14 Thanks to the COV members for generously giving their time and thanks to Pat, Pedro, and the staff for providing every assistance!