Office of Research and Development November 7 th, 2008 Randy Wentsel National Program Director Moving Toward Research Excellence Presentation to ORD Managers.

Slides:

Advertisements

Similar presentations

CDCs 21 Goals. CDC Strategic Imperatives 1. Health impact focus: Align CDCs people, strategies, goals, investments & performance to maximize our impact.

Advertisements

Developing an Evaluation Strategy – experience in DFID Nick York Director – Country, Corporate and Global Evaluations, World Bank IEG Former Chief Professional.

Overview of Mission Statement & Organization. Development Communication Division External Affairs Vice Presidency DevComm Vision To put communication.

SAI Performance Measurement Framework

High level expert meeting to develop the Near East Regional Action Plan to Implement the Global Strategy to improve Agricultural and Rural Statistics.

Alberta’s Strategy for Sustainability presented to Prairie Water Policy Symposium Beverly Yee, Assistant Deputy Minister Alberta Environment September.

The Budget The PART Wait! Don’t Leave! ORD Managers Meeting January 24, 2006 Howard Cantor.

Caird E. Rexroad, Jr. Wilbert H. Blackburn.  ARS is a matrix organization comprised of National Program Staff who establish national research program.

How Country Stakeholders Get Involved Group Exercise June 2013 MONITORING AND EVALUATION IN THE GEF.

1 LBNL Enterprise Computing (EC) January 2003 LBNL Enterprise Computing.

Leadership and Strategic Planning

PHAB's Approach to Internal and External Evaluation Jessica Kronstadt | Director of Research and Evaluation | November 18, 2014 APHA 2014 Annual Meeting.

COORDINATED AUDITS IN AFROSAI-E Presented By Josephine Mukomba.

Office of Science & Technology Policy Executive Office of the President The National Climate Assessment Version 3.0 Kathy Jacobs Assistant Director for.

HEALTHY KIDS LEARN BETTER A Coordinated School Health Approach.

Session 853 Developing a Framework that Integrates Program Design, Management, Accountability & Evaluation Session 853 Extending Organizational Capacity.

EVALUATION IN THE GEF Juha Uitto Director

MAFMC 5-Year Strategic Plan Public Comments August 14, 2013.

Allen Hepner Senior Planning & Performance Manager September 22, 2011

Live Healthy Napa County Creating and Sustaining a Common Agenda.

Organization Mission Organizations That Use Evaluative Thinking Will Develop mission statements specific enough to provide a basis for goals and.

Region III Activities to Implement National Vision to Improve Water Quality Monitoring National Water Quality Monitoring Council August 20, 2003.

Organize to improve Data Quality Data Quality?. © 2012 GS1 To fully exploit and utilize the data available, a strategic approach to data governance at.

Evaluation in the GEF and Training Module on Terminal Evaluations

1 Implementation of the National Academies’ Program Recommendations: NIOSH PPT Program Maryann D’Alessandro, Ph.D. Associate Director for Science, NPPTL.

1 Introduction to Evaluating the Minnesota Demonstration Program Paint Product Stewardship Initiative September 19, 2007 Seattle, WA Matt Keene, Evaluation.

Evaluation Assists with allocating resources what is working how things can work better.

Copyright 2010, The World Bank Group. All Rights Reserved. Planning and programming Planning and prioritizing Part 1 Strengthening Statistics Produced.

ADD Perspectives on Accountability Where are We Now and What does the Future Hold? Jennifer G. Johnson, Ed.D.

Performance Assessment Assessment of Organizational Excellence NSF Advisory Committee for Business and Operations May 5-6, 2005.

Progress on Coordinating CBP and Federal Leadership Goals, Outcomes, and Actions Principals’ Staff Committee Meeting 2/16/12 Carin Bisland, Associate Director.

Forging Partnerships on Emerging Contaminants November 2, 2005 John Vandenberg Associate Director for Health National Center for Environmental Assessment.

Technology Transfer Execution Framework. 2 © 2007 Electric Power Research Institute, Inc. All rights reserved. Relationship Between Your EPRI Value and.

Considerations for CIHR’s New Open Program. The Context Bottom up Strategy Reform of Open Suite of Programs Full spectrum of CIHR mandate Top Down Strategy.

Introduction to National Program Directors Randy Wentsel Office of Research and Development January 24, 2006 Randy Wentsel Office of Research and Development.

EPA’s Role in the Global Earth Observation System of Systems (GEOSS)

Session 853 Extending Organizational Capacity and Capability to Evaluate Federal Environmental Research Programs Howard Cantor U.S. EPA Office of Research.

EPA Geospatial Segment United States Environmental Protection Agency Office of Environmental Information Enterprise Architecture Program Segment Architecture.

Office of Research and Development National Exposure Research Laboratory, Atmospheric Modeling and Analysis Division S.T. Rao Director, NERL/AMAD U.S.

1 Office of Research and Development. 2 Confronting Today’s Reality Historically, ORD has targeted it’s research to provide essential science/products.

UNCLASS1 Dr. Gene Whitney Assistant Director for Environment Office of Science and Technology Policy Executive Office of the President WISP Meeting - July.

1 NOAA Priorities for an Ecosystem Approach to Management A Presentation to the NOAA Science Advisory Board John H. Dunnigan NOAA Ecosystem Goal Team Lead.

Sustainability and Indicators at EPA Ecoinformatics Meeting June, 2007 Copenhagen William Sonntag w/credit to Ethan McMahon Office of Environmental Information.

Introduction to SEPA The Scottish Environment Agency For CaSPr Waste Workshop Glasgow 19 October 2006 Claudette Hudes NetRegs Team Leader.

Office of Science U.S. Department of Energy U.S. Department of Energy Office of Science Jerry Elwood Director, Climate Change Research Division, Office.

Integrating Evaluation into the Design of the Minnesota Demonstration Project Paint Product Stewardship Initiative St. Paul, MN May 1, 2008 Matt Keene,

Overview of OPASI’s Division of Evaluation and Systemic Assessments (DESA) Deborah Guadalupe Duran, Ph.D. Chief, Systemic Assessments Branch.

Overview of Pesticides and Toxics Research Programs: Safe Pesticides/Safe Products and Endocrine Disruptors Elaine Z. Francis, Ph.D. National Program Director,

California Water Plan Update Advisory Committee Meeting January 20, 2005.

19 October 2004Enterprise Architecture in WSRP Portal 1 Foreword: Building Enterprise Architecture Through WSRP in Sample EPA Regional Portal FEA Goals:

The United States Foreign Assistance Reforms: An Overview.

Advisory Forum, July 2005 Outcome of the first retreat of ECDC Management Team (EXC) 4-5 July 2005 Krägga Herrgård Zsuzsanna Jakab Director ECDC.

1 A Multi Level Approach to Implementation of the National CLAS Standards: Theme 1 Governance, Leadership & Workforce P. Qasimah Boston, Dr.Ph Florida.

Office of Research and Development Photo image area measures 2” H x 6.93” W and can be masked by a collage strip of one, two or three images. The photo.

Collaborative Decision Making: What, When, Why, How?

1 “Good Practices in Managing for Results” Workshop Santiago, Chile October 27 th and 28 th, 2010 Benjamin Nelson Managing Director for Quality Office.

NIH Change Management Program Change Management Program Overview March 8,

New Ecological Science Advice for Ecosystem Protection The EPA Science Advisory Board (SAB) Staff Office supports three external scientific advisory committees.

K. Bruce Jones EPA Office of Research and Development U.S. EPA Science Advisory Board Regional Vulnerability Assessment Advisory Panel Meeting October,

Using Analysis and Tools to Inform Adaptation and Resilience Decisions -- the U.S. national experiences Jia Li Climate Change Division U.S. Environmental.

MONITORING AND EVALUATION IN THE GEF

The GEO-6 Matrix Drafting Approach

Research Program Strategic Plan

Evaluation in the GEF and Training Module on Terminal Evaluations

CIS progress June-November 2011: main highlights

MONITORING AND EVALUATION IN THE GEF

CBP Organizational Structure

MONITORING AND EVALUATION IN THE GEF

MONITORING AND EVALUATION IN THE GEF

MONITORING AND EVALUATION IN THE GEF

Presentation transcript:

Office of Research and Development November 7 th, 2008 Randy Wentsel National Program Director Moving Toward Research Excellence Presentation to ORD Managers

Office of Research and Development 1 Overview of Briefing Role of NPDs Research excellence Decision analysis tools Evaluation of our research Communication of our research Accountability Q & A

10/24/20152 Role of National Program Directors Lead planning and development of research programs Cross laboratory view for shifts in program Primary interface with program office and regions Scientific leader - responsible for scientific direction for specific areas of the research program Report to DAA for Science

10/24/20153 Planning and Development of Research Programs Lead Research Coordination Team (RCT) or similar group Lead annual research planning Lead drafting of Multi-year Plans (MYPs) Lead Board of Scientific Counselors reviews Lead PART review Lead science discussions with SAB

10/24/20154 How ORD Evolves its Research Program NPDs Decide What Research Area-Specific Work We Do and When We Do It Planning the Program Decision Inputs Programs and Regions (RCTs) EPA Strategic Plan Administration’s priorities Congressional mandates BOSC Reviews SAB, NAS, other external advice Stakeholders NPDs, SC, MC, EC ORD Executive Council Corporate Decisions on What We Do... and... How We Do It Evaluation Program and Regional Office Feedback BOSC Program Evaluations NAS, NAPA, and other advisory bodies PART Reviews Implementing the Program L/C Directors Decide How ORD Produces its Research Products NPDs Responsible for Communicating Products to Clients L/C Directors Responsible for Developing ORD’s Research Products

10/24/20155 ORD Programs and NPDs Goal 1 - Air (Dan Costa) NAAQS Air Toxics Goal 2 - Water Drinking Water (Audrey Levine) Water Quality (Chuck Noss) Goal 5 - Stewardship Sustainability & Biofuels (Alan Hecht) Goal 3 - Land (Randy Wentsel) Land Protection & Restoration Goal 4 – Safe Communities and Healthy Ecosystems Homeland Security (Greg Sayles) Human Health (Sally Darney) & Ecosystems (Rick Linthurst) Human Health Risk Assessment (John Vandenberg) Computational Toxicology (Robert Kavlock) Endocrine Disrupting Chemicals (Elaine Francis) Global Change (Joel Scheraga) Pesticides & Toxics (Elaine Francis) Nanotechnology (Jeff Morris)

10/24/20156 The Formula for Research Quality Quality Scientists/Engineers Creative, Innovative Broadly Trained Motivated Quality Facilities/Equipment Conductive to Quality Output Attracts High Quality S&E’s Quality Leadership/Management Environment for Innovation Stress Team-Work Provides Relevance and Timelines Quality Research Sustained Quality Leads to a Brand or Go To Status Potential to Make a Real Difference

10/24/20157 The Formula for Relevant Research Meets Needs of Decision Makers Multiple levels of communication Provides high quality science Informs decisions Timely Knowledge Transfer Meets client schedule Responsive to requests for assistance Enables Tech Transfer Strong Leadership/Management Environment for collaboration with customers Focus on outcomes Relevant Research Sustained Relevance Leads to a Brand or Go To Status Makes a Real Difference

10/24/20158 Research Excellence = f(Technical Quality x Relevance) Increasing excellence Job shopping Ivory tower Relevance Technical Quality Research Excellence Research Excellence – How To Measure It?

10/24/20159 Implementing an ORD Evaluation Framework Research Activities Research Outputs Short-Term Outcomes Intermediate Outcomes Long-Term Outcomes Resources Strategic Goals Strategic Objectives Research ProgramEnvironmental Outcomes and Results Clients Independent Expert Evaluation: Focus:Outcome-oriented progress; R&D investment criteria Evidence: MYPs; synthesis products; performance data; client feedback Attribution: Sphere of influence that includes ORD, EPA clients & partners Policy Development

10/24/ BOSC Program Evaluations  Help answer the questions: Are we doing the right science? Are we doing the science right?  Provide guidance for evolving the research program  Provide evidence for OMB evaluations using the Performance Assessment Rating Tool

10/24/ Program Assessment Rating Tool (PART) – Required program “audit” by OMB is likely to change, but not go away – Emphasis has been on 4 areas: Purpose/Design, Strategic Planning, Program Management, and Program Results – Ratings/scores based largely on meeting annual and long-term performance goals with emphasis on outcomes, and feedback from external program evaluations like BOSC

10/24/ Decision Analysis Tools Value of Information is a form of decision analysis that evaluates the benefits of information for reducing uncertainty in a decision making context Multi-Criteria Decision Analysis reduces complex problems to a singular basis in order to select a preferred alternative These tools may be applied to research planning and/or communicating the value of our research to BOSC, OMB, Congress 4

10/24/ Communication of our research accomplishments Performance Measurement and Evaluation –Suite of metrics for quality (bibliometrics), relevance (client citation), and performance (client use) –Outputs and outcomes, annual and long-term –Partner surveys = feedback and areas for improvement –Independent peer review = evaluation and improvement strategies –Informs decisions = impact on human health and the environment

10/24/ Communication Challenges Issues –Complete database of ORD publications that partners can use –Demonstrating use/impact of research – faster, cheaper, better science –How do we document impacts? –Value of ORD research To who? –Scientific community –Partners (POs, regions, user community) –Congress, OMB, policymakers

Performance Management Challenges – How can we improve our effectiveness as a team (line mgt & NPDs)? – What are methods to improve our communication to: the scientific community, partners (POs, regions, user community), Congress, OMB, and policymakers? – Death by Accountability--GPRA/BOSC/PART/Division-Reviews- -How can we measure and evaluate performance, and ensure that ALL individuals are working toward the mission? – If ORD is to place emphasis on integrated, multidisciplinary programs, what 2-3 specific actions can division directors and NPDs take together to foster such programs?