NIFA Reporting Web Conference December 10, 2009

Start the Recording…

Bart Hewitt Bart is the Accountability and Reporting Leader for the Office of Planning and Accountability. He leads the national effort for the State Plan of Work and Annual Report process under the Agricultural Research, Extension and Education Reform Act; manages the Multistate Research Fund Project Proposal approvals for NIFA; and develops performance information for the PART and NIFA Budget. (202) questions to

User Support (202) or Do not contact Texas A&M support FAQs and other information on the CSREES Reporting Web Conference web page at questions to

Format and Logistics questions to topic suggestions to Conferences are recorded and will be available on the Reporting Web Conference web page at questions to

To Receive Announcements An RWC list will notify interested parties on news, schedules, and other issues relating to the series. To subscribe: –Send an to –Skip your subject line and in the body of your message type: subscribe reportingwc. –Be sure you receive an confirming your subscription. questions to

Introduction to Plan of Work December 2009

Plan of Work – A Historical Perspective 1984 – 1991 NARS 1992 – 1995 PPARS 1994 – 1996 PPARS GPRA Pilot 1996 and 1997 PPARS Transition 1998 and 1999 GPRA Planning and Reporting System Present - AREERA questions to

Plan of Work A Historical Perspective NARS –Fully Narrative –Structured –Attempted to cover the total Extension Portfolio questions to

Plan of Work A Historical Perspective PPARS –Focused on Targeted Programs –Narrative and Numeric Data –DOS Based Computer Template Relational Database questions to

Plan of Work A Historical Perspective PPARS – GPRA Pilot –1994 – 1996 –Five States –Selected Targeted Programs –Targets for Each Year and a Baseline questions to

PPARS Components State Program Overview Targeted Programs Civil Rights questions to

State Program Overview Narrative Described the total Extension Program in the State Focused on Base Programs questions to

State Accomplishment Overview Narrative Executive Summary required each year Stressed program results and documented benefits Reflected significant achievements of the Base Programs questions to

Targeted Programs National Initiatives –Food Safety and Quality –Plight of Young Children –Communities in Economic Transition –Sustainable Agriculture –Waste Management –Youth at Risk –Decisions for Health 3(d) and Specially Funded Items –EFNEP –Farm Safety –Integrated Pest Management –Pesticide Applicator Training –Pesticide Impact Assessment –Renewable Resource Extension Act Other Programs Dislocated Farmers Assistance Grants Urban Gardening Program Conservation and Forestry Titles of the 1990 Farm Bill Environmental Education Limited Resource/Low Income Individuals and Families Multicultural Diversity Parenting and Family-Youth Programs Sea Grant/Marine Extension Programs Volunteer Development and Management questions to

Targeted Programs – POW Situation Statement Research and Empirical Base Objectives and Indicators –Data Collection Methodology Target Audience Implementation Plan –Staff Training –Internal and External Linkages –Volunteers –Key Components –Reaching Target Audience(s) –Mileposts Estimated Program Cost Estimated FTEs Number of Volunteers Plan of Work Contact questions to

Targeted Programs Annual Report Narrative Overview Success Stories Exemplary Programs Objectives and Indicators Estimated Program Costs Estimated FTEs Number of Volunteers State Contacts questions to

PPARS - Sample Indicator POW Proj Number of Participants Number Adopted Total questions to

PPARS – GPRA Pilot Year# of Participants Instructed # of Participants who became pregnant # who became pregnant in a comparable population Baseline 1993 TargetActualTargetActual questions to

Civil Rights Plan Goals and Procedures –Equal Employment Opportunity –Program Delivery –Public Notification –Civil Rights Training –On-site Compliance Reviews Population and Clientele Projections Reporting Options –Total (100%) Data Collection –Rotational Reporting –Independent Survey questions to

Civil Rights Annual Report Narrative Accomplishment Summary Success Stories Goals and Procedures –Equal Employment Opportunity –Program Delivery –Public Notification –Civil Rights Training –On-site Compliance Reviews Population and Clientele Projections questions to

GPRA Planning and Reporting System Based on PPARS GPRA Pilot database system Used 5 GPRA Goals instead of Targeted Programs Used GPRA Objectives Used Targeted Indicators which could be aggregated across the nation questions to

AREERA Agricultural Research, Extension, and Education Reform Act of 1998 Plan of Work Annual Report of Accomplishments and Results Multistate Extension Integrated Research and Extension Multistate Research name questions to

2000 – 2004 Plan of Work Arranged around the 5 GPRA goals established by USDA when the 5 year Plan of Work began Unstructured text driven by the states’ perceived need for maximum flexibility and an aversion to a database driven template Annual Reports began using Key Themes as a means to try to gain some order to the unstructured text data questions to

2005 – 2006 POW Update Submitted to Old CSREES by April 1, 2004 Extended the current Plan of Work Cycle to include FY 2005 & – 10 page document –States could add any relevant information not included in 2000 – 2004 Plan of Work –Outlined changes in focus over two year period from the original Plan questions to

Original Plans for New POW 2007 – 2011 (Rolling 5 Year Plans) New plan was due June 1, 2006 Proposed written guidance published POW Cycle in June 2005 in Federal Register 30 Day Comment Period Final Guidelines published January questions to

Initial Plan of Work Software Web-based and database driven Pilot testing in three phases –September –October Released on November 16, 2005 –“Sandbox” version Production Version Released in January questions to

Structure of 5-Year POW 1.Plan Overview Section 2.Planned Programs Section 3.Multistate Extension and Integrated Research and Extension (Sections 105 & 204 of AREERA) questions to

Structure of 5-Year POW 1.Plan Overview Section Plan Overview Text Overall FTEs/SYs Stakeholder Input Merit Review (Program Review) Evaluation of the Multis & Joint Programs questions to

Structure of 5-Year POW 2.Planned Programs Section Name of State Defined Program Unit Knowledge Area Classification Program Duration Basic Logic Model Format Evaluation Strategy questions to

Structure of 5-Year POW 3.Multistate Extension and Integrated Research and Extension (Sections 105 & 204 of AREERA) Base Percentage Target Percentage for All Future Years Plan Target Dollars for Duration of Plan questions to

Plan of Work – A Key Component of the One Solution How does the Plan of Work relate to the One Solution Initiative? –Part of the larger One Solution –Web-based system for Formula Funds –Same goals as One Solution questions to

One Solution Initiative Goals of One Solution –Simplify Reporting –Reduce burden on Partners and NIFA –Improve quality of accountability data –Meet increasing performance & budget reporting expectations questions to

Plan of Work Goals Why do we have a new Plan of Work Electronic System? –Responds to AREERA –Opportunity to reduce reporting burden over time –Maximize usefulness of information –Improve Plan of Work accountability –Improve Quality of the data received from States –Meet increasing performance & budget reporting expectations Agency Program management OMB and USDA Congress questions to

Plan of Work – What is Covered? What State work must be included in the Plan of Work? –NIFA Formula and required Matching Funds Federal Funds: –Hatch –Smith-Lever 3(b) and (c), [but NOT 3(d)] –1890 Extension –Evans-Allen All required non-federal matching funds questions to

Plan of Work – Requirements Approved Plan of Work is an integral requirement for release of funds to institutions A completed plan needs to include: –Stakeholder Input Documentation –Peer Review for Research –Merit Review for Extension –Multi-state Extension and Integrated Research and Extension Financial Data –Planned Programs questions to

Logic Model Why organize Planned Program data around the Logic Model? –Conceptual depiction of key elements of any program or project –Endorsed by GAO –Commonly used by: Federal Research Agencies Private Research Organizations Cooperative Extension Services –Useful for planning, describing & implementing programs, and designing evaluations questions to

Planned Programs Structure Each Planned Program in the Plan of Work contains these key components –Situation What problems, needs, or opportunities will the program address? –Assumptions Explains why particular activities to address situation were chosen Validated by research, literature and experience –External Factors That can help or hinder results, and alternative explanations –Inputs What assets and resources will be applied? –Activities/Outputs What program activities do you plan to do, who will you serve? What are some of the products produced? –Outcomes What changes in Knowledge, Action, and/or Condition are expected. What changes in Knowledge, Action, and/or Condition will be measured? –Evaluation What Types of Evaluation is Planned? What methods will be used? questions to

Occur when a societal condition is improved due to a participant’s action taken in the previous column. For example, specific contributions to: - Increased market opportunities overseas and greater economic competitiveness - Better and less expensive animal health - Vibrant & competitive agricultural workforce - Higher productivity in food provision - Better quality-of-life for youth & adults in rural communities - Safer food supply - Reduced obesity and improved nutrition & health - Higher water quality and a cleaner environment Generic Logic Model for NIFA Reporting NIFA – Office of Planning & Accountability (This model is intended to be illustrative guide for reporting on NIFA-funded research, education and extension activities. It is not a comprehensive inventory of our programs.) Outcomes Actions InputsSituationActivities Knowledge What we invest: - Faculty - Staff - Students - Infrastructure - Federal, state and private funds - Time - Knowledge - The collection of stakeholder opinions Occurs when there is a change in knowledge or the participants actually learn: - New fundamental or applied knowledge - Improved skills - How technology is applied - About new plant & animal varieties - Increased knowledge of decision-making, life skills, and positive life choices among youth & adults - Policy knowledge - New improved methods Description of challenge or opportunity - Farmers face increasing challenges from globalization - Opportunity to improve animal health through genetic engineering - Insufficient # of trained & diverse professionals entering agricultural fields - Youth at risk - Invasive species is becoming an increasing problem - Bioterrorism - Obesity crisis - Impaired water quality External Factors - A brief discussion of what variables have an effect on the portfolio, program or project, but which cannot be changed by managers of the portfolio, program, or project. For example, a plant breeding program’s success may depend on the variability of the weather...etc. Occur when there is a change in behavior or the participant’s act upon what they’ve learned and: - Apply improved fundamental or applied knowledge - Adopt new improved skills - Directly apply information from publications - Adopt and use new methods or improved technology - Use new plant & animal varieties - Increased skill by youth & adults in making informed life choices - Actively apply practical policy and decision-making knowledge Conditions Assumptions - These are the premises based on theory, research, evaluation knowledge etc. that support the relationships of the elements shown above, and upon which the success of the portfolio, program, or project rests. For example, finding animal gene markers for particular diseases will lead to better animal therapies. What we do (Activities): - Design and conduct research - Publish scientific articles - Develop research methods and procedures - Teach students - Conduct non-formal education - Provide counseling - Develop products, curriculum & resources Who we reach (Participation): - Other scientists - Extension Faculty - Teaching Faculty - Students - Federal, state & private funders - Scientific journal, industry & popular magazine editors - Agencies - Policy and decision- makers - Agricultural, environmental, life & human science industries - Public Outputs Version New fundamental or applied knowledge - Scientific publications - Patents - New methods & technology - Plant & animal varieties - Practical knowledge for policy and decision-makers - Information, skills & technology for individuals, communities and programs - Participants reached - Students graduated in agricultural sciences

Plan of Work – Benefits –Reduced reporting burden on institutions –Easier for Partners to review what other states are doing POWs are Published in REEIS Selected Data are published in the Leadership Management Dashboard –Easier for both Partners and NIFA to identify performance measures & track progress with structured format –States will receive better and more timely feedback on their Plan of Work and Annual Report from NIFA questions to

POW– Benefits for NIFA –More efficient & timely plan reviews by our NPLs –Database queries make it easier to locate & analyze data –Annual Reports will Provide supporting documentation for Portfolio reviews, PART, budget submission, other external requirements –More efficient and accurate linkages to USDA Strategic Plan (Goals and Objectives) by linking Knowledge Areas to portfolios questions to

Plan of Work Data Use How does NIFA use the information from the Plan of Work for planning and accountability? –NPL State Liaison Communication –Portfolio Reviews –OMB PART process –Budget Performance Integration –GAO and OIG inquiries –Answer Congressional & Departmental inquiries Brings greater visibility of successes of Formula Funded Programs questions to

Evaluating Research & Education Efforts: Portfolio Approach –OMB demands led to development of the portfolio assessment tool and measures; –Portfolio analysis used to assess progress toward goals; guide RFAs/POWs –Tracking papers, citations, patents, products, educational efforts, adoption of products/ practices –Uses OMB R&D criteria (relevance, quality, performance) questions to

Evaluating Research & Education Efforts: Portfolio Approach –Knowledge Areas (KA’s) for each Planned Programs –KAs for each Outcome in Annual Report –KAs are the link between Plan of Work and Portfolio Reviews questions to

Cascading Alignment Portfolio Knowledge Area code Goal Mission Strategic Objective Planned Program/POW

Plan of Work Evidentiary materials for Portfolios Strategic and evaluation plans Tracks papers, citations, patents, products, educational efforts, adoption of products/ practices Evaluation studies and special analyses Input tables to show portfolio priorities and emphasis questions to

Hints for the 2009 Annual Report

Outputs and Outcomes Limit Outputs and Outcomes –Report Outcomes that stand out Most useful for our reports –Qualitative Outcome Statements in conjunction with Quantitative Measure questions to

What does a Quality Outcome Statement Look Like? Brief Effective Impact Statements: –Provide quantifiable evidence of change or difference the program made. (It really is all about the money. Bucks are the gold standard. Audiences want to know the return of investment.) –Provide only enough detail to be easily understood. –Highlight public benefits, outcomes, payoffs. questions to

Sample Qualitative Outcome Issue: Producers, seedsmen, grain merchandisers, processors, crop consultants, plant breeders, and extension staff are interested in new cultivars that bring them increased revenue. questions to

Sample Qualitative Outcome What has been done: The new varieties Faller Wheat, Lariat and Stampede pinto beans, Sheyenne non-transgenic soybean, RG7008RR soybean, and Pinnacle two-row barley were released for use. questions to

Sample Qualitative Outcome Results: The estimated dollar value to producers, seedsmen, grain merchandisers, processors, crop consultants, and plant breeders on these new varieties is $290,600,000 for Moreover, because of best management practices developed by research and extension, wheat and barley producers reduced economic losses by $40,000,000 through use of better varieties of wheat and through use of fungicides. questions to

Types of Outcomes Change in Knowledge Occur when there is a change in knowledge or the participants actually learn: –New fundamental or applied knowledge –Improved skills –How technology is applied –About new plant & animal varieties –Increased knowledge of decision-making, life skills, and positive life choices among youth & adults –Policy knowledge –New improved methods questions to

Types of Outcomes Change in Action or Behavior Occur when there is a change in behavior or the participant’s act upon what they’ve learned and: –Apply improved fundamental or applied knowledge –Adopt new improved skills –Directly apply information from publications –Adopt and use new methods or improved technology –Use new plant & animal varieties –Increased skill by youth & adults in making informed life choices –Actively apply practical policy and decision- making knowledge questions to

Types of Outcomes Change in Condition Occur when a societal condition is improved due to a participant’s action taken (Change in Action). For example, specific contributions to: –Increased market opportunities overseas and greater economic competitiveness –Better and less expensive animal health –Vibrant & competitive agricultural workforce –Higher productivity in food provision –Better quality-of-life for youth & adults in rural communities –Safer food supply –Reduced obesity and improved nutrition & health –Higher water quality and a cleaner environment questions to

Types of Outcomes Needed Outcomes related to USDA Priorities: –Global Food Security and Hunger –Climate Change –Sustainable Energy –Childhood Obesity –Food Safety

Brevity and Conciseness in the Executive Summary The Executive Summary is a brief overview narrative of your total program. Copy and paste text into this field from your current state’s yearly brief publication. Two to five pages should suffice. Highlights of your State program Let the Planned Programs section attend to detail. questions to

Future of the Plan of Work? Farm Bill Requirements Expert Panel Meets May 2010 –Chosen through regional executive directors –Two panelists from each region (Total 11 chosen) –5 panelists from NIFA –Make recommendations for further streamlining

Plan of Work Additional Information –NIFA Plan of Work Web Page –Plan of Work Training Page –Contact Information Bart Hewitt, Accountability and Reporting Leader questions to

Questions? questions to

See you in February! Next NIFA Reporting Web Conference is scheduled for Thursday, February 11 from 2-4 pm (Eastern) topic suggestions for February to Visit the conference web site at for: –The recording of this conference –The slides from this conference –Announcements questions to