OUR MISSION NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS NASCSP NATIONAL ASSOCIATION FOR STATE COMMUNITY SERVICES PROGRAMS ROMA Next Generation NASCSP | Moving Forward with the Performance Management Framework

NASCSP | ROMA Next Generation Presenters Katy Kujawski, Research Director, NASCSP Jeannie Chaffin, Consultant, NASCSP NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION Agenda Review the Performance Management Framework: State and Federal Accountability Measures Organizational Standards CSBG State Plan and Annual Report Explore ROMA Next Generation: What is ROMA NG? Theory of Change (TOC) CSBG Annual Report and the change from the CSBG IS Survey Community Level Indicators Analysis Feedback for CAAs NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION ABOUT NASCSP | What we do NASCSP members are state administrators of the U.S. Department of Health and Human Services Community Services Block Grant (CSBG) and the U.S. Department of Energy's Weatherization Assistance Program (DOE/WAP). NASCSP provides research, analysis, training and technical assistance to State CSBG and WAP offices, Community Action Agencies, and State Associations in order to increase their capacity to prevent and reduce poverty and build economic and energy security. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation ABOUT NASCSP | CORE STRENGTHS Building Collaborative Relationships Training & Technical Assistance Advocacy & Leadership Communications & Resources NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION ABOUT NASCSP | OUR PARTNERS NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation ABOUT NASCSP | RESEARCH We provide information. Each year NASCSP documents the sources of funding, services provided, clients and communities assisted, and the results achieved. This information is shared with the network and other research organizations in the form of national reports, articles, and a promising practices database. NASCSP | ROMA Next Generation

Performance Management Framework Now, more than ever data and performance management are critical to the longevity in the short and long term of CSBG. Before we discuss ROMA NG, we need to lay the foundation and discuss how we got here. The Performance Management Framework outlines: How well did the Network operate? and What difference did the Network make? The ROMA System and the NPIs are integral to the PM Framework NASCSP | ROMA Next Generation

Performance Management Framework A “complete package” of Local, State, and Federal Accountability. Includes an a greater emphasis on the ROMA cycle. Allows for progress on data modernization, infrastructure, and capacity for data analysis to improve outcomes. Positions CSBG and Community Action as a model for credible national performance management. the CSBG Network will make better program decisions and generate stronger results for low-income families and communities. The Performance Management Framework is actualized via the CSBG State Plan and the CSBG State Annual Report. Among other things the CSBG State Plan requires the State to explain their plans for Organizational Standards, the State Accountability Measures and ROMA. The CSBG Annual Report requires the State to report on achievements around Organizational Standards, State Accountability Measures and ROMA. NASCSP | ROMA Next Generation

STATE ACCOUNTABILITY MEASURES OUR MISSION STATE ACCOUNTABILITY MEASURES The performance framework includes accountability at all levels. States are now being held to certain standards to ensure they are accountable both to agencies and to OCS. NASCSP | ROMA Next Generation

State Accountability Measures OUR MISSION State Accountability Measures State accountability measures are actualized using three mechanisms: CSBG State Plan CSBG Annual Report, Module 1 The American Consumer Satisfaction Index (ACSI)    Developed to: Track, measure, and improve performance by State CSBG Lead Agencies; Capture performance data about the critical activities and functions performed at the State level; Demonstrate how efficiently and effectively a State implements the activities described in their State plan; and Connect the State’s efforts with the performance of local eligible entities. NASCSP | ROMA Next Generation

CSBG State Plan and Annual Report OUR MISSION CSBG State Plan and Annual Report Development of the State Plan Distribution of Funds Use of Funds Grantee Monitoring and Corrective Action Training and Technical Assistance Data Collection Organizational Standards State Linkages and Communication States are required to work with agencies on planning, provide feedback to agencies across multiple areas and report to OCS. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION ACSI Use feedback to improve communication: Monitoring and Corrective Action Development of the State Plan Linkages Distribution of Funds Use of Discretionary Funds Training and Technical Assistance This is agencies opportunity to provide feedback on the performance of the state. NASCSP | ROMA Next Generation

ORGANIZATIONAL standards NASCSP | ROMA Next Generation

Organizational Standards Developed for local CEEs. Designed to reflect many of the requirements of the CSBG Act, applicable Federal laws and regulations, good management practices, and the values of Community Action. Ensures that all eligible entities have appropriate organizational capacity, not only in the critical financial and administrative, but also in areas of unique importance for CSBG funded eligible entities. State Offices provide consistent and high-quality oversight and technical assistance. NASCSP | ROMA Next Generation

Organizational Standards OUR MISSION Organizational Standards Agencies will be assessed annually. Agency progress will be reported to OCS by the state in the CSBG Annual Report. While agencies are expected to meet 100% of the standards, it is understood that implementation is a progression. Many standards are related to ROMA and standards 4.3 and 7.9 specifically require the services of a ROMA trainer (or equivalent) or the availability of a ROMA training. Standard 4.3 |Private The organization’s Community Action plan and strategic plan document the continuous use of the full Results Oriented Management and Accountability (ROMA) cycle or comparable system (assessment, planning, implementation, achievement of results, and evaluation). In addition, the organization documents having used the services of a ROMA-certified trainer (or equivalent) to assist in implementation. Standard 7.9 |Private The organization conducts or makes available staff development/training (including ROMA) on an ongoing basis. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation Building on the ROMA Foundation ROMA NEXT GENERATION What does Community Action do? Are Community Action Agencies managed efficiently? What services and/or strategies does Community Action provide? Does Community Action use CSBG to provide services or are funds used mostly for administrative support? Does/can Community Action tackle systemic, complex community issues? Who is Community Action serving? What does Community Action achieve? What is Community Action’s impact on the lives of people with low income and communities? Is Community Action doing what works? Is Community Action responding to current community needs? Is CSBG spread around communities so thin it has little impact? NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation ROMA is a system for continuous quality improvement to enable the network to measure, analyze, and communicate performance. ROMA NG moves Community Action at all three levels to a culture of continuous learning rather than a compliance and reporting culture. Ultimately, ROMA NG will help the CSBG Network generate more robust results for individuals with low-incomes and the communities served. The new Annual Report is the vehicle for making ROMA NG real. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OCS, States and CAAs use performance data to achieve greater stability and economic security for families and communities. Each level of the CSBG Network executes the full ROMA cycle. uses data on people, services, and outcomes for decision making. The foundation of ROMA NG is the National Community Action Theory of Change. ROMA NG provides agencies and states with the groundwork for better understanding what services and strategies produce the best outcomes for specific family groups and communities. ROMA NG will help ensure that agencies and states are using their limited resources to do what works best to achieve Community Action’s long-term goals. NASCSP | ROMA Next Generation 14

What’s new in ROMA Next Generation? OUR MISSION What’s new in ROMA Next Generation? The adoption of a National Community Action Theory of Change. Renewal of the CSBG network understanding of the necessity of working toward community change as well as individual and family outcomes. Focus on improved collection and analysis of data. Increased use of data at every point of the full ROMA Cycle. Integration of the phases of the ROMA Cycle. *Improved quality, accuracy, flexibility ROMA NG ensures that the principles and practices related to the full ROMA cycle are operational in all levels of the CSBG network (not just local agencies, as ROMA has been understood in the past) NASCSP | ROMA Next Generation

Community Action Theory of Change The foundation of ROMA NG is the National Community Action Theory of Change. NASCSP | ROMA Next Generation

What is the Community Action Theory of Change? 5/22/2018 What is the Community Action Theory of Change? The National Community Action TOC is the articulation of the underlying beliefs and assumptions that guide a service delivery strategy. It identifies the “why” and the “how” of the activities undertaken and it identifies the “what” of change and improvement produced. This national statement can be adapted for specific local needs and approaches by local CAAs. Describe the elements of a common FRAMEWORK CAAs have to produce change Portray the broad picture of Community Action: long term goals, core principles, system capacity Identify the assumptions that guide the design of services and strategies to reach the long-term goals it hopes to achieve. identify the links that connect the purpose of the network, the understanding of the network’s national system, and the actual strategies implemented by local Community Action Agencies (CAA). NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation Why We Need It Make a complicated system easier to understand. Unify the network. Articulate core principles. Identify national goals. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION This draft went out for comment and we heard from you. Agencies did not necessarily identify with Housing Placement or Employment Counseling. We heard that agencies found this hard to read and didn’t necessarily identify with the nuclear family…. NASCSP | ROMA Next Generation

OUR MISSION Current draft. West to East. Cities. Rural. Services and Strategies.

Questions? What do you think about the new draft TOC? NASCSP | ROMA Next Generation

ROMA NG: The CSBG Annual Report and the CSBG IS Report NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation Where are we now? CSBG Annual report received a 3-year clearance from the Office of Management and Budget (OMB) on January 12, 2017. This starts the phase-in of the CSBG Annual Report and the phase-out of the CSBG IS Survey. NASCSP | ROMA Next Generation

Where are we going? CSBG Annual Report Implementation: Two Phases FY16 & FY17 – Module 1 only. Only affects state offices and not agencies. Local Agency data is completed in the CSBG IS Survey Phase 2 FY 18 Modules 2-4 are implemented. No data is reported in the CSBG IS Survey NASCSP | ROMA Next Generation

Reporting Timelines: October 1 – September 30 Federal Fiscal Year (FFY) State Reporting Period October 1 – September 30 Data Submission for States: Data Submission for local CSBG Eligible Entities:   FFY 2016: October 1, 2015-September 30, 2016 State Reporting Period: October 1, 2015- September 30, 2016 Due March 31, 2017: CSBG IS Survey is submitted to NASCSP. Module 1 is completed based on the 2016 Federal Fiscal Year and submitted through the On-Line Data Collection system (OLDC).  CSBG IS Survey FFY 2017: October 1, 2016-September 30, 2017 October 1, 2016- September 30, 2017 Due March 31, 2018: CSBG IS Survey is submitted to NASCSP. Module 1 is completed based on the 2017 Federal Fiscal Year and submitted through OLDC. FFY 2018: October 1, 2017-September 30, 2018 October 1, 2017- September 30, 2018 Due March 31, 2019: CSBG Annual Report (Modules 1-4) is submitted through OLDC. CSBG Annual Report (replaces CSBG IS Survey) MO has an Oct – Sept reporting period. Agencies will begin collecting data for the new report in October of this year, 2017. (In Progress) NASCSP | The State of CSBG Data Collection and Submission

Assessment and Planning How do we get there? Assessment and Planning Communication & Coordination Training and Technical Assistance NASCSP | ROMA Next Generation

Assessment and Planning NASCSP: Data procurement templates, survey states for technical assessment and implementation timeline, automation of Modules 2-4 in OLDC. Local Agencies: Review of current data systems, timelines, T/TA needs, crosswalk what was reported in the CSBG IS that can be reported in the CSBG Annual Report, identify staff or committee to lead implementation. States: Review of current data systems, timelines, and T/TA. NASCSP | ROMA Next Generation

Preparing to Report on Modules 2-4 Research Modules 2-4. Talk with your state, talk with your association, identify new indicators that you plan to utilize Do you need new intake forms, data collection processes or training? Begin Data Collection. How can your state or NASCSP assist? If you are going to start collecting data these are the things you’ll need to consider…. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation NATIONAL: Federal Partners NASCSP CAP ANCRT CAPLAW OCS Specialists DATA Task Force States: State CSBG Offices State Associations RPICs Local Agencies: Local Experts Neighbor CAAs COMMUNICATION – WHO, What, How often, work together. Utilize and leverage each other. NASCSP | ROMA Next Generation

CSBG DATA (Data, Analysis, and Technical Assistance) Task Force 5/22/2018 CSBG DATA (Data, Analysis, and Technical Assistance) Task Force Convened by NASCSP to assist OCS and NASCSP in understanding and addressing the CSBG Network’s Data needs, including collection and storage; Use of data for analysis; and Continual improvement of results. DATA Task Force is a consultative body focused on: The transition from the CSBG IS Survey to the CSBG Annual Report; and Ongoing assistance in the implementation of the CSBG Annual Report. Create a data oriented culture. Local, state, associations and national – data needs – yes but also data collection and storage needs.  the systems for collection and storage (in silos in most cases) is part of what needs to be considered by this group. “data friendly” org culture.  We understand the value of data and want to be sure our data is accurate and complete.  The culture of the network has to be changed to embrace these ideas and values.  NASCSP | ROMA Next Generation

ROMA NG: The CSBG Annual Report and the CSBG IS Report NASCSP | ROMA Next Generation

New Annual Report: Modules 2 - 4 Question: How do I know what data I need to collect for Modules 2 – 4? NASCSP | ROMA Next Generation

Annotated CSBG Annual Report This is the and annotated version of Module 1. This is a crosswalk of the CSBG IS and the new Annual Report. NASCSP | The State of CSBG Data Collection and Submission

What is in the new Annual Report? Module 2 - CSBG Eligible Entity Expenditures, Capacity, and Resources – Many data points will be familiar. NASCSP | The State of CSBG Data Collection and Submission

What is in the new Annual Report? Module 3 Community Level – This may look different NASCSP | ROMA Next Generation

What is in the new Annual Report? Module 4 Individual and Family Leve Many data points will be familiar. NASCSP | The State of CSBG Data Collection and Submission

New Annual Report: Modules 2 - 4 Question: Some of this looks very similar to what I currently report, what is different in Modules 2 – 4 than the IS Survey? NASCSP | ROMA Next Generation

New Annual Report: Modules 2 - 4 Answer: Module 3 has some new elements. We understand it may take time to phase in. Let’s talk about Module 3. Status page and rates of change. Look in the CAP Plan NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION Module 3 Community Level NPIs Community Strategies NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION Community Needs When we hear agencies talk about needs that were assessed, they are often community level needs: Our community lacks living wage jobs There is a need for more affordable housing We have a high unemployment rate We have a low high school graduation rate in the school district that serves our customers with low income Community Level Work: Renewal of the CSBG network’s understanding of the necessity of working toward community change as well as individual and family outcomes. NASCSP | ROMA Next Generation

Community Responses – Identifying Outcomes OUR MISSION Community Responses – Identifying Outcomes ROMA Next Generation asks the entire CSBG network to consider how we will respond to these needs. What will change in communities? Increase in living wage jobs, affordable housing units? Reduced unemployment rate? Increased high school graduation rate? Discussion – why should agencies get involved in this kind of response? Because it is the appropriate way to address the identified needs. Individual responses will help in some ways (people get jobs, families find housing, teens graduate), but will not turn the curve on the situations facing the entire community. NASCSP | ROMA Next Generation

Community Responses – Identifying Strategies OUR MISSION Community Responses – Identifying Strategies ROMA Next Generation asks the entire CSBG network to consider how we will respond to these needs. What will agencies do to help make these kinds of changes in their communities? Who will we partner with (as these efforts usually involve multiple organizations.) Discussion – why should agencies get involved in this kind of response? Because it is the appropriate way to address the identified needs. Individual responses will help in some ways (people get jobs, families find housing, teens graduate), but will not turn the curve on the situations facing the entire community. NASCSP | ROMA Next Generation

Reporting on Community Work OUR MISSION Reporting on Community Work Counts of Change  These are basic measures that provide the number of units being measured. e.g. jobs, houses, resources, etc. that have been added (created) and in some cases maintained, or undesirable conditions subtracted (eliminated), in the community the CSBG Eligible Entity has targeted.   The number of job opportunities or housing units created are examples of “counts” The percent of unemployment and of graduation are “rates” NASCSP | ROMA Next Generation

Reporting on Community Work OUR MISSION Reporting on Community Work Rates of Change These measures call for reporting the “percent change”—the increase or decrease of some rate from one year to the next.   Require more data and information than counts These indicate the magnitude of the impact a community initiative has had in a community.   The number of job opportunities or housing units created are examples of “counts” The percent of unemployment and of graduation are “rates” NASCSP | ROMA Next Generation

Rates of Change Examples NASCSP | ROMA Next Generation

Rates of Change Examples NASCSP | ROMA Next Generation

New Annual Report: Modules 2 - 4 Question: Who is entering data into OLDC for Modules 2 – 4? States or CAAs? NASCSP | ROMA Next Generation

New Annual Report: Modules 2 - 4 Question: At this point in the planning phase, States will be entering data (collected from local agencies) for Modules 2 – 4 in OLDC in March 2019. NASCSP | ROMA Next Generation

New Annual Report: Modules 2 - 4 Question: Will there be a new numbering system for Modules 2 - 4? NASCSP | ROMA Next Generation

New Annual Report: Modules 2 - 4 NASCSP | ROMA Next Generation

Improved Data Analysis and Use Integration of the ROMA Cycle OUR MISSION Improved Data Analysis and Use Integration of the ROMA Cycle Focus on improved collection and analysis of data – and use of data at every point of the full ROMA Cycle What else is ROMA NG? It sounds like a lot of reporting… NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation 5/22/2018 DATA ANALYSIS – DIGGING DEEPER WITH ROMA NG Implementation of the full ROMA cycle includes identifying what will change, based on the locally assessed needs, and how the agency will know if the change has happened. Each part of the ROMA cycle involves data and the opportunity for analysis of that data. Preparation for data collection and analysis has to be included in the agency’s activity model. ROMA Next Generation requires greater use of data and data analysis. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION Questions to Consider ROMA Next Generation recognizes the importance of analysis to identify: Trends that are happening over time across an agency which may impact services or strategies. Efforts that are making a positive change in the lives of individuals and families with low income; and the conditions in communities where they live. Resources that can be combined to produce greater and more comprehensive change. Common barriers that go beyond the influence of a single local agency, which should be addressed at state, regional or federal level. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation DATA Analysis What does Community Action do? Are Community Action Agencies managed efficiently? What services and/or strategies does Community Action provide? Does Community Action use CSBG to provide services or are funds used mostly for administrative support? Does/can Community Action tackle systemic, complex community issues? Who is Community Action serving? What does Community Action achieve? What is Community Action’s impact on the lives of people with low income and communities? Is Community Action doing what works? Is Community Action responding to current community needs? Is CSBG spread around communities so thin it has little impact? NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION DATA ANALYSIS/ EVALUATION Does the data in the community needs assessment contain qualitative and quantitative data? What conclusions are made? Does the CNA provide data on who is being served? Do the services match the needs of the people served? Does the strategic plan address the needs identified in the needs assessment? What about the CAP Plan? ROMA NG = greater emphasis on data analysis and evaluation. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION DATA ANALYSIS/ EVALUATION Is the data reported accurately? Is it reliable? Can you report on a an unduplicated count served? Can you connect the people served to outcomes obtained? What types of services were provided? What outcomes did the services produce? What community strategies were reported? What outcomes (NPIs) were achieved? NASCSP | ROMA Next Generation

putting it all together OUR MISSION putting it all together NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION ROMA NG Community Needs Assessment Standard 3.1 – completed at least once every three years. Strategic Plan Standard 6.1 - approved within the past 5 years. Community Action Plan Standards 4.2 – 4.4 – outcome based, documents ROMA cycle, annual update to board. CSBG Annual Report Standard 9.4 – completes the annual report that contains outcomes and demographics. NASCSP | ROMA Next Generation

1.2 State ROMA Support Activities OUR MISSION 1.2 State ROMA Support Activities States will reporting to OCS on how they supported you in the annual report. Some things they may be assisting with… The state provided training or technical assistance to Eligible Entities on community assessments or strategic planning. The state developed a CAA state-wide tool/template for use in community assessments or strategic planning. The state increased the number of Certified ROMA Trainers and/or Implementers to increase the capacity of local agencies to implement the full ROMA Cycle. NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation OUR MISSION 1.3 State Review of Data State will be reviewing your data more in depth. Some way in which you may see greater state involvement: The state utilized the diagnostic tools from the IS Survey and made comparisons. The state reviewed the agency CNA and strategic plan to find the connections between needs and planned actions. State office staff conducted a thorough review and evaluation of all monthly (or quarterly) performance reports prior to approving them.  The state reviewed agency reports to see if outcomes matched the needs of both planned and actual actions. Describe any systems changes such as new data bases, new procedures for data collection and analysis. NASCSP | ROMA Next Generation

I.4 State Feedback to CEEs on Meeting ROMA Goals OUR MISSION I.4 State Feedback to CEEs on Meeting ROMA Goals Are these the same as last year? Any new outcomes? Did outcomes match the needs in the agency’s CNA? Did outcomes match planned targets? How does this year’s performance compare with previous achievements? Do the NPI and client demographic reports match the budgets? Do budget and performance categories connect? How do the current year expenditures compare with prior year expenditures? Are they reporting expenditures in the same categories? Are they different? NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation Questions? What have you already done to get ready for implementation of modules 2-4? How have you considered changing current planning or reporting processes? What do you need from NASCSP and other partners? NASCSP | ROMA Next Generation

NASCSP | ROMA Next Generation RESOURCES National Theory of Change How does ROMA NG fit into ROMA CSBG – ROMA – ROMA Next Generation NASCSP | ROMA Next Generation