1. Rate your experience level producing the Annual Statewide Performance Report Narrative
Master: I am well versed in all aspects of the WIOA Statewide Performance Report Narrative requirements. I am a key author or contributor.
Pro: I am familiar with the requirements of the WIOA Statewide Performance Report Narrative. I am engaged in most facets of developing the narrative.
Entry level: I have limited experience with the WIOA Statewide Performance Report Narrative. I am new to the process.

2. WIOA Annual Statewide Performance Report Narrative
September 12, 2019
WIOA Annual Statewide Performance Report Narrative
An Overview featuring common questions about ETA’s WIOA Annual Statewide Performance Report Narrative

3. Employment and Training Administration
Shelia F. Lewis
Workforce Analyst
Employment and Training Administration
U.S. Department of Labor

4. Senior Program Analyst
Kellen Grode
Workforce Analyst
Employment and Training Administration
U.S. Department of Labor
Gloria Salas-Kos
Senior Program Analyst
Employment and Training Administration
U.S. Department of Labor
Sean Fox
Office of Federal Contract Compliance Programs
U.S. Department of Labor

5. Christina Eckenroth Toquir Ahmed
Workforce Analyst
Employment and Training Administration
U.S. Department of Labor
Toquir Ahmed
Workforce Analyst
Employment and Training Administration
U.S. Department of Labor

6. In today’s session we will prepare for the Annual Report Narrative:
Give feedback on the FY 2017 Annual Report Narratives,
Provide resources for FY 2018,
Address questions.

7. PART I: BACKGROUND

8. Background
WIOA Common Performance Reporting - OMB Control No requires an annual report narrative to complement the WIOA Annual Statewide Performance Report (ETA 9169) requirements defined in OMB Control No
Refer to Training and Employment Guidance Letter 5- 18, Workforce Innovation and Opportunity Act (WIOA) Annual Statewide Performance Report Narrative (published November 7, 2018)

9. What programs are affected?
Background
What programs are affected?
Title I grantees: Adult, Dislocated Worker, and Youth
Title III grantees: Wagner-Peyser Employment Service
When is it due?
December 1st. The report is due annually on December 1st, or the first business day thereafter should that date fall on a weekend.

10. Background What Format? 25 pages or less 508 Compliant
Submitted electronically in machine readable format
No limit to appendices (not required)

11. Background What Content? Components to address:
Waiver that the state has had in place for at least one program year,
Effectiveness in Serving Employers performance indicator pilot approaches, and any state established measures,
Evaluations,
Customer Satisfaction approach,
Progress made in achieving the state's strategic vision and goals,
State's performance accountability system, and
Activities provided by state funds.

12. Benefits of the narrative?
Background
Benefits of the narrative?
Share your story
Opportunity to describe State’s progress towards meeting the strategic vision and goals for the workforce system.
Include additional information about your programs that may not be reflected in other performance reports.

13. No separate partner narrative required
Background
No separate partner narrative required

14. Reminders
Submit an electronic copy of the PY WIOA Annual Statewide Performance Report Narrative to by the close of business on December 2, 2019
Copy to the state's respective ETA Regional Administrator and Federal Project Officer
Submit in a 508-compliant PDF format.
IMPORTANT

16. Part II: Waivers

17. Today’s Objectives: Waivers
Review components of the waivers portion,
Discuss some questions to consider in drafting the waivers portion of the annual narrative reports.

18. Waivers: Components to address
Identify each waiver that the state has had in place for at least one program year,
Provide information regarding the state’s progress toward achieving the goals and performance outcomes in ETA’s letter of approval for the waiver and outlined in the state’s waiver request (when applicable),
Discuss how activities carried out under each approved waiver have directly or indirectly affected state and local performance outcomes,
To the extent possible, provide quantitative information.

19. Example Questions to consider
A State approved for a waiver to increase the OJT employer reimbursement rate would need to demonstrate fiscal adherence to the waiver-approved rate, and assess whether the waiver is allowing businesses to expand the use of OJT as a tool to increase the skills of workers.
Has your State undertaken a monitoring effort to oversee effective waiver implementation?

20. Questions to consider…
Has your State achieved the programmatic and service goals outlined in the approved waiver request?
Has your State achieved the outcomes and other tangible benefits for jobseekers or employers outlined in the approved waiver request?
Has your State identified how each waiver has favorably impacted services for disadvantaged persons, persons with barriers to employment, or business?
If you have the waiver to decrease the 75 percent out- of-school youth expenditure, have you been able to maintain a satisfactory level of support for out-of-school youth while documenting the benefits for in-school youth as a result of the waiver authority?

22. Effectiveness in Serving Employers Measures
PART III:
Effectiveness in Serving Employers Measures

23. Effectiveness in Serving Employers: Components to address
The two approaches selected
Retention (Retention with the same employer)
Repeat Business Customers (Percentage of repeat employers using services within the previous three years)
Employer Penetration Rate (Percentage of employers using services out of all employers in the State)
Any state-established measures, or other employer engagement metric

25. How do you collect your evaluation information to meet the annual report requirements?
I contact my agency’s evaluation and research office.
I receive evaluation announcements and publications from state and local leadership.
I review other local, state, and Federal evaluation announcements and publications.
I gather input from state and local workforce agency program managers and partners.
All of the above.
None of the above.

26. PART IV: EVALUATIONS
“Research and evaluation begins where performance reporting and results end.”
–Wayne Gordon, Director of Research and Evaluation, USDOL-ETA

27. Today’s Objectives: Evaluations
Review the WIOA evaluation requirements within the context of the state annual reports,
Provide examples that may fulfill the WIOA evaluation requirements,
Share resources and contact information for additional technical assistance.

28. Lessons Learned from PY 2017 Annual Reports
Evaluation vs. Monitoring
Tell us more!

29. Evaluations: Components to Address
Current or planned evaluations and related research projects, including methodologies used (Conduct);
Coordinate the development of such projects with WIOA core programs, other state agencies and local boards (Coordinate and consult);
Completed evaluations and related reports, with publically available and accessible links;
Efforts to provide data, survey responses, and timely site visits for Federal evaluations (Cooperate); and
Continuous improvement strategies applied from study results and evidence-based practices.

30. Lessons Learned from Program Year 2017
Confusion with monitoring and performance reporting as a form of evaluation and related research activities,
Missing information about current or planned evaluations and related research activities,
Lack of information about plans to develop evaluation and related research with WIOA core programs, other state agencies and local boards,
Missing data on your coordination with Federal evaluations to provide data, survey responses, and timely site visits.

31. How can states address the WIOA evaluation requirements?
Describe impact or descriptive evaluations completed or that are underway.
Conduct a literature review to find impact or descriptive evaluations using Workforce System Strategies.
Use the Clearinghouse for Labor and Research (CLEAR) to identify any systemic reviews of evidence-based practices.
Work with WDQI partners to leverage administrative data.
Engage your RESEA or other WIOA core partners.
Ask about and coordinate with Federal partner evaluations.

32. What is relevant research and evaluation?
Identify and describe research and evaluation reports for difference state agencies (workforce, higher education, K- 12 education).
Share examples of data dashboards and outcome scorecards (student and workforce success measures) for stakeholders.
Describe efforts to develop evaluation and/or procure evaluators to design and implement descriptive and impact studies.
Share links to your publicly available evaluations and research.
Evaluation Capacity
Workforce Data Quality

33. Evidence- Based Research
How can states build capacity to address the WIOA evaluation requirements?
Catalogue recent research and evaluation publications produced about your workforce programs
Ask stakeholders about the types of research and evaluations that are responsive to their interests
Utilize research arms of your agencies or consult with external partners or universities
Evaluations
Evidence- Based Research

34. Evaluation and Research Resources
Evaluation and Research Hub
A community point of access to support policymakers and program managers with their efforts to use evaluations to improve services and choose evaluations and research to inform program policies and evidence- based practices.
Workforce System Strategies
Contains over 1200 profiles of workforce development research and evaluation studies, toolkits, and other resources to support program management and operations, education and training, and employment, retention or advancement activities.

35. Evaluation and Research Resources
USDOL-ETA Division of Research and Evaluation
Manages ETA’s rigorous research and evaluation approach under Section 169 of the Workforce Innovation and Opportunity Act
CLEAR
CLEAR identifies and summarizes many types of research on labor topics, including descriptive statistical studies and outcome analyses, implementation, and causal impact studies

36. What are other Partner Resources?
Reemployment Eligibility Services and Assessment (RESEA) Evidence-Based Guidance provides evaluation technical assistance resources to help states meet legislative requirements.
Adult Education provides evaluation resources that include impact studies and tools.
HHS’s Self-Sufficiency Research Clearinghouse provides access to high-quality research on employment and family and child well-being.
RSA’s National Technical Assistance Center on Transition is a dedicated technical assistance center defines likely effectiveness and evidence levels

37. Any Questions?
Interested in learning more about the Evaluations, contact:
Gloria Salas-Kos, Evaluation TA Coordinator
Wayne Gordon, Director, Division of Research and Evaluations

38. Part V: Data Validation

39. Data Validation: Components to address
The state's approach to data validation and ensuring data integrity

40. Data Validation: Components to address
TEGL 7-18 Guidance for Validating Jointly Required Performance Data Submitted under the Workforce Innovation and Opportunity Act (WIOA) December 19, 2018
Each State must develop data validation procedures that include:
Written procedures for data validation;
Regular data validation training for appropriate program staff;
Monitoring protocols;
A regular review of program data;
Documentation that missing and erroneous data have been corrected;
Regular assessment of the effectiveness of the data validation process.

42. Part VI: Customer Satisfaction

43. Customer Satisfaction
Not a required performance measure
No specific method or measure required

44. Customer Satisfaction: Components to address
the state’s methodologies;
the number of individuals/employers who were provided customer satisfaction outreach, the response rate, and efforts made to improve the response rate;
the results and whether the results are generalizable to the entire population of customers; and
a description of any continuous improvement processes for incorporating the customer satisfaction feedback.
Address the state’s approach to customer satisfaction
Describe: