1. DEI FPO Orientation August 18, 2016

2. Agenda ETA Grants Office: Brief Introduction Overview of DEI
DEI Round VII Funding Opportunity Announcement
DEI Technical Assistance and Training Support
DEI Independent Evaluation Component 2

3. Brief Introduction OGM Grants Officer – Jimmie Curtis
Grant Officer/Supervisory Grants Management Specialist
Employment & Training Administration
(202)
No Cost Extension
Contract Modification
Brief Q&A

4. Overview of DEI
DEI funds allow enhancements and build capacities in the workforce system to support employment efforts of people with disabilities through the American Job Centers (AJC)
DEI funds are designed to address the following needs:
Improve the training and employment of individual with disabilities focusing on career pathways which lead to economic self-sufficiency;
Build effective partnerships that leverage public and private resources to better serve individuals with disabilities; and
Expand the public workforce developments system’s capacity to become Employment Networks for beneficiaries under the Social Security Ticket to Work (TTW) Program.
DEI funds are awarded to state workforce agencies with resources going to Local Workforce Development Areas
DEI funds help fund Disability Resource Coordinators (DRCs) at AJC’s

5. Grant Implementation Timeline

6. Who Received DEI Grants?6

7. DEI Project Focus All DEI projects currently focus on the following:
Increasing participation in Career Pathway Systems;
Developing collaborative partnerships across multiple service delivery systems;
Blending and braiding of funds to leverage resources;
Providing flexible opportunities to persons with disabilities for training and employment; and
Creating systemic change.

8. Key Players in DEI Administration
National DEI Program Office
ETA
ODEP
ETA Federal Project Officers
Regions 1 – 6
Contractors
NDI Technical Assistance Team
DEI Evaluation Team 8

9. DEI Round VII Funding Opportunity Announcement (FOA)

10. DEI Round VII FOA
Funding Opportunity: On June 27th, DOL released a FOA for the seventh round of DEI projects with availability of $15.6 million to fund approximately eight cooperative agreements to state workforce agencies ranging from $1.5 million to $2.5 million each.
Performance Period: The period of performance will be 42 months starting October 1, 2016.
Population Focus: DOL intends to award at least one agreement in each of 3 focus areas – Adults, Youth, and Significant Disabilities.
Deadline: The application deadline was August 1st.

11. Leading DEI Round VII Objectives
Employment Outcomes – Improve employment outcomes of individuals with disabilities.
Career Pathways – Adapt and expand capacity of Career Pathways.
Strengthen Partnerships – Build effective partnerships across multiple service delivery systems that leverage resources.
Employment Network (EN) - Expand public workforce development system’s capacity to become an EN for beneficiaries under Social Security Ticket to Work program.
Systemic Change – Create systemic change by being innovative.

13. Technical Assistance- NDI Consulting, Inc.

14. NDI-DEI TA Team
Contracted to provide information, training, and technical assistance to the national DEI program office, regional FPOs, DEI grantees, and DEI project staff and the workforce development system.
Team members include:
Michael Morris, Project Director and Laura Gleneck, Project Manager
Training and Technical Assistance Liaisons and key areas of subject matter expertise: Miranda Kennedy (DEI Director of Training and Technical Assistance), Brian Ingram (DEI Workforce Access and IRT approach), Dave Mayer (DEI Career Pathways); and Nikki Powis (DEI Youth and Business Engagement)
Kevin Nickerson and Nanette Goodman (DEI Ticket to Work Team)
DOL Onsite Disability Workforce Specialist: Alexandra Kielty and IT Specialist: Jayson Gleneck

15. Menu of Options NDI Team Learning Medium Description Project Listserv
Designed to reach every level of federal, state and local stakeholder audience to disseminate and gather information.
TA Project Website
Resources and materials related to, or developed specifically for, Grantees and Project Staff.
Suite of Technical Assistance Assessment and Needs tools
Use of systematic and comprehensive self-assessment and needs tools to objectively measure understanding and approach to advance knowledge translation activities across key indicators of performance.
Self-Paced Online
Distance Learning
Allows grantees their choice of time and pace to improve critical knowledge and skills.
Expert Facilitated Webinars
Provide opportunities for real-time interactive discussions; remote in-depth technical assistance and formalized training.
Thirty Second Trainings and Case Simulations
Provide innovative approaches to skill building; stimulate learner interest and introduces users to other training opportunities.
One-on-One Telephone and Electronic Communication
Provide the opportunity to engage in brain-storming and problem-solving to promote greater knowledge, understanding and action.
Peer Support Networks
Effective use of a peer support system for skill and capacity building and innovative approaches to translate knowledge to action.
Peer Communities
of Learning
Provide an opportunity for peer learning to improve problem-solving skills and opportunity to network and strengthen relationships leading to an enhanced peer to peer support model.
Virtual and In-Person
Site Visits
Provide an opportunity for tangible hands-on training and targeted technical assistance.
Grantee Meetings
Provide the opportunity for formal and informal learning and training in relation to the implementation and growth of the grant.

16. Contracted Tasks
Provide multi-modal, targeted technical assistance designed to identify the status of DEI grantee implementation and to target and address the specific needs of individual DEI grantees.
Provide comprehensive training to include both virtual and informal methods. A key component centers on facilitating peer communities of learning among DEI grantees.
Provide analysis of DEI grantee status reports to identify key elements of program implementation and impact of interventions, and respond to other requests for information.
Provide collateral materials development, resources, information, and technical support to the national DEI Program Office and the public workforce system.

17. Overview of Services
Information -- Host a listserv to compile and disseminate resources and training information and collect project-related data (e.g., Resources of the Week).
Maintain a DEI TA Collection site and Disability and Employment CoP
Technical Assistance -- designated TA Liaison is assigned to serve as the direct contact and support for individual DEI grantees and their project staff.
Available for one-on-one discussion, problem solving, and strategic planning by request to support the implementation of the DEI Initiative within the workforce development system.

18. Start-up and Implementation / Targeted TA & Training
Start-up and Implementation -- Suite of TA and training tools and resources to provide start-up support to the DEI grantees.
Provide national and TA team with information to develop and implement a strategy to help projects move the DEI forward.
Updated and/or expanded for each round based on the statement of work and lessons learned from previous grantees.
Ongoing Targeted TA and Training activities – includes hosting and facilitating ongoing administrative meetings
Conducting DEI technical assistance onsite visits.
Enhancing and developing new training and technical assistance materials.
Ticket to Work and Employment Network Support.
Training and learning opportunities.

19. Grantee TA Onsite Visits
Provide tangible hands-on training and targeted TA and allow TA Liaisons to provide support, as well as observe, analyze, and identify strengths and challenges within a DEI project.
Pre-site visit planning activities – TA Liaison and/or DEI project leadership inform designated FPO and national DEI Program Office.
FPOs are extended an opportunity to be involved in agenda development including reviewing and approval of draft agenda.
In developing agenda, national / regional representatives are extended an opportunity to be part of visit in person or via phone.
All national/regional partners are made aware of date / time /location of visit.
Post site visit activity -- TA Liaisons develop post-site visit reports that highlight key activities that took place and also identify key strengths and challenges along with lessons learned.

20. To Contact the NDI-DEI team
Laura Gleneck, Project Manager /
Technical Assistance Liaisons:
Miranda Kennedy (CT, ME, MA, MN, NY 4-6, /
Brian Ingram (AL, CA, GA, HI, IL 5, KS, VA, /
Dave Mayer (ID, IL 4, SD, WA) /
Nikki Powis (AK, IA) /

21. Evaluation- ODEP and Social Dynamics

22. Presentation Outline Six Key Research Questions for the DEI
Overview of the DEI Rounds 5-6 Design
Outcome and Impact Evaluation
Comparison Site Survey
Improving Data Quality
Social Dynamics Evaluation Liaisons

23. Key DEI Evaluation Research Questions
What start-up issues were identified? How were these issues resolved? Sources: Site visits and telephone interviews (Component: Implementation)
What are the impacts of the DEI grants on customer utilization and disability employment outcomes such as employment, earnings, and employment retention?
Source: Administrative data—WIASRD/W-P/DEI Data System (Components:
Customer Outcomes & Impact)
3. What services and strategies appear to be the most successful in helping customers achieve employment outcomes? Sources: Site visits, telephone interviews, and administrative data
(Component: Implementation) DK

24. Key Research Questions (continued)
What are the challenges that need to be addressed to improve the employment outcomes of PWDs? Sources: Site visits and telephone interviews (Component: Implementation)
What systems changes need to be made to overcome these challenges? Sources: Site visits and telephone interviews
(Components: Implementation & System Change Outcomes)
What best practices have the treatment sites identified that address these systems changes? Sources: Site visits and telephone interviews (Component: Implementation) DK

25. Overview of the DEI Evaluation Design
The DEI Evaluation is a comprehensive implementation/process, outcome, and impact study.
Implementation: Collection of information on systems change and the strategies implemented by each of the DEI grantees (site visits/telephone interviews).
Outcomes: Focus on customer-level outcomes based on administrative data (WIASRD, W-P, DEI Data System).
Impact: Focus on comparing DEI treatment sites with comparison sites (WIASRD, W-P, DEI Data System).
The DEI Evaluation has three main components. Implementation, Outcomes and Impact. The Implementation component collects and analyzes information on systems changes, the implementation of the DEI strategies and the utilization of Job Center services by customers with disabilities. The Outcomes components examines customer-level outcomes such as job placement and wages and the Impact Evaluation compares the DEI treatment sites with the comparison sites. Most of the data we use comes from the information that is already collected by Job Center staff. But we added in 2010 a set of DEI Data Elements that are not included in most state systems.

26. DEI Evaluation Design (continued)
Data Collection Methods
Site Visit Interviews/Focus Groups, One site visit per WDA to collect start-up and implementation status. (Component: Implementation)
WIASRD/WIOA, Wagner-Peyser and the Participant Tracking System: Data is sent quarterly to the Evaluation Team. (Component: Outcome & Impact)
Early on in Rounds 1-4 we randomly selected WDAs for site visits or telephone interviews due to the shear number of them. We did the same thing for control sites. For Rounds 5-6, there are 29 WDAs total. By comparison, there are 166 Rounds 1-4 total.

27. DEI Evaluation Design Outcome Analysis
The DEI evaluation will measure the following outcomes for Rounds 5 and 6:
Number and percentage of DEI customers, broken down by number served in core, staff-assisted core, WIA-WIOA intensive and WIA-WIOA training
Number and percentage in Career Pathways programs
Characteristics of customers with disabilities served at each WDA (WDA), including type of disability and activities of daily living
Academic outcomes (credentials, certificates)
Employment and wage outcomes

28. Comparison Site Survey
DEI Evaluation Design
Comparison Site Survey
A very important component of the evaluation
Its purpose is to ensure a statistical match between the treatment and comparison group for the future impact analysis
Will be implemented in fall 2016 and continue into later years depending on response rate
Survey includes the same set of questions on disability type and activities of daily living for the treatment and comparison groups
Treatment group: through existing Participant Tracking System
Comparison group: through comparison site survey

29. Comparison Site Survey Cont’d
Provides a more detailed look at the types of disabilities participants have
Currently, WIASRD/WIOA has very basic information on disability type (e.g., physical, mental or both)
What are comparison WDA staff being asked to do?
1. Offer survey consent form to individuals that enroll in a Job Center and self-disclose a disability
2. Put each signed consent form in an envelope and keep in file
3. Once per month, FedEx pick-up
$15.00 stipend for complete survey response

30. What Improves Data Quality?
Completion of all required WIASRD/WIOA Data Elements.
Are all Data Elements being completed by staff at each American Job Center?
Consistency in American Job Center operations.
Do all WDA American Job Centers ask the “disability”, “career pathways/training” and “certification/diploma completion” questions?
Is the eligibility criteria for certain services approximately the same across WDAs.
Is all Participant Tracking System Data Being Entered by Treatment Sites DK

31. Data Quality (continued)
How can the DEI Evaluation Team help with Data Quality?
Careful review of all quarterly WIASRD/WIOA data to ensure that they collect the required DEI Data Elements
Provide Data Quality Reports that pinpoint missing responses or errors
Quarterly Data Quality Reports sent to DEI State Leads
Webinars for State, LWIB, and American Job Center staff on DEI data collection requirements and use of the PTS
Toll-Free Telephone Technical Assistance available to DEI State Leads, State IT staff, and American Job Center staff DK

32. Grantees’ Direct Link to the DEI Evaluation: State Evaluation Liaisons
Each grantee has a State Evaluation Liaison who will:
Respond promptly to DEI Evaluation-related questions from State, LWIB, and American Job Center personnel;
Ensure smooth implementation of the DEI Data System at the American Job Center level; and
Coordinate, design, and schedule technical assistance related to the DEI Evaluation (e.g., Data System site visits, webinars, conference calls, etc.).;
Provide site visit schedule to DEI stakeholders: ODEP, ETA, FPOs and NDI.
Susan Labin, Ph.D.
Gary Shaheen, MPA
Cori DiBiase
Doug Klayman, Ph.D. DK CA IA AK GA HI NY IL KS SD WA MA MN

33. DEI Evaluation Resources
DOL
DEI Leads
DRCs
ECs
FPOs
Rounds 5 and 6 DEI Evaluation Bulletins Disability Description Guide Frequently Asked Questions Disability Categories & ADLs DEI Rounds 5-6 Data Collection Requirements Webinars DEI Participant Tracking System √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √
State Presentations
Data Quality
Updates on State Data
Updates on Data Collection Requirements
Why do we do all of this? Because there’s human element to large-scale evaluations that is extremely influential in terms of data quality and integrity. We respond to many requests for help on various aspects of the DEI evaluation. At the same time, we are often invited to speak at regional meetings. CA, IA, MA, VA. √ √ √ √ √ √ √ √ √ √
**Please see the online DEI Evaluation Document Library for more resources.

34. To Contact the DEI Evaluation Team:
Douglas Klayman, Ph.D.
Project Director
x 105
Toll free:
Visit us online at:
Have people available all day to answer questions about data.

35. Q & A

36. Additional Resources Innovation and Opportunity Network (ION)
Workforce GPS
For DEI grant implementation tools and ongoing support for state-level project staff and other key stakeholders, go to
For Disability and Employment Community of Practice, go to
For Career Pathways resources, tools and information on Career Pathways, go to
Innovation and Opportunity Network (ION)
National, regional, state and local alliance that facilitates
information sharing to implement the vision of WIOA,
go to

37. Contact Information
David Jones, National DEI Program Office
Title: Workforce Analyst
Phone: