1. Virginia’s Employer Follow-up: An Examination of Response and Non- Response Patterns Presenters: Eric Lichtenberger Jim Washington

2. Organization of Presentation ● Introduction ● Description of follow-up procedures ● Adaptations over the past few years to improve response rates ● Going beyond reporting requirements (providing useful information to stakeholders)

3. Brief History of Our Research Center ● Center for Assessment, Evaluation, and Educational Programming (CAEEP) ● Virginia Tech’s School of Education’ Office of Education Research and Outreach ● Conducting the annual follow-up for the Virginia Office of Career and Technical Education Services for 30 years ● We also perform similar work for the Virginia Department of Adult Education and Literacy and the Virginia Department of Correctional Education

4. Overview of the Follow-up ● Local school divisions are highly involved in the follow-up process ● Several years ago we moved towards a flexible on-line reporting system that blends paper based, on-line, and telephone data collection. ● Currently we have 5 years of parallel data ● 30,000 CTE completers (concentrators) and 4,500 employers ● Description of survey Description of survey

5. Improvements to the Process ● Developed the Address Checker Tool ● Used Division Letters, Division Envelopes, and Allowed Divisions to Conduct their Own Employer Follow-up ● Obtaining Employer Phone Numbers ● Establishing 75% as the standard response rate

6. Summary Numbers from the 2006 Survey ● State Numbers: Divisions and Regional Centers may obtain specifics for their area. – 41.15% of 4,578 employers responded. – 30.23% of 4,578 employers provided usable responses. – 89.52% of 1,384 employers were either Satisfied or Very Satisfied with the Employee's high school preparation – These numbers have steadily increased since 2002

7. Summary Numbers from the 2006 Survey ● 35.48% of 1,384 employers rated the employee's high school preparation as Equally Prepared ● 44.29% of 1,384 employers rated the employee's high school preparation as Better Prepared ● 79.77% of 1,384 employers were either Satisfied or Very Satisfied with the employee's Technical Skills as they related to the job

8. Employer Comments ● Survey question asks for specific comments in reference to the Workplace Readiness Skills, categorized by “Satisfied” and “Unsatisfied” ● About half of the respondents provide comments ● Average of three comments per survey that has comments ● Comments tend to be 4-to-1 favorable ● We let the computer do the summarizing

9. Comments Summary ● Satisfied – Attitude – Work Ethic – Computer Skills – Attendance – Teamwork – Working with Others – Math – Reading – Quick Learner ● Dissatisfied – Attendance – Writing – Decision-making – Initiative – Math – Spelling – Work Ethic – Listening – Attitude – Working with Others

10. Satisfaction Vs. Applicability ● “Not Applicable” ● Low N/A = High Applicability ● High N/A = Low Applicability ● Satisfaction Where Applicable, e.g., (368+364)/(893-140)

11. Satisfaction Where Applicable ● 97% Reading ● 93% Math ● 93% Computer ● 92% Technical Skills ● 91% Writing ● 90% Teamwork ● 89% Attendance ● 89% Pos. Attitude ● 89% Speaking & Listening ● 86% Work Ethic ● 86% Self- Presentation Skills ● 85% Reasoning and Problem-solving ● 83% Independence and Initiative ● 81% Big Picture

12. Applicability ● 94% Attendance ● 93% Work Ethic ● 92% Pos. Attitude ● 92% Teamwork ● 90% Independence & Initiative ● 90% Speaking & Listening ● 87% The Big Picture ● 87% Reasoning & Problem-Solving ● 85% Technical Skills ● 84% Reading ● 79% Math ● 79% Writing ● 62% Computer

13. Address Checker Tool ● Local school divisions are required to verify employer addresses ● Employer response rate has improved since implementing this tool ● Once an employer address is entered the information is compared against a list of deliverable addresses ● There are numerous outcomes based on that comparison

14. EMPLOYER MAIL ADDRESS STATUS - STATE REPORT Address StatusNumberResponsesResponse Rate Bad state abbreviation11100% Insufficient address6056310% Missing street directional7457% No street address number1193126% Not found8338% Out-of-state371232% Possibly deliverable3294155947% Probably not deliverable311342% Street not found1173328% Unknown Status513671% Verified OK44413029% Statewide4714188540%

15. 2006 Employer Response Rates by Follow-up Procedure ProcedureEmployersResponse rate Traditional314736% Division Envelope46929% Division Letter4262% Division Conducted105657%

16. Employers that Traditionally Do Not Respond ● Created a database of employers who due to company policy, do not respond ● Categorized the information by year and branch ● Some companies are consistent and with others it depends on the branch or the year ● Wal-Mart and Lowe’s have the greatest number of such responses ● Lowe’s Lowe’s ● School divisions could use the information to contact those who do not respond to further explain the purpose of the follow-up

17. Other Things we do with the Follow-up Data ● We’ve been able to do some other things with the data that did not require an employer response ● In Virginia, students who are employed full- time provide their employer’s address ● That address is used in the employer follow- up, but not all employers respond ● We are still able to use that address even if the employers do not respond ● Applicable to states who perform data matching ● Providing school division with this information makes the 75% response rate standard more bearable

18. Distance Distance between the completer’s school and their employer ● Within the same county or independent city as the completer’s school ● Keep in mind, the distance field is only a proxy variable ● Major weakness: do not know if the completer moved

19. Distance Trends for CTE Completers Working Full-Time Graduation Year Number Employed Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) 20013,96214.252,303 (58%)3,647 (92%) 20023,54014.262,005 (57%)3,258 (92%) 20033,43917.292,031 (59%)3,164 (92%) 20043,98519.212,327 (58%)3,642 (91%) 20054,01117.592,253 (56%)3,670 (92%)

20. Distance and Gender (Full-Time Only) Number Employed Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) Male10,63417.615,960 (56%)9,653 (91%) Female8,30315.224,959 (60%)7,728 (93%)

21. Distance and Relatedness for CTE Completers Employed Full-Time Relatedness to CTE Program Number Employed Full-Time Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) Closely6,49915.023,746 (58%)6,005 (92%) Somewhat5,48617.813,186 (58%)5,010 (91%) Unrelated6,40517.113,661 (57%)5,860 (91%)

22. Distance and Skill Application for CTE Completers Employed Full-Time Skill Application Number Employed Full-Time Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) Most6,31715.413,617 (57%)5,813 (92%) Some5,70617.033,332 (58%)5,251 (92%) Little3,25016.881,866 (57%)2,971 (91%) None3,00617.871,720 (57%)2,734 (91%)

23. Distance and Earnings for CTE Completers Employed Full-Time Hourly Wages Number Employed Full-Time Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) More than $121,46628.51686 (47%)1,229 (84%) $9 to $125,50117.542,889 (53%)4,958 (90%) $6 to $8.999,54114.525,781 (61%)8,922 (94%) Less the $61,11813.79774 (69%)1,059 (95%)

24. Credentials by Service Area Service AreaNumber of Completers Number with Credentials/ License Percent Agriculture6,1112363.9% Business42,1062,3165.5% FCS16,1869194.1% Health5,6441,99435.3% Marketing16,6037464.5% Technology18,9071,0305.5% T & I28,1894,40115.6%

25. Skill Application by Service Area Service AreaPercent Reporting None Percent Reporting Little Percent Reporting Some Percent Reporting Most Agriculture15%17%44%24% Business14%23%44%20% FCS21%20%34%25% Health24%18%16%42% Marketing7%11%41% Technology30%26%33%11% T & I23%18%25%34%

26. Impact of Credentials on Earnings Service AreaPercent of Yes Earning More than $9 Percent of No Earning More than $9 Difference in Percentage Impact Agriculture56%45%+11%+20% Business38%33%+5%+13% FCS25%26%-1%-4% Health37%36%+1%+3% Marketing55%36%+19%+35% Technology52%44%+8%+19% T & I57%47%+10%+18%

27. Conclusions ● Going beyond meeting the basic reporting requirements is difficult; however, the information can readily be used for program improvement ● Providing the information back to those involved in the follow-up process in a useful format gives them more of an incentive to meet response rate objectives ● Using information from previous years to aid local school divisions in meeting their response rate objectives can also be beneficial

28. Contact Information ● Eric Lichtenberger – elichten@vt.eduelichten@vt.edu ● Jim Washington – jwashin@vt.edu