Virginia’s Employer Follow-up: An Examination of Response and Non- Response Patterns Presenters: Eric Lichtenberger Jim Washington

Organization of Presentation ● Introduction ● Description of follow-up procedures ● Adaptations over the past few years to improve response rates ● Going beyond reporting requirements (providing useful information to stakeholders)

Brief History of Our Research Center ● Center for Assessment, Evaluation, and Educational Programming (CAEEP) ● Virginia Tech’s School of Education’ Office of Education Research and Outreach ● Conducting the annual follow-up for the Virginia Office of Career and Technical Education Services for 30 years ● We also perform similar work for the Virginia Department of Adult Education and Literacy and the Virginia Department of Correctional Education

Overview of the Follow-up ● Local school divisions are highly involved in the follow-up process ● Several years ago we moved towards a flexible on-line reporting system that blends paper based, on-line, and telephone data collection. ● Currently we have 5 years of parallel data ● 30,000 CTE completers (concentrators) and 4,500 employers ● Description of survey Description of survey

Improvements to the Process ● Developed the Address Checker Tool ● Used Division Letters, Division Envelopes, and Allowed Divisions to Conduct their Own Employer Follow-up ● Obtaining Employer Phone Numbers ● Establishing 75% as the standard response rate

Summary Numbers from the 2006 Survey ● State Numbers: Divisions and Regional Centers may obtain specifics for their area. – 41.15% of 4,578 employers responded. – 30.23% of 4,578 employers provided usable responses. – 89.52% of 1,384 employers were either Satisfied or Very Satisfied with the Employee's high school preparation – These numbers have steadily increased since 2002

Summary Numbers from the 2006 Survey ● 35.48% of 1,384 employers rated the employee's high school preparation as Equally Prepared ● 44.29% of 1,384 employers rated the employee's high school preparation as Better Prepared ● 79.77% of 1,384 employers were either Satisfied or Very Satisfied with the employee's Technical Skills as they related to the job

Employer Comments ● Survey question asks for specific comments in reference to the Workplace Readiness Skills, categorized by “Satisfied” and “Unsatisfied” ● About half of the respondents provide comments ● Average of three comments per survey that has comments ● Comments tend to be 4-to-1 favorable ● We let the computer do the summarizing

Comments Summary ● Satisfied – Attitude – Work Ethic – Computer Skills – Attendance – Teamwork – Working with Others – Math – Reading – Quick Learner ● Dissatisfied – Attendance – Writing – Decision-making – Initiative – Math – Spelling – Work Ethic – Listening – Attitude – Working with Others

Satisfaction Vs. Applicability ● “Not Applicable” ● Low N/A = High Applicability ● High N/A = Low Applicability ● Satisfaction Where Applicable, e.g., ( )/( )

Satisfaction Where Applicable ● 97% Reading ● 93% Math ● 93% Computer ● 92% Technical Skills ● 91% Writing ● 90% Teamwork ● 89% Attendance ● 89% Pos. Attitude ● 89% Speaking & Listening ● 86% Work Ethic ● 86% Self- Presentation Skills ● 85% Reasoning and Problem-solving ● 83% Independence and Initiative ● 81% Big Picture

Applicability ● 94% Attendance ● 93% Work Ethic ● 92% Pos. Attitude ● 92% Teamwork ● 90% Independence & Initiative ● 90% Speaking & Listening ● 87% The Big Picture ● 87% Reasoning & Problem-Solving ● 85% Technical Skills ● 84% Reading ● 79% Math ● 79% Writing ● 62% Computer

Address Checker Tool ● Local school divisions are required to verify employer addresses ● Employer response rate has improved since implementing this tool ● Once an employer address is entered the information is compared against a list of deliverable addresses ● There are numerous outcomes based on that comparison

EMPLOYER MAIL ADDRESS STATUS - STATE REPORT Address StatusNumberResponsesResponse Rate Bad state abbreviation11100% Insufficient address % Missing street directional7457% No street address number % Not found8338% Out-of-state371232% Possibly deliverable % Probably not deliverable311342% Street not found % Unknown Status513671% Verified OK % Statewide %

2006 Employer Response Rates by Follow-up Procedure ProcedureEmployersResponse rate Traditional314736% Division Envelope46929% Division Letter4262% Division Conducted105657%

Employers that Traditionally Do Not Respond ● Created a database of employers who due to company policy, do not respond ● Categorized the information by year and branch ● Some companies are consistent and with others it depends on the branch or the year ● Wal-Mart and Lowe’s have the greatest number of such responses ● Lowe’s Lowe’s ● School divisions could use the information to contact those who do not respond to further explain the purpose of the follow-up

Other Things we do with the Follow-up Data ● We’ve been able to do some other things with the data that did not require an employer response ● In Virginia, students who are employed full- time provide their employer’s address ● That address is used in the employer follow- up, but not all employers respond ● We are still able to use that address even if the employers do not respond ● Applicable to states who perform data matching ● Providing school division with this information makes the 75% response rate standard more bearable

Distance Distance between the completer’s school and their employer ● Within the same county or independent city as the completer’s school ● Keep in mind, the distance field is only a proxy variable ● Major weakness: do not know if the completer moved

Distance Trends for CTE Completers Working Full-Time Graduation Year Number Employed Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) 20013, ,303 (58%)3,647 (92%) 20023, ,005 (57%)3,258 (92%) 20033, ,031 (59%)3,164 (92%) 20043, ,327 (58%)3,642 (91%) 20054, ,253 (56%)3,670 (92%)

Distance and Gender (Full-Time Only) Number Employed Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) Male10, ,960 (56%)9,653 (91%) Female8, ,959 (60%)7,728 (93%)

Distance and Relatedness for CTE Completers Employed Full-Time Relatedness to CTE Program Number Employed Full-Time Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) Closely6, ,746 (58%)6,005 (92%) Somewhat5, ,186 (58%)5,010 (91%) Unrelated6, ,661 (57%)5,860 (91%)

Distance and Skill Application for CTE Completers Employed Full-Time Skill Application Number Employed Full-Time Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) Most6, ,617 (57%)5,813 (92%) Some5, ,332 (58%)5,251 (92%) Little3, ,866 (57%)2,971 (91%) None3, ,720 (57%)2,734 (91%)

Distance and Earnings for CTE Completers Employed Full-Time Hourly Wages Number Employed Full-Time Average Distance Number Employed within Same Area (%) Number Employed within 25 Miles (%) More than $121, (47%)1,229 (84%) $9 to $125, ,889 (53%)4,958 (90%) $6 to $8.999, ,781 (61%)8,922 (94%) Less the $61, (69%)1,059 (95%)

Credentials by Service Area Service AreaNumber of Completers Number with Credentials/ License Percent Agriculture6, % Business42,1062,3165.5% FCS16, % Health5,6441, % Marketing16, % Technology18,9071,0305.5% T & I28,1894, %

Skill Application by Service Area Service AreaPercent Reporting None Percent Reporting Little Percent Reporting Some Percent Reporting Most Agriculture15%17%44%24% Business14%23%44%20% FCS21%20%34%25% Health24%18%16%42% Marketing7%11%41% Technology30%26%33%11% T & I23%18%25%34%

Impact of Credentials on Earnings Service AreaPercent of Yes Earning More than $9 Percent of No Earning More than $9 Difference in Percentage Impact Agriculture56%45%+11%+20% Business38%33%+5%+13% FCS25%26%-1%-4% Health37%36%+1%+3% Marketing55%36%+19%+35% Technology52%44%+8%+19% T & I57%47%+10%+18%

Conclusions ● Going beyond meeting the basic reporting requirements is difficult; however, the information can readily be used for program improvement ● Providing the information back to those involved in the follow-up process in a useful format gives them more of an incentive to meet response rate objectives ● Using information from previous years to aid local school divisions in meeting their response rate objectives can also be beneficial

Contact Information ● Eric Lichtenberger – ● Jim Washington –