Reporting Post-school Outcome Data May 2008 Cinda JohnsonMary Kampa Center for Change in Wisconsin Post High Survey/ Transition Services Indicator 14 Seattle University CESA #11 Seattle, WA Turtle Lake, Wisconsin

History: Washington Data collection beginning in 1980 with Edgar’s work Consistent post-school outcome data collected since 1996, (Edgar, et al.) Consensus data collection (all districts, all leavers) beginning in school districts, 5,000 leavers 80% contact rate

Methods: Washington Participants: Special education graduates and drop-outs from all 248 school districts. Annually: Demographic information collected in year prior to leaving school. Information collected from final IEP. Telephone survey conducted with youth or family member within one year of graduating or dropping out (WA has one diploma).

Methods: Washington Washington State Post-school Survey ( ) Secure password protected website

Instrumentation: Washington Survey questions includes:  Demographics including gender, age, disability  Post-secondary goals from final IEP  Agency linkages on final IEP Data gathered includes:  Post-school outcomes in post-secondary education, training and employment  Details of those outcomes (type of school, training program, job, wages, hours, etc.)  Agency linkages

Procedures: Washington Training: Data manager at state level Data manager at district level Data users at district level Confidentiality, assure consent IRB agreements

Procedures: Washington Districts are responsible to conduct interviews with former students. Teachers conduct the majority of interviews. Training for interviewers is provided on-line, teleconferences, on site, meetings, trainings, and conferences. At least 3 attempts made to reach youth during different times of day.

Table School Staff Completing Post-School Surveys 2006 Graduates Interviewed Staff PositionNumberPercent* Teacher1,23746% Administrator713% Paraprofessional/Educational Assistant80030% Administrative Assistant27110% Transition Specialist472% School Psychologist622% Other2208% Total2,708100% *May not add to 100% due to rounding. **Excludes 609 with missing information. Source: CCTS, Special Education, June 2007.

Reporting Post-school Data: Washington Reports to state and districts. Data disaggregated by district, region, county, high school. Outcomes compared to state and previous years. Data reported to agencies (DVR, DDD), Governor’s office, ESDs and parent groups. Assure confidentiality (cell size).

Reporting Data for Program Improvement: Washington Leadership Facilitator Examine the data with colleagues Participate in surveying former students and share the stories Develop goals based on the data Tie post-school outcome data to school improvement activities

Post-school Outcome Data: Washington Collecting (by whom and when; from whom) Analyzing (representative of pool) Reporting (LEA’s, SEA, Public, APR, SPP) Setting goals and benchmarks (SEA, LEA’s) Informing practices Improving outcomes

Using Post-school Outcome Data to Inform Change: LEA, Washington Competitive employment for youth with developmental disabilities was low Met with agencies to clarify language and definitions (competitive employment) “Carved-out positions” were not leading to competitive employment Assessed OJT’s with agencies and modified to meet criteria of skills defined for competitive employment

History: Wisconsin Began in 2001 with exiters Consistent core questions : based on NLTS and other surveys at the time, including WA, TX and OR Dual data collection method: Statewide sample (odd years) and individual district census (even years) 400 school districts in state  Nearly 8,000 exiters with disability annually  app. 81 districts and Milwaukee (13 schools) each year  Census within selected districts 70% response rate (districts = 80%); lower in 2007 (38%) due to change in method

Methods: Wisconsin Each LEA in Wisconsin must participate in an outcomes survey once between and to comply with SPP Indicator #14. LEA participation is aligned with the DPI Self- Assessment Monitoring Cycle Within one year of exiting, contact former students who exited with IEP and:  A regular diploma  A certificate of attendance  Reached maximum age of eligibility  Dropped-out

Methods: Wisconsin The Wisconsin Post High School Outcomes Survey (WPHSOS) ( is a web-based tool designed to ensure consistency in data collection and reporting requirements of SPP Indicator 14. Outcomes information is collected from former students and their families through a telephone interview conducted on behalf of the former student’s high school by a professional survey center.

Methods: Wisconsin District results can be viewed immediately and used for transition data analysis and planning LEAs may also use the WPHSOS website to conduct their own surveys or include exiters without disabilities when not participating in the statewide survey

Procedures: Wisconsin Work with school districts to prepare them for a two year data collection process  Year 1: Districts - Collect contact information all year for youth in final year Track drop-out contact information Explain upcoming survey to youth and parent Give questionnaire to youth and parent view and use resourceswww.psocenter.org  Year 2: Districts - Feb: Verify former student contact information March: Send former students district and SEA letters April-June: LEAs surveyed/view survey results

Procedures: Wisconsin SEA Data manager prepares exiter information based on LEA Exiter Report Data transferred through secure FTP site to outcomes website Interviewers and district apply for user name and password; sign confidentiality agreement

Procedures: Wisconsin attempts made to reach each exiter; different times of day, weekends, special operators, other languages, jail, military Responses also accepted from family member if knowledgeable about HS and current activities Training for district interviewers is provided using developed training materials

Instrumentation: Wisconsin Survey questions include :  Living situation, community participation, social activities, adult agencies and community supports  Postsecondary education and training – types, accommodation and disclosure  Employment – type, setting, hours, wages, benefits, accommodations  HS experiences/IEP plans  Open-ended questions – if not living, working or going on to school, why not?  What is something positive that happened in HS that helped you met your goals?

Post-school Data: Wisconsin LEAs access outcomes data at district and building level as it is collected Statewide data available Sept. 1 All survey questions disaggregated by gender, ethnicity, disability, exit type, HS Can create additional reports by region, county, CESA, school size, other as requested Cell size for confidentiality is 5

Post-school Data: Wisconsin Only Indicator 14 “%” is publicly reported LEA is provided:  District GEDE Report  District Summary Report  District Report Starter  Improvement Planning Tools Youth Leadership Council will provide youth prospective and take an active role in co- presenting with WPHSOS director at regional trainings and conferences

Post-school Data: Wisconsin Districts collect outcomes data because they have to Districts use outcomes data because they  have to  want to

Post-school Data: Wisconsin HAVE to:  Focused-Monitoring – identified by DPI as in need of focused- monitoring; districts are required to participate in improvement planning strategies (e.g. Indicator 1 - Graduation Rate)  Indicator 14 Districts – identified by DPI as “in need of assistance” e.g. districts 30% or more below State Indicator 14 WANT to:  Districts seeking assistance with data analysis – statewide data tools can be easily used in districts or CESAs involved in data analysis, improvement planning or retreats  WSTI TAT Districts – Participate in the development of a pilot professional development that includes data analysis for the Year 2 report-out (mini-data retreat - LEAs use their data to identify weaknesses and improvement strategies)

Goals of the WPHSOS: Wisconsin The WPHSOS will:  Align identified goals and activities of the post school follow-up project with WSTI (Wisconsin Statewide Transition Initiative) and SPDG (State Professional Development Grant), using a complementary technical assistance approach among projects to help districts use data to improve outcomes  Provide technical assistance to move the outcomes website from a data collection and reporting tool to a tool LEAs use to identify local needs and determine where improvement strategies are needed to positively impact Stakeholder Advisory identified Indicator 14 SPP Targets  Develop statewide information sharing, resources and sustained professional development on Indicator 14

Data Sharing: Wisconsin Several important pieces in place at the current time:  Indicator 1 and 2 data are shared with the post high website, and are connected with Indicator 14 outcomes reporting  Indictor 13 data and Indictor 14 data share a common web designer  Databases are connected currently create a joint report: Indicator 1, 2, and 14 data are imported into the “Transition in IEP Checklist Report”  Individual student identification numbers (though not yet shared with the websites)

New Direction: Wisconsin Taking the website from data collection to data use helping districts go from “why do we need to collect this information?” to “how can we use our results to effect change?” State & Compliance (expert) District (expert) Administration Teachers Youth Parents Community

New Direction: Wisconsin Dissemination  Shift from working with just directors of special education (because it is password protected information and “special education”) to information dissemination to all district administration: State School Board Convention State Superintendent’s Conference State Secondary Principal’s Conference State School Psychology Convention

Sharing Post-school Data: Wisconsin Conferences/Poster Sessions/General Information  National, State/Regional, Local State Superintendent’s Leadership Conference WI. Council of Administrators of Student Services Wisconsin Rehabilitation & Transition Conference Wisconsin Statewide Transition Conference DPI presentations and Stakeholder Advisory Meeting CESA board/RSN meetings upon request Department of Workforce Development Department of Health and Human Services State Sheltered Workshop Group

Sharing Post-school Data: Wisconsin Conferences/Poster Sessions/General Information TAC, TAN and TAT meetings ( upon request (e.g. Washburn Co. - transportation grant and Barron Co. – county council activities) Others upon request and availability Community of Practice Groups/ Work Groups  Wisconsin Community on Transition and Practice Groups (  SPDG – statewide transition project, data, newsletters, conference planning, needs assessment Print Materials/Resources  Resources/Professional Development  Statewide Outcomes/Indicator 14 Reports/Newsletters

Wisconsin Indicator 14 Any Postsecondary Ed./Training by Survey Year

Wisconsin Indicator 14 Postsecondary Education Summary Overall participation in postsecondary education and training has been consistent over time (46% - 48%) Types of postsecondary education have changed  a higher % in 2-Yr, 4-Yr and Tech College  a higher % of female exiters participating  significant increase in participation in Tech College  participation in 4-Yr increasing more than 2-Yr Inclusion of dropouts did not affected the overall % of participation (46% vs 47%), although as a group, dropouts participate less in all types of postsecondary ed./training

Wisconsin Indicator 14 Any Employment by Survey Year

Wisconsin Indicator 14 Employment Summary Employment from to decreased but rebounded in Employment over time:  Employment in the community has increased  Hours per week worked have remained stable  Wages have increases slightly Beginning with , “competitive employment” to be included in employment, for Indicator 14

Wisconsin Indicator 14 Employment Summary Several HS indicators have changed  Fewer report paid employment while in HS  Fewer are obtaining a valid driver’s license in HS Inclusion of drop-outs did not affected the overall % of those competitively employed (33% vs. 34%) As a group, dropouts are employed at comparable rates, but a higher %  work more hours per week  earn less per hour

Wisconsin Indicator to Major Exiter Outcomes Data (n = 389) (n = 600) (n = 725) (n = 358) Postsecondary ed./training only11% 9% Employment only36%34%33%39% Competitive employment only---33% Current postsecondary ed./training and current employment34%38%29%28% Current postsecondary ed. and current competitive employment---10% Some Postsecondary ed. and/or employment88%87%84%89% Never any postsecondary ed. or employment12%13%16%11% Never any postsecondary education or competitive employment since HS---35% Indicator 14 (postsecondary ed. or training, and/or competitive employment since HS)---65%

Wisconsin Indicator 14 Summary of Major Outcomes 79% = 283 of youth who had IEPs and are no longer in secondary school have been competitively employed (full-time or part-time), enrolled in some type of postsecondary school, or both, within one year of leaving high school divided by the 358 youth assessed who had IEPs and are no longer in secondary school. 283/358 = 79% 2007 Report of Indicator 14 Baseline Data (n = 358) t

Wisconsin Indicator 14 Summary of Major Outcomes Data collection on youth with disabilities one year after exiting high school indicates:  There is a consistent % of youth attending postsecondary education or training over time  The % of youth employed increased last year, following three years of decline in employment  High school factors may influence competitive employment post high school  Fewer than ½ of youth who are employed are “competitively employed”

IDEA Partnership Grant: Wisconsin Review the available evidence-based practices and research on NSTTAC, NDPC-SD, NPSO, and other sources, addressing both rural and urban concerns Begin creating a database of frameworks, strategies and factors that are connected to post high outcomes Begin creating a tool LEAs and teachers can easily access and use in data analysis and planning of professional development activities

IDEA Partnership Grant: Wisconsin Synthesize this information and identify:  District/building level: e.g. curriculums, engagement and school climate surveys, how and what HSs implement in their buildings (may be related to HS Re-design), connections with Indicators 1 and 2  Teacher/classroom: material and activity specific: e.g. programs, curriculums, activities  Youth /parents/family: related to known risk or success factors: e.g. HS employment, not failing more than two classes, attendance, reading scores, poverty factors, information from Indicator 13, state or district assessment scores, IEP goals, specific transition or outside agency services, senior exit survey  Community/Adult Service Providers: DHFS, DVR, employment agencies, independent living centers, etc.

Final Thoughts - Washington Post-school outcomes may not increase at the aggregate level  More respondents (harder to reach included)  Dropouts  Economy Attention to Indicator 14 at disaggregate level may improve Indicator 13

Final Thoughts - Wisconsin COLLECT: Beyond the Indicator 14 percentage, get outcomes data to the LEA and teacher level USE: Once district is looking at the data, have improvement resources readily available SHARE: with statewide and regional partners – help them access and use data IMPROVE: Local improvement will lead to increased state Indicator 14

Contact Information Cinda Johnson, Ed.D. Seattle University Center for Change in Transition Services

Contact Information Mary Kampa, CESA #