1. Zarrow Center TAGG: A New On-Line Transition Assessment Jim Martin and Amber McConnell Dept. of Educational Psychology Zarrow Center University of Oklahoma

2. Zarrow Center Agenda 1.TAGG Overview – Jim 2.TAGG Demonstration - Amber 3.TAGG Development 4.TAGG Validity Evidence 5.Obtain the TAGG

3. Zarrow Center TAGG Developed with a Grant from the National Center for Special Education Research and OU Zarrow Center Funds THANKS TO NATIONAL CENTER FOR SPECIAL EDUCATION RESEARCH

4. Zarrow Center TAGG OVERVIEW

5. Zarrow Center TAGG Purpose Assess non-academic skills associated with and predictive of post-school further education and employment Provide IEP team student strengths, needs, a written summary, and annual transition goals matched to common core standards to facilitate writing I-13 compliant IEPs

6. Zarrow Center Whom? TAGG Designed to Assess – Secondary-aged students with IEPs who plan to be competitively employed and/or enrolled in further education after graduation Each TAGG set includes 3 versions – Student – Family – Professional

7. Zarrow Center Versions and Format On-line written English – May be printed and taken by hand, but item scores must be entered into website to produce results – In next few months TAGG versions in Spanish will be added If needed now we have paper versions Users may choose to listen to audio or watch ASL videos for TAGG instructions and items

8. Zarrow Center Reading Level Professional 10.4 grade level Family 5.7 grade level Student 4.8 grade level

9. Zarrow Center TAGG Web-Generated Results Profile Graphic results by constructs Written summary Relative and greatest strengths Relative and greatest needs Annual transition goals Components may be copied and pasted into IEP

10. Zarrow Center TAGG DEMONSTRATION

11. Zarrow Center TAGG DEVELOPMENT

12. Zarrow Center Standards for Educational and Psychological Assessment Guided TAGG Development

13. Zarrow Center Development of TAGG Items TAGG items derived from research studies that identified behaviors of former students with disabilities engaged in post-high school employment and/or further education The research team initially used the research studies to develop – 10 construct definitions – Items developed from constructs 15 iterative TAGG versions were created before field testing began

14. Zarrow Center Initial Structure: Ten Initial Constructs Knowledge of strengths and limitations Actions related to strengths and limitations Disability awareness Employment Goal setting and attainment Persistence Proactive involvement Self-advocacy Supports Utilization of resources

15. Zarrow Center Establish Initial TAGG Structure Users from six states completed the initial (year 1) test- version TAGG – 349 high school students with disabilities – 271 family members – 39 professionals Applied various factor analyses statistics Went from 10 constructs to 8 Went from 75 items to 34

16. Zarrow Center After FA: Professional and Family TAGG Stayed 1.Strengths and Limitations 2.Disability Awareness 3.Persistence 4.Interacting with Others 5.Goal Setting and Attainment 6.Employment 7.Student Involvement in IEP 8.Support Community Dropped 1.Actions Related to Strengths and Limitations 2.Utilization of Resources TAGG-P: (    df=499, RMSEA=.058, CFI=.92, TLI=.91, RMSR=.0597) TAGG-F: (  2 =862.74, df=499, RMSEA=.057, CFI=.91, TLI=.90, RMSR=.058)

17. Zarrow Center Student Version Constructs After FA After FA Constructs 1.Strengths and Limitations & Support Community 2.Disability Awareness 3.Persistence 4.Student Involvement in IEP 5.Interacting with Others 6.Goal Setting and Attainment 7.Employment Dropped Constructs 1.Actions Related to Strengths and Limitation 2.Utilization of Resources Combined Constructs 1.Strengths and Limitations 2.Support Community TAGG-S: (  2 =819.00, df=505, RMSEA=.047, CFI=.89, TLI=.88, RMSR=.064)

18. Zarrow Center Replicate Factor Structure Two additional studies from TAGG users across the country Results confirmed strong factor structure

19. Zarrow Center Three Years of Data Collection 2,556 participants from 42 states and 162 schools – 1,291 secondary students with disabilities who had postsecondary further education and/or competitive employment goals – 172 professional completed TAGG on 7 to 8 of their students – 847 family members completed TAGG on their child

20. Zarrow Center IN BRIEF: VALIDITY EVIDENCE

21. Zarrow Center Internal Reliability Generally, a score between.7 and.8 is considered “good” – Each TAGG version has great overall internal consistency and satisfactory subscale consistency (ranging from α =.89 to α =.95)

22. Zarrow Center Test-Retest Reliability Scores of.7 or higher represent good or satisfactory test-retest reliability – 14 weeks after the first TAGG was completed, same users completed the TAGG again – A large correlation was found between the first and second administrations.80 for professional TAGG.70 for family TAGG.70 for student TAGG

23. Zarrow Center Fairness Validity Evidence: Gender Do differences exist by gender? – No overall difference by gender on TAGG-P, TAGG-S – On TAGG-F small overall differences – Some construct differences exist. On TAGG-S – females rated themselves higher on student involvement in IEP than males – males rated higher on employment

24. Zarrow Center Measurement Invariance Structure held across all disability categories The three TAGG versions are appropriate for students regardless of – Students’ time in general education classes – Number of transition education classes completed – High School grade level – Disability category

25. Zarrow Center Fairness Validity Evidence: Disability Categories Some construct differences do exist, but these are small differences – 5 Disability Categories were included Autism, ED, ID, OHI, and SLD – On most constructs OHI and LD scored higher than Autism, ED, and ID

26. Zarrow Center Students with ID Scored Higher TAGG-P and TAGG-F – Scored > students with Autism and ED in Interacting with Others TAGG-P – Scored > students with ED in Support Community TAGG-F – Scored > students with Autism in Disability Awareness and Involvement in the IEP

27. Zarrow Center Areas Students with ID Scored Lower TAGG-P and TAGG-S – Scored < OHI and SLD in Strengths and Limits TAGG-P and TAGG-F – Scored < SLD in Persistence TAGG-P – Scored < OHI and SLD in Disability Awareness, Goal Setting, and Involvement in the IEP

28. Zarrow Center Areas Students with ID Scored Lower TAGG-F – Scored < SLD in Goal Setting, Involvement in the IEP, and Support Communities TAGG-S – Scored < OHI in Interacting with Others and Goal Setting – Scored < OHI and SLD in Employment

29. Zarrow Center Students with Autism Students with Autism scored themselves as having no problems with interacting with others. – (Professionals and Family members scored students with Autism lower than ID, OHI, SLD) Students with Autism did not score significantly higher than any other category on any construct.

30. Zarrow Center Students with Autism Scored Lower TAGG-P – Scored < ID, OHI, and SLD in Interacting with Others – Scored < OHI and SLD in Strengths and Limits, Goal Setting, and Involvement in the IEP – Scored < SLD in Persistence, Employment, and Support Community

31. Zarrow Center Students with Autism Scored Lower TAGG-F – Scored < ID, OHI, and SLD in Strengths and Limits, Disability Awareness, Interacting with Others, and Involvement in the IEP – Scored < OHI and SLD in Support Community – Scored < SLD in Persistence and Goal Setting

32. Zarrow Center Students with Autism Scored Lower TAGG-S – Scored < OHI and SLD in Employment and Involvement in the IEP – Scored < SLD in Persistence

33. Zarrow Center Free/reduced lunch eligibility No significant differences for construct scores on TAGG-P or TAGG-S. Only small differences for TAGG-F scores. Family employment No significant differences for construct scores Family education Significant differences- Highest family education lower TAGG scores Fairness Validity Evidence: SES

34. Zarrow Center How Close Are Students, Professionals, and Family TAGG Scores? How closely do the different TAGG versions assess the same student? – Medium correlations across Parent, Educator, and Student versions when assessing the same student – This is excellent for this type of assessment

35. Zarrow Center TAGG & AIR Self-Determination Assessment Same users completed TAGG and AIR Self- Determination Assessment – Medium Correlation This implies the TAGG addresses some self- determination skills and assesses other skills, too.

36. Zarrow Center Predictive Validity Process Follow-up of 297 former high school students who completed the TAGG while in high school Logistic regressions examined relations between TAGG non-academic behavior constructs and postsecondary education and employment

37. Zarrow Center Constructs Predicting Further Education Interacting with Others Student Involvement in the IEP Support Community Goal Setting and Attainment

38. Zarrow Center Constructs Predicting Employment Employment Student Involvement in IEP Support Community Interacting with Others

39. Zarrow Center Item Response Theory Advantages of IRT include – The ability to scale different item types – Provides a common metric for scales with different number of items – Weights items differentially by their validity for assessing the construct of interest

40. Zarrow Center IRT Algorithms Produce Results Profile 1.Placed each scale onto a common score metric 2.Projected item characteristics (e.g. item difficulty) onto the scale score metric 3.Conducted a within-student comparison of scale scores across constructs to determine relative strengths and weaknesses 4.Conducted a within-construct comparison of a student’s scale score to item responses (e.g. difficulty) to generate appropriate goals for identified weaknesses

41. Zarrow Center Overall Score The overall score is a weighted combination of all items. The overall score is not an average of all the construct scores.

42. Zarrow Center OBTAIN THE TAGG

43. Zarrow Center TAGG Details $3 per set (Professional, Student, Family versions) – Used to pay for on-going TAGG development and operational costs TAGG profiles saved for 7 years Data kept on OU high-speed secure cloud servers Purchased credits may be transferred to other registered TAGG users

44. Zarrow Center TAGG Web Site Location 1.The OU Zarrow Center’s Web Page http://zarrowcenter.ou.edu 2.The TAGG Section of the ZC Web Page https://tagg.ou.edu/tagg/

45. Zarrow Center The On-Line TAGG and Result Profile SAMPLE TAGG SCREEN SHOTS

46. Zarrow Center

47. Disability Awareness Profile

48. Zarrow Center Combined Score Profile

49. Zarrow Center Greatest and Relative Strengths

50. Zarrow Center Areas of Greatest and Relative Need

51. Zarrow Center Summary Statement for IEP Chad Bailey’s skills were assessed using the TAGG, a norm-referenced assessment with research-based items known to be associated with post-school employment and education. Compared to similar students, Chad’s scores are average. Results indicate greatest strengths are in the areas of Goal Setting and Attainment. Chad’s relative strengths include Disability Awareness and Student Involvement in the IEP. Greatest needs are in the area of Strengths and Limitations, with Employment being a relative need.

52. Zarrow Center Suggested Annual Transition Goals To prepare for success in employment, the student will write an essay describing three situations where the student used his or her strengths with 90% grammar and context accuracy by the end of the essay writing unit.

53. Zarrow Center Contact Information Jim Martin jemartin@ou.edu Amber McConnell ambermcc@ou.edu Phone: (405) 325-8951 Website: https://tagg.ou.edu/tagg/https://tagg.ou.edu/tagg/