Use of Portfolios in Specialist Training Programs to Demonstrate Positive Impact on Student Learning Laurie McGarry Klose, Ph.D., LSSP Associate Professor and Director, School Psychology Trinity University San Antonio, TX

Learning Objectives Participants will learn how portfolio products can be utilized to demonstrate measurable positive impact on students. Participants will observe the utility of electronic portfolio platforms for managing program approval data.

Agenda Review course requirements preparing future interns to evaluate impact of services. Examine internship requirements related to demonstrating positive impact. Look at information from Trinity's last NASP approval process. Discuss generation of impact data charts. Questions and comments.

Measurable Positive Impact Data

Coursework Preparation Counseling Class Behavior Analysis and Intervention Class Advanced Statistics Class Research Evaluation Class

Internship Requirements Completion of two consultation cases and reports- one that addresses academic concerns and one that addresses behavioral problems. Your cases should use single-case designs and document positive impacts on student learning and behavior, respectively. Reports should include goals, charts, graphs and specific data. Completion of two counseling cases- individual and/or group. Document with summary of goals and outcomes of both. One case must be summarized in a report using Goal Attainment Scaling and will be assessed using the attached rubric.

Documentation Assessment 6

This assessment is based on data collected by students during Internship and presented in their Portfolios. As described in Assessment 5, Interns are required to complete several specific types of cases during this year. A minimum of two consultations are expected and the process of determining outcomes is specifically taught prior to this final course. Experience and practice with this is begun with the first year spring class ED 5384 Behavior Management and Special Education with the behavior change project. The fall second year course ED 6381 Research Methods II has a case study project as the major assignment as does the second year spring class ED 6380 Consultation Theory and Methods. One of the Internship consultations must address a behavior problem and the other an academic problem. A sequence and approach with required statistics is familiar to the students and the rubric has also been utilized previously in Practicum; however, at this point, a positive outcome is expected which must be demonstrated through both Goal Attainment Scaling and an Effect Size calculation. In addition, a Social Validity scale and a Behavior/Academic Intervention Rating Scale must be completed by the consultee.

Included are the data from two cohorts Included are the data from two cohorts. In the first class, there were problems with one of the outcome scores for several cases; however at least one outcome score was accurately presented for each case. Faculty learned from the procedural weaknesses apparent in these case presentations and the second group’s data met our expectations completely. Most importantly, all the cases demonstrated a positive impact on students’ academic and behavioral performance in school settings.

2012-2013 Academic Consultation Cases Intern   Student’s Grade Problem/Goal BIRS Goal Attainment Scale Effect Size DB 3rd Turn in assignments 5.71 +2 5.23 MC 5th Use math strategies 4.14 +1 12.6 TC 1st Word recognition 5.29 6.31 LM 9th Work completion 4.7 2.5 AM 11th 4.29 2.68 EM K Handwriting improvement 4 DR Utilize ELL strategies 4.71 6.43 LS Subtraction using manipulatives 5 * 7.67 TY 2nd Math facts 6 8.16 Means 4.87 +1.86 6.25

2012-2013 Behavioral Consultation Cases Intern Student’s Grade Problem/Goal BIRS Goal Attainment Scale Effect Size DB 1st Increase on-task behavior 5 +2 6 MC 5th Attempt work 5.3 * TC 3rd Remain seated 5.29 +1 11 JG Decrease off-task behavior 3.14 AM 8th Compliance with adult requests 5.14 1.97 EM 2nd Reduce off-task behaviors 4.28 DR Increase on-task behaviors 5.33 4.7 GT 4th Focus on work 4 TY Decrease task refusal 3.6 Means   4.79 +1.56 5.53

2013-2014 Academic Consultation Cases Intern Student’s Grade Problem/Goal BIRS Goal Attainment Scale Effect Size CB 2nd Increase reading fluency 5 +1 3.9   ED 3rd Increase math skills 4 2.84 EF Increase addition skills 6 +2 6.9 LF 12th Completing work 7.78 RF 1st Letter identification 3.8 2.9 CL Reading fluency 3.86 2.29 DM Answering math problems 3.88 4.29 LP 5th Using tutoring services 3.57 12.5 AW Increase sight words 5.9 Means 4.56 +1.33 5.08

2013-2014 Behavioral Consultation Cases Intern Student’s Grade Problem/Goal BIRS Goal Attainment Scale Effect Size CB K On-task behavior 4.7 +1 3   ED 3rd Time in class 2.9 +2 8.87 EF 4th au Remaining calm 5.42 .9 LF 3rd au Eat lunch 5 2.89 RF 6th On-task behaviors 4 1.4 CL 5th Time on task 4.2 2.01 DM 4th Increase time on task 4.43 3.15 LP Increase coping skills 4.29 2.83 AW 2nd Follow school rules 5.9 3.8 Means 4.54 +1.22 3.2

NASP Program Approval Board Response Element 9.6 is rated as Acceptable. The school psychology program employs a systematic and valid process by which program faculty ensures that interns demonstrate competencies to begin effective practice as school psychologists, evidenced by direct, measurable, positive impact on children, families, schools, and other consumers. The program provides comprehensive data that candidates demonstrate positive effects on P- 12 student learning. Candidates analyze data from case studies and are required to select and compute effect size and goal attainment scaling.

Comments, Questions, Discussion