1. School Counselors Using Data Jay Carey and Carey Dimmitt Center for School Counseling Outcome Research UMass Amherst CT Guidance Leaders March 12, 2004

2. Data Can Be Used To: ► Evaluate interventions and programs ► Measure results and outcomes ► Assess cost-effectiveness ► Make decisions ► Monitor student progress ► Change systems

3. Data Can Be Used To: ► Identify needs ► Describe problems ► Discover patterns ► Target interventions ► Identify best practices ► Plan programs

4. Why Use Data? ► Data can convince people of the need to change. ► Data can uncover problems that might otherwise remain invisible. ► Data can confirm or discredit assumptions about students and school practices. ► Data can get to the root of problems, pinpoint areas where change is most needed, and guide resource allocation.

5. Why Use Data? ► Data can help schools evaluate program effectiveness and keep the focus on student learning outcomes. ► Data can provide feedback to counselors, teachers and administrators about what is working, what needs to continue, and what can be cut.

6. Why Use Data? ► Data can prevent over-reliance on standardized test scores. ► Data can prevent one-size-fits-all and quick- fix solutions. ► Data can give schools the ability to respond accurately to accountability questions. ► Data can help build a culture of inquiry and continuous improvement.

7. Why Use Data? Jay Carey’s Postulates ► Focusing on data can free people from guilt and blame and allow them to have conversations that would not happen otherwise. ► Good people can have bad data.

8. Data-Driven School Counseling Programs Data-Driven School Counseling Programs ► Implement comprehensive programs based on national design and local need ► Use data to determine directions ► Measure results ► Share successes

9. What Data Do We Use? ► Student Achievement Data ► Achievement-Related Psychosocial Data ► Career Development Data ► Student Diversity Data ► School Data

10. Student Achievement Data 1. Norm-Referenced Standardized Tests  Scores referenced to national average  PSAT, SAT, ACT, Iowa, Metropolitan  Content related to “invisible national curriculum”  Predictive Validity 2. Criterion-Referenced Standardized Tests  Scores referenced to performance standards  State achievement tests  Content related to state curriculum frameworks  Content Validity

11. Student Achievement Data 3. Performance tests or changes in achievement levels (advancement in Math or achievement levels (advancement in Math or English, for example) English, for example) 4. Portfolios 5. Course grades and GPA 6. Course enrollment patterns 6. Completion of college prep requirements 7. Drop-out rate

12. Achievement-Related Psychosocial Data 1.Attendance rates 2.Behavioral problems 3.Student attitudes 4.Discipline referrals 5.Suspension rates 6.Drug, Tobacco, and Alcohol use patterns 7.Parent involvement 8.Extracurricular activities

13. Career Development Data 1. College Placements 2. Financial Aid Offers 3. Vocational Placements 4. Percentage of students who:  Have 4-year plans  Participate in job shadowing  Have completed career interest inventories

14. Student Diversity Data ► Race ► Gender ► Limited English Proficiency ► English Language Learners ► Free or Reduced School Lunch ► Mobility ► Special Needs ► Achievement Quartile ► Grade

15. School Data ► Course Taking  Gate courses: 8 th Grade Algebra, Algebra 2  Honors and Advanced Placement Courses ► Special Programs  Special Education  Bilingual Education Services  Academic Support Programs  Guidance Programs

16. School Data ► School Culture and Climate  Policies  Expectations  Equity  Safety ► Student-Parent Connection and Engagement

17. School Data ► Curriculum  Rigorous, Meaningful and Inclusive  College Prep (with supports) ► Instruction  Teacher Knowledge  Teacher Expectations  Effective Instructional Practices  Effective Use of Homework  Frequent Evaluation

18. What Are We Doing With All This Data, Again? ► Evaluate interventions and programs ► Measure results and outcomes ► Assess cost- effectiveness ► Make decisions ► Change systems ► Identify needs ► Describe problems ► Discover patterns ► Target interventions ► Identify best practices ► Monitor student progress ► Plan programs

19. Program Evaluation Data ► The ASCA National Model identifies three types of program evaluation data:  Process Data  Perception Data  Results Data

20. Program Evaluation: Process Data ► Process Data: What was done for whom?  Who received services? ► Ninth graders? Students at risk of failing math?  What did they receive? ► Curriculum intervention? Small-group intervention?  When did they receive it? ► All year? Twice? For 30 minutes?  Where and How was it provided? ► In the classroom? After school?

21. Program Evaluation: Process Data ► Process data alone does not tell us whether or not the student is different (in behavior, attitude or knowledge) as a result of this activity. ► Coupled with results data, process data can help identify what factors may have led to success in an intervention.

22. Program Evaluation: Perception Data ► Perception data measures how students are different as a result of an intervention.  Did students gain competencies? ► Every 10 th grade student completed an interest inventory. ► 85% of 6 th graders identified the steps in the conflict resolution process.  Did they gain knowledge? ► 87% of 9 th graders demonstrated knowledge of graduation requirements.  Were there changes in their attitudes or beliefs? ► 86% of students believe smoking cigarettes is unhealthy.

23. Program Evaluation: Perception Data ► Differences in student knowledge, competency and attitudes are measured through:  Pre-post tests ► What do students know/believe before and after the intervention?  Completion of an activity ► Completion of a 4-year plan  Surveys ► What do students say they believe or know?

24. Program Evaluation: Results Data ► Results data is the proof that the intervention has or has not influenced students’ behaviors. An intervention may occur, students may know the information, but the final question is whether or not the students are able to utilize the knowledge, attitudes and skills to affect behavior. ► Attendance, behavior and achievement data are all results data.

25. Program Evaluation: Results Data ► Results data can be complex because many factors impact behavior change. A decrease in disciplinary referrals for bullying may be the result of a violence prevention curriculum, but there are likely to be other things influencing that outcome as well. Conversely, finding no changes in results data does not mean that an intervention has necessarily been unsuccessful.

26. Program Evaluation Summary ► Using process, perception, and results data allows school counselors to find out how school counseling program components are impacting student learning. ► In today’s educational climate, we need to constantly assess the effectiveness of programs and practices. ► Efforts that are not leading to higher academic outcomes for all students need to be re-evaluated.

27. Remember: Disaggregate your data

28. All this will not be finished in the first one hundred days. Nor will it be finished in the first thousand days, nor in the life of this administration, nor even perhaps in our lifetime on this planet. – John F. Kennedy