The DESSA Comprehensive System: A New Tool to Assess Social-Emotional Strengths & Resilience Discuss Devereux, ICTR. Discuss our history doing pathology-based.

The DESSA Comprehensive System: A New Tool to Assess Social-Emotional Strengths & Resilience
Discuss Devereux, ICTR. Discuss our history doing pathology-based assessments. Change to strength-based. Paul A. LeBuffe, Devereux Center for Resilient Children Valerie B. Shapiro, University of Washington Jack A. Naglieri, University of Virginia & Devereux Center for Resilient Children © 2010 Devereux Center for Resilient Children

Today’s Goals Review the context for the DESSA
Present the DESSA Comprehensive System Discuss applications in the schools

The Devereux Center for Resilient Children

© 2010 Devereux Center for Resilient Children
DCRC Mission The mission of the DCRC is to promote social and emotional development, foster resilience, and build skills for school and life success in all children and the adults who care for them. © 2010 Devereux Center for Resilient Children

What is Resilience? Resilience means the personal and community qualities that enable us to rebound from adversity, trauma, tragedy, threats, or other stresses - and to go on with life with a sense of mastery, competence, and hope. New Freedom Commission, 2003

Use of word “Resilience”
© 2010 Devereux Center for Resilient Children

Established Recovery and Resilience as the mental health goals for the United States Established Recovery and Resilience as the mental health goals for the US © 2010 Devereux Center for Resilient Children

The Context (continued)
Social & Emotional Learning © 2010 Devereux Center for Resilient Children

The Collaborative for Academic, Social and Emotional Learning
Based at the University of Illinois – Chicago Research and public policy advocates for “Social and Emotional Learning” or SEL Recent Meta-analyses of the research literature © 2010 Devereux Center for Resilient Children 10

CASEL (continued) “CASEL Framework” – 5 key social-emotional skills for school and life success Primary basis of the DESSA scales CASEL Framework is being incorporated into state and local educational standards CASEL taking a leadership role in writing “common core” SEL standards © 2010 Devereux Center for Resilient Children 13

Federal SEL Legislation
HR 4223 – “The Academic, Social and Emotional Learning Act.” Establish a National Technical Assistance and Training Center for SEL Provide grants to support evidence-based SEL programs Conduct a national evaluation of SEL programs © 2010 Devereux Center for Resilient Children

SEL Standards Established
Illinois Kansas* Ohio Oklahoma* New York Tennessee* Washington State Vermont Anchorage, AK New Jersey (proposed) British Columbia Pennsylvania (in committee)

Does Resilience Matter?
Results of a predictive validity study…

The Context (continued)
NASP Integrated and Comprehensive School Psychological Services Model © 2010 Devereux Center for Resilient Children

NASP’s Integrated and Comprehensive School Psychological Services Model (SPR, 2010, 39, (2), ) Identifies 10 domains that, “provide a general framework of basic competencies” “effective strategies and skills ..to help (all) students success academically, socially, behaviorally and emotionally.” “All school psychologists are expected to possess at least a basic level of competency in all of the 10 domains”

Data-Based Decision Making and Accountability
NASP’s Integrated and Comprehensive School Psychological Services Model (cont) Data-Based Decision Making and Accountability “School psychologists have knowledge of varied models and methods of assessment and data collection methods for identifying strengths and needs..” School psychologists use valid and reliable assessment techniques”

NASP’s Integrated and Comprehensive School Psychological Services Model (cont)
Interventions and Mental Health Services to Develop Social and Life Skills “School psychologists have knowledge of…evidence-based strategies to promote social-emotional functioning and mental health” “Use assessment and data-collection methods.. that support socialization, learning and mental health.”

Preventive and Responsive Services
NASP’s Integrated and Comprehensive School Psychological Services Model (cont) Preventive and Responsive Services “School psychologists have knowledge of principles and research related to resilience and risk factors in learning and mental health..” “School psychologists promote recognition of risk and protective factors” “School psychologists promote wellness and resilience”

MEB Disorders 20 – 25% of school-aged children Number is growing
247 Billion dollars per year Hardship for students, parents and schools

What if we could quickly, efficiently and accurately identify those children at risk of developing MEB disorders and intervene before they begin to exhibit the problem behaviors?

The Devereux Comprehensive System

The Goal Strength-based assessment of behaviors related to social and emotional well-being Simple, practical, and easy to use Meet or exceed professional standards Provide teachers and mental health professionals with tools not found in other assessments Lead to interventions Useful in documenting outcomes These are the goals of all Devereux scales. Might mention that we are a special ed/mental health treatment provider, not an academic setting. We want real-world solutions. (be careful not to offend any academics – if you have them identify themselves beforehand, you would know if there are academics there). With outcomes you might mention performance based contracting. © 2010 Devereux Center for Resilient Children

The DESSA Comprehensive System
Universal screening with an 8-item, strength-based behavior rating scale, the DESSA-mini Provide a snapshot of the overall social-emotional competence of your students Support universal strategies Follow-up assessment with at-risk students with the DESSA Support targeted strategies Periodic progress monitoring with the DESSA-mini and Ongoing Progress Monitoring Form. © 2010 Devereux Center for Resilient Children

Universal Screening with the DESSA-mini

Prevention and the School Psychologist
There is a movement in School Psychology toward prevention of academic failure In order to prevent academic problems universal screening is indicated In the past, tests of achievement were given Today, some have proposed using short one minute reading fluency tests, for example, as a way to identify children who are having trouble reading

Similarly, is a movement in School Psychology toward prevention of mental health problems … Universal screening is one way to prevent academic problems Preventing mental health problems requires screening for those factors that help protect children from developing mental health issues This is an important role for school psychologists

NASP’s mission is “to … enhance the mental health and educational competence of all children.” Ensuring the mental health of students is a key role of school psychology How can school psychologists assess potential mental health problems?

School psychologists may… evaluate mental health status of children when they are referred for an evaluation By the time children with behavioral and emotional problems become noticed by the school psychologist the emotional issues the drive the problem behaviors may be firmly entranced Mental health problems need to detected as early as possible

Mental Health Screening
According to the US Department of Health and Human Services (1999) about 20% of school aged children have behavioral, social-emotional, and mental health disorders Universal screening to identify children at risk of developing mental health problems is needed

School personnel can conduct universal screening to help identify these children at risk of mental health problems Screening results should help assess the level of resilience for each teacher’s class Screening tests results could be used to determine if interventions that teach protective factors should be applied Universal screening should be brief to complete and have evidence of reliability and predictive validity

One way to screen for potential mental health problems is to evaluate children’s protective factors Protective factors are those variables that lead to resilient outcomes (e.g., good mental health) Protective factors lead to more positive behavioral and psychological outcomes as well as improved academic performance Protective factors help children be resilient

Protective factors can be measured with the DESSA mini This is an 8-item rating inventory for school-aged children It is a very brief, but effective, tool Several 8-item versions are in development The scale can be completed in less than one minute The scale is effective

DESSA – mini (Naglieri, LeBuffe & Shapiro, 2010)
A brief measure of social-emotional competencies comprised of four separate forms that can be used for universal screening and ongoing progress monitoring

The DESSA-mini The DESSA-mini allows for: Universal screening Repeated assessment Determination of need for instruction Four equivalent 8-item forms Completed in 1-2 minutes by teachers Yields one score – Social-Emotional Total Score © 2010 Devereux Center for Resilient Children

Four Forms of DESSA-mini

Four Forms of DESSA-mini
36 8 Need 2 1 12

DESSA-mini Scores One Score – Social Emotional Total (SET) T Scores Mean of 50, SD of 10 Percentiles Descriptive Terms for Score Ranges > 60 = Strength (≈ 16%) 41-59 = Typical (≈ 68%) < 40 = Need for Instruction (≈16%) Percentiles easy for parents to understand, but have mathematical limitations for analysis. Therefore, also have T-scores Discuss descriptive terms , especially “concern” – not pathology, problem © 2010 Devereux Center for Resilient Children

DESSA mini DESSA mini normative group
Standardization data for Teacher Raters (N = 1,249) Region: NE = 24.6%; South = 39.1%; Midwest = 22.3%; West = 14.0% 50.8% Males Grades Kindergarten through 8

DESSA-mini Psychometrics

DESSA-mini Research Manuscript to appear in Psychology in the Schools (2011) We studied the percentage of agreements between mini and full DESSA

DESSA-mini Research Overall category agreement is excellent Table 5.
DESSA and Each DESSA-mini Means, SDs, and ns for Grades K - 8. Dessa-Mini N n of Agreements % Agreements Mini 1 1,222 1,171 96.5% Mini 2 1,173 96.2% Mini 3 1,160 94.8% Mini 4 1,167 95.9% Overall category agreement is excellent

False negatives have scores <40 on mini but >40 on Total
True Positives have scores >40 on the mini and >40 on the Total True Negatives have scores <40 on Total and mini False Positives have scores >40 on mini but <40 on Total

These children have a score of 32 Use ongoing progress monitoring How do we describe changes over time?

Monitoring Progress with the DESSA-mini OPM

Ongoing Progress Monitoring Form
Remember that T scores have mean of 50 and SD of 10. So a difference of 5 points is ½ SD (5/10) © 2010 Devereux Center for Resilient Children

Poor Response to Instruction

Response to Instruction

Conclusions Universal screening of factors that predict resilience can be efficiently conducted School psychologists should take on this important mental health screening role Once evaluated, teachers and school psychologists should work together to deliver a curriculum that improves protective factors and maximize resilience

Assessment & Planning for Intervention with the DESSA

The Devereux Student Strengths Assessment
Based on resilience theory & SEL principles 72 items 8 scales Completed by parents, teachers, and/or after-school / community program staff Takes 15 minutes to complete On-line administration, scoring and reporting available © 2010 Devereux Center for Resilient Children

Purposes of the DESSA Identify social-emotional strengths and needs of elementary and middle school children. Tier I - Produce classroom profiles that guide universal prevention strategies. Tier II - Identify at-risk children so that targeted interventions can occur. Tier III - For special education students, identify important strengths that can be incorporated into IEPs. Mention the Public Health Service Model – three levels of prevention services: Universal – about 90% of kids Targeted – 5-7% Indicated 3-5% The DESSA supports all three © 2010 Devereux Center for Resilient Children

Additional Purposes of the DESSA
Foster collaboration between parents and teachers Document outcomes for individual students, classrooms, and communities Serve as a research measure Might mention our experiences with collaboration and the DECA © 2010 Devereux Center for Resilient Children

Development of the DESSA
Review of the literature – 765 items National Pilot Study – 156 items Eliminate those with item-total r of < .60 Eliminate those with a d-ratio of < .50 Eliminate those that were “N/A” Standardization Form – 81 items Eliminate age trends DESSA Final Form – 72 items © 2010 Devereux Center for Resilient Children

Standardization 2,500 children, grades K-8 All 50 states included in sample Representative of US Population Rater: Teacher/Staff 50%; Parent/Relative 50% 60% Primary Grades (K-3) 40% Middle Child (4-6) Gender: Male 50% ; Female 50% Free/Reduced Lunch (Poverty): 22% Region: Northeast 22%; South 36%; Midwest 22%; West 20% © 2010 Devereux Center for Resilient Children

DESSA Scale Structure Self Awareness Self Management Social Awareness
Social Emotional Composite Self Awareness Self Management Social Awareness Relationship Skills Goal Directed Behavior Personal Responsibility Decision Making Optimistic Thinking Social Emotional Composite Eight Scales Self Awareness Self-Management Social-Awareness Relationship Skills Goal-Directed Behavior Personal Responsibility Decision Making Optimistic Thinking © 2010 Devereux Center for Resilient Children

Administration of the DESSA

DESSA Raters Raters provide the ratings Teachers, After School Staff Parents Read English at about the 6th grade level Sufficient opportunity to observe the child What about raters who have literacy issues or are ELLs? © 2010 Devereux Center for Resilient Children

DESSA Results T Scores Mean of 50, SD of 10 Percentiles Descriptive Terms for Score Ranges > 60 = Strength 41-59 = Typical < 40 = Need for Instruction Individual Profile Classroom Profile Percentiles easy for parents to understand, but have mathematical limitations for analysis. Therefore, also have T-scores Discuss descriptive terms , especially “concern” – not pathology, problem © 2010 Devereux Center for Resilient Children

DESSA Rating Form

Individual Student Profile

Reliability - Internal Consistency

Interpreting the DESSA

Interpreting the DESSA
Three Step Process Social-Emotional Composite 8 Individual Scale Scores Individual Item Analysis Funnel or hourglass model. Begin with broadest, most reliable index (TPF) – gives an overall barometer or indication. 8 individual scales give info on the nature and severity of the problem(s) Individual item analysis and strength/concern clusters – transition to treatment/education planning © 2010 Devereux Center for Resilient Children

Case Study Charles 11 year old / 5th Grade Special education student (EBD) Placement in RTF since age 7 Born into foster care, adopted at age 2.5 History of serious behavioral concerns (see handout – Charles’ Scoring Assistant Report) © 2010 Devereux Center for Resilient Children 80

DESSA Item Analysis Significant Item Score - Strength Never Rarely Occasionally Frequently Very #18 – ask to take on additional work or responsibilities ? Nonsignificant Item Score - Typical Never Rarely Occasionally Frequently Very #37 – follow the example of a positive role model? Based on standardization data. Lowest 16% of the Standardization sample (less than one SD below the mean). Significant Item Score - Need Never Rarely Occasionally Frequently Very #69 – use available resources (people or objects) to solve a problem? © 2010 Devereux Center for Resilient Children 85

DCRC Approach to Intervention
Strengths, Goals and Strategies Step 1 – identify goal(s) of most concern to stakeholders Step 2 – identify relevant strengths that can be leveraged Step 3 – identify strategy Critical – communicate by beginning with strengths!

Utilizing the DESSA to Guide Intervention
“Assessment only has value if it leads to an improved outcome for the child” (NAEYC) Many good existing curricula SAMHSA CASEL Committee for Children Short-Term we’ll provide crosswalks Long-Term we’ll provide DESSA-guided strategies © 2010 Devereux Center for Resilient Children

Advanced Interpretation
Rater Comparisons Two teachers, two parents, parent-teacher Based on the standard error of the difference Scale level agreement or disagreement Strong basis for collaboration Supports planning across environments Might introduce the notion of measurement error here – gas mileage. The SE of the difference takes into account the measurement error on both scores. Four scenarios per scale – We both see it, I see a concern/strength and you don’t, you see it and I don’t, neither of us see it. What are the implication of each? How does this support planning? © 2010 Devereux Center for Resilient Children

Rater Differences

Rater Comparisons (See Handout #2: DESSA Record Form)
58 62 4 9 39 32 7 11 51 56 5 8 60 51 9 10 48 47 1 9 58 55 3 8 40 53 13 11 53 55 2 9 51 53 2 4 Mrs Smith 10/10/07 Mrs. E. 10/18/07 (See Handout #2: DESSA Record Form) © 2010 Devereux Center for Resilient Children

Advanced Interpretation
Pretest-Posttest Comparisons Multiple comparisons overtime Based on the standard error of prediction Demonstrates statistically reliable change Evaluation of the effectiveness of strategies © 2010 Devereux Center for Resilient Children

Pretest-Posttest Comparisons
X 58 50-65 65 39 31-49 48 X 51 50 X 44-58 X 60 51-67 69 48 40-56 38 X 51-64 62 X 58 40 32-50 57 X 53 45-60 59 X 48-54 X 51 57 Mary Smith 10/10/07 02/28/08 Mary Smith (See Handout #2: DESSA Record Form) © 2010 Devereux Center for Resilient Children

The DESSA Comprehensive System Summary

Ongoing Progress Monitoring Form

Student receives full DESSA Time 1 DESSA-min ≤40? Yes NO DESSA ≤ 40? NO Student Receives Universal (Tier 1) Instruction Yes Student reassessed With DESSA mini or if indicated Results reviewed with School Counselor. Plan for Targeted (Tier 2) Supports Developed Continued development of social emotional support. Maybe begin with universal instruction in September But jack points out that the DESSA-mini would be a stimulus to looking at SEL Implement Targeted Strategies, monitor with RTI approach (See page 2) © 2010 Devereux Center for Resilient Children

Ongoing Monitoring for Students at Tier 2 (Targeted Instruction) Increase frequency & intensity of Tier 2 supports. Implement individualized strategies Consult with SAP Time 2 YES* DESSA-mini ≤ 40? NO Implement Universal Supports Time 3 Student receives full DESSA to evaluate outcome at the individual scale level Two positives in a row to get back to typical. Consider other variables – after school program. Is it an example or recommended model. Not best practice. Envision it being used. Recommendation for implementation. Model does not really promote strength (T-score above 60). As a screener is based on a deficit model. 40 is not the ultimate goal. DESSA-mini for all students, excepts Tier 2 © 2010 Devereux Center for Resilient Children

Utilization of the DESSA Comprehensive System with a Tiered Approach
Tier I - Primary Prevention Universal Use of the DESSA Classroom Profile Identifies common areas of strength and concern Leads to classroom-wide strategies Tier II - Secondary Prevention Identification of children at-risk due to low PFs Individual Child Profile Targeted Strategies © 2010 Devereux Center for Resilient Children

Utilization of the DESSA (cont.)
Tier III - Tertiary Prevention Identification of strengths and needs in identified children Incorporated into IEPs Strong basis for collaboration with parents Program Evaluation Evaluate progress at the scale, child, and classroom level Basis for quality improvement and continuing education efforts © 2010 Devereux Center for Resilient Children

Ways the DESSA adds Value
Assess competencies related to academic achievement Support implementation of SEL and similar programs More specific implementation Outcome determination Meet requirements to do meaningful strength-based assessment

Can Change Teacher Perceptions
I have one student who has a lot of trouble being bossy …and I am frustrated with her a lot. But the DESSA showed me all the areas she is strong in, and gave me some ideas for channeling some of her difficult behaviors to utilize her strengths. Being that my students are in a self-contained special ed. Classroom,... I was surprised that several of my students are "typical" in more areas than I would have thought. This allowed me to write strength statements and share good news with the parents. One of my students is very compassionate. He has very strong S.A. skills. Cool. Direct quotes from teachers in Anchorage, Alaska SD These are direct quotes from teachers in Anchorage AK who were involved in a year-long research and demonstration project that involved using a standardized measure of social-emotional competencies (DESSA) to guide and evaluate SEL.

Can Change the Student’s Perception
I had a couple kids who don't say nice things about themselves. However, they are wonderful in all other areas. I made a point of letting both of these children know how great I think they are. They both were very happy with that. Direct quote from teacher in Anchorage, Alaska SD

Can Contribute to Teacher Satisfaction
I really liked this experience. It was not too laborious and had very worthwhile results. I would tell everyone to make the time and do this. Using an assessment tells teachers it's time to start paying close attention to the social skills. It helps teachers’ awareness but it also gives teachers a way to address these behaviors that is proactive! Many times we're only reacting to students (often negatively or with much frustration) but we aren't doing enough to help them. I liked how it made me see my students, and it really took my teaching with SEL in a new direction Direct quotes from teachers in Anchorage, Alaska SD

Essential to Prevention
Chemung County NY 406 preschool children Pre-Post design with no intervention Results N = 406 % PreBC Concern %PostBC Concern Strength on Pretest Total Protective Factors (n = 76) 0% Typical on Pretest Total Protective Factors (n = 275) 14% 13% Concern on Pretest Total Protective Factors (n = 55) 51% 70%

Family of Protective Factor Assessments
DECA for Infants and Toddlers (0 to 3 years old) DECA for Preschoolers (2 through 5 years old) DECA-C (Clinical) for Preschoolers (2 through 5 years old) DESSA – Mini ! Two key findings: For risk, measured about 35 Major Life Events – highly traumatic, but limited duration e.g. death of a parent, witnessing a murder. And 35 Daily Hassles – low intensity, but chronic (frequent changes in caregivers, living quarters; sharing a bed with more than one sibling, inability to put food on the table on a regular basis or pay bills on time). Some researchers think these are more deleterious than Major life events. Third bar (low risk/low PF) do as poorly as second bar (high risk/high pf) 2) Overall, Pfs are as good a predictor of BCs as risk factors This is the study that made us believers, and no faith like that of a convert. DESSA (Kindergarten-8th Grade) (Kindergarten-8th Grade) © 2010 Devereux Center for Resilient Children

About After-School Programs (ASP)
In 2004, 6.5 million children (K-12) participated in ASP nationwide. ASP aims to provide safe and supervised environments for children during after-school hours (3pm-6pm). National accreditation standards for ASPs include SEL. The Obama Administration intends to double the funding for ASPs. Including measures to maximize performance and effectiveness across grantees nationwide. Will support an additional 1 million children As of 2004, it is estimated that 6.5 million children are being served within after-school programs nationwide. Each program has it’s own unique mission and vision but the overarching theme for every afterschool program is to provide a safe and supervised environment for kids to spend time between the end of school and the time that many of America’s working parents get home. Going beyond that, another overarching goal for afterschool providers is to facilitate healthy prosocial relationships between children and adults as well as children and their peers. Like most organizations serving young people, they must be accredited in some way. The National Afterschool Association provides accreditation for afterschool programs nationwide. There standards do include social-emotional components. These standards can be found in your hand out. It has been the case that for many years, After School programming has not received a great deal of funding and therefore it’s resources and capabilities have been limited. However, the Obama Administration intends to double the current level of funding being appropriated to After school. This increase in funding will most certainly raise the need for an assessment of how well after school programs are effecting children. Funders will want to know how effective the programs are in order to justify high levels of spending for these programs. The DESSA can certainly be a great tool that after school providers could use to asses the social-emotional learning that takes place in their programs. © 2010 Devereux Center for Resilient Children 108

Definition of Strength-based Assessment
The measurement of emotional and behavioral skills and characteristics that create a sense of accomplishment contribute to satisfying relationships with others improve one’s ability to cope with stress promote social and academic development Epstein & Sharma (1998)

What Strength-based Assessment is Not
Simply asking “What are your strengths?” Writing a 5-page report with one sentence describing interests or hobbies Informal, impressionistic approaches, such as examination of refrigerator doors Ignoring or minimizing problems or pathology

Advantages of Strength-Based Approaches
Leads to Prevention Expands Definition of Health Teaches Replacement Behaviors Attractive to Stakeholders Teachers more confident Parents less defensive Better basis for collaboration Lends to Prevention – talk about Devereux’s history © 2010 Devereux Center for Resilient Children

The DESSA Comprehensive System: A New Tool to Assess Social-Emotional Strengths & Resilience Discuss Devereux, ICTR. Discuss our history doing pathology-based.

Similar presentations

Presentation on theme: "The DESSA Comprehensive System: A New Tool to Assess Social-Emotional Strengths & Resilience Discuss Devereux, ICTR. Discuss our history doing pathology-based."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

The DESSA Comprehensive System: A New Tool to Assess Social-Emotional Strengths & Resilience Discuss Devereux, ICTR. Discuss our history doing pathology-based.

Similar presentations

Presentation on theme: "The DESSA Comprehensive System: A New Tool to Assess Social-Emotional Strengths & Resilience Discuss Devereux, ICTR. Discuss our history doing pathology-based."— Presentation transcript:

Similar presentations

About project

Feedback