Summative Assessment – ACCESS for ELLs 2.0 Scores and Reports MODULE G Summative Assessment – ACCESS for ELLs 2.0 Scores and Reports As you recall in Module F, a summative assessment is an outcome assessment that is state approved, norm referenced and/or criterion referenced. For our English Language Learners in a Title III funded district, the state outcome assessment is the ACCESS for ELLs 2.0. For public schools in the state of NJ, progress in achieving English language proficiency is defined by NJDOE and measured by a state determined language proficiency assessment. Please refer to the NJDOE, Bureau of Bilingual Education website for additional guidance.

The raw score is then converted into a scale score. ACCESS for ELLs 2.0 Scores Raw Scale ELP Levels ACCESS for ELLs is a reliable and valid test of English language proficiency. Nevertheless it, like all tests, is subject to a statistical concept known as the standard error of measurement. It quantifies the variation of scores achieved if a student was able to take the same test over and over again without any change in his or her ability. Raw Scores are the total number of items correct. The raw score is then converted into a scale score. The particular scale score a student receives ranges from 100-600. These numbers are determined by complex statistical formulas to ensure that the scores are valid and reliable for decision making and to make sure scores are comparable from year to year. Then these scores are transformed into proficiency levels. Proficiency levels help make scores more useful for differentiation of instruction and assessment. These levels range from 1.0 to 6.0 and are aligned to the WIDA rubrics and WIDA ELD Standards. 2

ACCESS for ELLs 2.0 Scores Composite Scores Listening Reading Writing Domain Scores Listening Reading Writing Speaking Composite Scores Oral Language Literacy Comprehension Overall ACCESS for ELLs 2.0 scores provide data for each of the language domains (listening, reading, writing and speaking). They also include composite scores which will be discussed further in the slide following. We use the overall composite score of 4.5 as one of the measures to determine if a student is ready to exit the Bilingual/ESL program. 3

Composite Scores Oral Language Score Literacy Score Listening (50%) Speaking (50%) = + Literacy Score Reading (50%) Writing (50%) = + Comprehension Score Listening (30%) Reading (70%) = + Listening (15%) Speaking (15%) The four composite scores are Oral Language, Literacy, Comprehension, and Overall score. The slide above presents the percent contribution, or the weighting, of language domains for each composite score. Students receive four different composite scores derived from a combination of the weighted language domain scores. Composite scores are not calculated using a combination or average of domain proficiency level scores. Composite scores are compensatory. Compensatory means that a high score in one language domain could inflate the composite score, compensating for a low score in another language domain; conversely, a low score in a language domain could bring down the composite. Composite scores should be used with caution after careful consideration of their compensatory nature. Attention must be given to the individual language domain scores that comprise the composite score as well as their weights. Reading and Writing scores are weighted higher. Once composite scale scores are calculated, they are then interpreted into corresponding composite proficiency levels. Only students that complete all sections of ACCESS for ELLs 2.0 will receive the four composite scores. The same Overall Scale Score for two students can reflect two very different profiles. For example, one student may be very strong in Listening and Reading, but weaker in Speaking and Writing, while another student with the same Overall Scale Score is strong in Reading and Writing, but weaker in Listening and Speaking. A student’s individual performance in each language domain provides a more comprehensive and realistic profile than that a single overall score.   Overall Score = + Reading (35%) Writing (35%) Intrepretive Guide for Score Reports, Spring 2015 4

Scale Scores WIDA ACCESS for ELLs 2.0 scale scores are psychometrically derived measures of student proficiency. Range from 100 to 600 (above 500 is rare). There is a separate scale for each language domain: Listening; Speaking; Reading; and Writing. Scale scores allow student performances (i.e., raw scores) across grades and tiers to be compared on a vertical scale. The vertical scale allows scale scores across grade levels to be compared to one another within any single domain. Scale scores are useful for monitoring a student’s progress from year to year. There is a separate scale for each language domain: Listening, Speaking, Reading, and Writing. Because each domain has its own scale, a scale score of 300 in Listening does not mean the same as a scale score of 300 in Speaking. For each domain, scores are reported on a single vertical scale from Kindergarten to Grade 12. The lowest possible scale score is 100. The upper limit is 600, although scores above 500 are rare. WIDA/CAL (c) 2007 WIDA/CAL Grade Level Cut Score Review Study 5

Proficiency Level Scores Proficiency Level Scores are socially-derived interpretations of the ACCESS for ELLs Scale Scores in terms of the six proficiency levels defined in the WIDA Standards. Comprised of a whole number and a decimal, e.g. 2.5 The whole number indicates the proficiency level into which the student’s scale score places him or her (e.g. 2 = Emerging) The decimal indicates how far, in tenths, the student’s scale score places him or her between the lower and the higher cut score of the proficiency level (e.g. 2.5 = 5/10 or ½ of the way between the cut score for level 2 and level 3) Proficiency level scores are interpretive scores. That is, they provide stakeholders with an interpretation of the scale scores. They help stakeholders to understand what the numeric scores mean in terms of the language skills of the student. They describe student performance in terms of the six WIDA language proficiency levels (1-Entering, 2-Emerging, 3-Developing, 4-Expanding, 5-Bridging, 6-Reaching). 6

ACCESS for ELLs 2.0 Interpretive Guide The ACCESS for ELLs 2.0 Interpretive Guide for Score Reports contains detailed information on the use of scores from this assessment. Download the document from www.wida.us. Make sure you are looking at the most recent guide. The Interpretive Guide for Score Reports offers detailed information on the meaning and the use of scores received for English language learners on ACCESS for ELLs 2.0. It gives a detailed description of each score report and suggestions for stakeholders on data applications. It is recommended that presenters and participants download the guide from www.wida.us. As with all assessments, ACCESS for ELLs 2.0 scores should be considered one of multiple criteria in educational decision making. 7 7

ACCESS for ELLs 2.0 Score Reports Parent/ Guardian Teacher Student Roster Individual Student Report School Frequency District Frequency Each of these reports will be examined more closely throughout the remainder of this module. Individual report components offer a starting point for informing the areas of curriculum, instruction and assessment of ELLs. 8

Demographic information about the student Oral Language Score Student’s ELP Level by Domain Literacy Score Description of the ELP Levels Comprehension Score The student’s parent or guardian gets this report. It is provided in English. Translations of the report are available in other languages (visit www.wida.us/translations) . A letter to accompany the report in parents’ primary language is suggested, a sample of which is available at www.wida.us. Overall Score

What does the Parent/Guardian Report tell us? The Parent/Guardian Report contains individual student data Score Report Audience or Stakeholder Types of Information Parent/Guardian Students Parents/ Guardians Teachers School Teams Individual student’s Overall Score and levels of English language proficiency for language domains (Listening, Speaking, Reading, and Writing and Comprehension) The parent report contains information about the student’s English language proficiency in relation to the WIDA ELD Standards – NOT information on the student’s academic achievement. The parent indicates the extent to which a student has acquired listening, speaking, reading and writing – reflective of a test given annually. 10

Guiding Questions for Parent/Guardian Score Report What information would you want to share with the family? What information are you required to share with the family? When, where and how will you share this information? How will you ensure that the family understands what you are sharing? The following Guiding Questions are suggested as you share the parent/guardian score report. If there are multiple participants completing this module at the same time, it is recommended that you jigsaw each report evenly among your participants. This will allow each group of participants to actively share the purpose of each report with their colleagues at the conclusion of this portion of the module. 11 11

Parent Handouts These one-page documents provide answers to several basic questions about ACCESS for ELLs 2.0, Kindergarten ACCESS for ELLs, and Alternate ACCESS for ELLs, and are available in multiple languages. The Parent Guides for ACCESS for ELLs 2.0 Score Reports may be found here: https://www.wida.us/downloadLibrary.aspx Handout Folder – What is ACCESS 2.0?

Teacher Report Demographic Information About the Student Student’s Scale Score by Domain Student’s ELP Level by Domain Student’s Scale Composite Scores Student’s Composite Proficiency Level Scores Student’s Comprehension by Standard Teachers and other stakeholders, such as administrators, have access to this report. Suggestions for the differentiation across levels of language proficiency can be found in the strands of the model performance indicators of the WIDA ELD Standards (www.wida.us/standards/eld/aspx). In addition, you may use the rubrics for each of the language domains to provide support to students based on their ACCESS for ELLs scores to transition them to higher levels of language proficiency. Student’s Speaking Performance by Standard Student’s Writing Performance by Standard Description of the ELP Levels 13

Teacher Report (top) The teacher report includes confidence bands for both domain and composite and scale scores. Confidence bands are a graphic depiction of the standard error of measurement of the scale score. Confidence bands are important as they remind test users that a single test score represents a range of possible outcomes. The confidence bands assure that there is a 95% probability that the student’s average score if he/she were to take the test over and over again is within the confidence band reported on the score report. For example, Sample Student above, would score between 336 to 408 in Listening if he/she were to take the test over and over again within a given time frame. 14

Teacher Report (bottom) Raw Scores by Standard In this section of the report, raw scores are provided for the different parts of the test. Raw scores cannot be compared across grade level clusters or across tiers within a grade level cluster. Writing tasks are given a proficiency level score, by WIDA ELD standard and by features of academic language of the WIDA Writing Rubric (Linguistic Control, Vocabulary Usage, and Language Control). The writing scoring rubric was based directly on the six proficiency levels of the WIDA Standards. Scores on the writing tasks do reflect a common meaning across tiers and grade levels (though developmental differences across grade level clusters are taken into account). Speaking Tasks receive a raw score whereas Comprehension Tasks receive number of items correct by WIDA ELD standard. 15

This is a Tier C writing sample from the 3-5 cluster Teacher Report Writing Tasks Writing raw scores are presented by standard next to the maximum number of points for the given standard(s) and scoring category (features of academic vocabulary) reported Please note that some test forms contain more than one Writing task addressing the same standard while another standard may not be addressed. This is particularly true in the lower grades and tiers. For example, Tier A for grade cluster 3-5 does not contain a task addressing the language of Social Studies. This is a Tier C writing sample from the 3-5 cluster 16

What does the Teacher Report tell us? The Teacher Report contains individual student data. Score Report Audience or Stakeholder Types of Information Teacher Teachers Administrators Individual student’s scale scores and proficiency levels for each language domain, Oral Language, Literacy, Comprehension, and Overall Score; Raw scores for Comprehension Tasks, Speaking, and Writing Tasks by English language proficiency standard This report provides information regarding the levels of social and academic English language proficiency the student has attained. Social language is used to communicate for everyday purposes. Academic language is used to communicate the content of language arts, mathematics, science, and social studies. This report can be used to monitor progress from year to year and to help determine instructional strategies by content areas and standards. Please refer to the ACCESS for ELLS Interpretive Guide for more information on the meaning and use of these scores. You may also refer to the Complete Interpretive Guide for Score Reports at www.wida.us for more detailed information. 17

Guiding Questions for Teacher Score Report How are the domain scores similar to the composite score? How are the domain scores different to the composite score? What language may the student be able to comprehend or produce according to these scores (domain and composite)? Why might it be helpful to look at scores by domain rather than just the overall composite scores? The following Guiding Questions are suggested as you share the Teacher score report. If there are multiple participants completing this module at the same time, it is recommended that you jigsaw each report evenly among your participants. This will allow each group of participants to actively share the purpose of each report with their colleagues at the conclusion of this portion of the module. 18 18

Tier Cluster Scale Score and ELP Level by Domain Scale Score and ELP Level by Composite: Oral Language, Literacy, Comprehension and Overall District administrators may examine scores from each language domain within a Tier and grade level cluster to detect any patterns, if applicable.

What does the Student Roster Report tell us? Audience or Stakeholder The Student Roster Report lists the scale scores and proficiency levels for a group (or class) of students Score Report Audience or Stakeholder Types of Information Student Roster Teachers Program Coordinators/ Directors Administrators Scale scores and proficiency levels for each language domain, Oral Language, Literacy, Comprehension, and Overall Score by school, grade, student, tier, and grade level. The student roster report lists individual scale scores along with their corresponding ELP levels. Its audience includes Teachers, Program Coordinators/Directors, and Administrators This language proficiency test is standards-referenced, therefore, any comparison should be made between students in relation to the criteria or standards. Its purpose can be for the development of school and district improvement plans for ELLs and/or the development of school staffing plans and scheduling The report provides a starting point for grouping students for support services according to their Overall Score. 20

Guiding Questions for Student Roster Report How could teachers at a grade-level team meeting use this report? What would teachers need to keep in mind when they look at this data? When would teachers want to focus on the overall score? When or why would teachers want to focus on the domain scores? The following Guiding Questions are suggested as you share the Student Roster Report. If there are multiple participants completing this module at the same time, it is recommended that you jigsaw each report evenly among your participants. This will allow each group of participants to actively share the purpose of each report with their colleagues at the conclusion of this portion of the module. Additional Guiding Questions: To what extent are there differences in student performance between the language domains? Are these differences attributed to second language development or delivery of instructional services? 21 21

Individual Student Report This report shows the eight scores that a student could receive on the ACCESS for ELLs 2.0. If a student took all four sections of the test, he/she will receive all eight scores. N/A indicates no score was reported. 22

Guiding Questions for Individual Student Report What language may the student be able to comprehend or produce according to these scores (domain/proficiency levels and overall score)? Why might it be helpful to look at scores by domain rather than just the overall score? The following Guiding Questions are suggested as you share the previously viewed Individual Student Report. If there are multiple participants completing this module at the same time, it is recommended that you jigsaw each report evenly among your participants. This will allow each group of participants to actively share the purpose of each report with their colleagues at the conclusion of this portion of the module. Please refer to the WIDA rubrics to support you as you answer these guiding questions. Handout Folder – WIDA RUBRICS 23 23

Highest & Lowest Scores District Frequency Report Number of Students Tested who scored at each ELP level by Domain and Composite % of Total Students Tested who scored at each ELP level by Domain and Composite Highest & Lowest Scores The District Frequency Report indicates: number of students and percent of total tested for language domains (including the range of scale scores, i.e., lowest to highest), Comprehension, Oral Language, and Literacy by proficiency levels for grade levels. This report may serve as a district’s estimate of the number of students who have met the state’s exit criteria. Total Tested 24

What does the District Frequency Report tell us? The District Frequency Report lists the numbers of students tested in each domain of ACCESS by grade level within a district Score Report Audience or Stakeholder Types of Information District Frequency Program Coordinators/ Directors Administrators Boards of Education Number of students and percent of total tested for each language domain, Oral Language, Literacy, Comprehension, and Overall Score by proficiency levels for grade levels within a district. Information will be useful in planning, designing, or restructuring program services. Information provided in this report may have to be further contextualized to be meaningful. A description of the students in terms of their language, cultural and experiential backgrounds would provide a fuller portrait of a district’s ELLs. This report provides a glimpse of the performance of all ELLs across language domains and combination of domains in a district at the time of testing. 25

Highest & Lowest Scores School Frequency Report % of Total Students Tested who scored at each ELP level by Domain and Composite Number of Students Tested who scored at each ELP level by Domain and Composite Highest & Lowest Scores The School Frequency Report indicates: number of students and percent of total tested for language domains (including range of scaled scores), Comprehension, Oral Language, and Literacy by proficiency levels for grade levels within a school. Results should not be generalized and need to be contextualized in order to provide meaningful information on curricular, instructional or assessment decisions. School Frequency Reports for two consecutive years provide cross-sectional data. In communicating results of this report, use both the numbers and their corresponding percents. If numbers are low, the percent may appear distorted if shown in isolation. Use the information contained in the report to gain a sense of the school-wide effort in educating English language learners. Total Tested 26

What does the School Frequency Report tell us? Audience or Stakeholder The School Frequency Report lists the numbers of students tested in each domain of ACCESS by grade level within a school Score Report Audience or Stakeholder Types of Information School Frequency Program Coordinators/ Directors Administrators Number of students and percent of total tested for each language domain, Oral Language, Literacy, Comprehension, and Overall Score by proficiency levels for grade levels within a school. This report shows the distribution of ELLs according to their language proficiency levels in a stated grade of a specified school. In schools with low ELL enrollment this distribution might be quite small. Results should not be generalized unless there are relatively large numbers of students. 27

Guiding Questions for District and School Frequency Reports How could administrators use these reports? How could these reports be used in making decisions about resources and staffing? How could these reports be used in evaluating the effectiveness of ELL instructional services, program model or curriculum? The following Guiding Questions are suggested as you share the District and School Frequency Reports. If there are multiple participants completing this module at the same time, it is recommended that you jigsaw each report evenly among your participants. This will allow each group of participants to actively share the purpose of each report with their colleagues at the conclusion of this portion of the module. 28 28

Here ends this Module Series

Contact Information Please email lep@doe.state.nj.us with any questions or comments you may have regarding this module.