MCAS-Alt WELCOME to the Fall 2014 ADMINISTRATORS OVERVIEW of and Related Topics for Students with Disabilities Presenter: Dan Wiener Administrator of Inclusive Assessment Student Assessment Services Massachusetts Department of Elementary and Secondary Education
Today’s Agenda Historical perspective Who should take the MCAS-Alt? 2014 MCAS-Alt Results Statewide participation, performance, and trends Portfolio scoring Why a “complete” portfolio is important Principals’ role in MCAS-Alt MCAS-Alt security Resources for teachers PARCC and students with disabilities Massachusetts Department of Elementary and Secondary Education
15 Years of MCAS-Alt! In 2001, 5,000 students participated in MCAS-Alt for the first time (as did the first training specialists). Portfolios were scored by Massachusetts teachers each summer through 2008. Portfolios are now scored out-of-state under the supervision of ESE and Measured Progress. The greatest number of portfolios was submitted in 2012 (9,386). The number has decreased since then. On average, 1.7% of all assessed students, or about 9% of SPED students, have taken the MCAS-Alt each year. Massachusetts Department of Elementary and Secondary Education
Purposes of Alternate Assessment Include difficult-to-assess students in assessment and accountability (it’s the law). Ensure that standards-based skills and content are taught at levels that are meaningful and challenging. Determine which skills and content students have learned. Provide information to schools and parents on the achievement of students with significant disabilities. Massachusetts Department of Elementary and Secondary Education
2015 Educator’s Manual for MCAS-Alt, Pp. 9-11 How Should Students with Disabilities Participate in MCAS? Decision needed annually in each content area. IEP Teams decide: Can student take standard MCAS test? Can student take standard MCAS test with accommodation(s)? Which accommodations are appropriate? Does student need an alternate assessment? Massachusetts Department of Elementary and Secondary Education
Who Should Take MCAS-Alt? A student with a disability who is…. Working on learning standards that have been substantially modified due to the severity of the disability, and is Receiving intensive, individualized instruction in order to acquire, generalize, and demonstrate knowledge and skills, and is Unable to demonstrate knowledge and skills on a paper and pencil test, even with accommodations, . . . should take the MCAS-Alt in that subject. Massachusetts Department of Elementary and Secondary Education
Can other students with disabilities be considered for MCAS-Alt? Yes. A student with a disability may also require an alternate assessment if he or she. . . Is working on learning standards at or near grade-level expectations, and Is sometimes able to take a paper-and-pencil test, either with or without accommodations, but Cannot fully demonstrate knowledge and skills on an MCAS test because of a unique and significant (primarily non-cognitive) disability, and these challenges cannot be overcome through use of accommodations. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education
2014 MCAS-Alt Statewide Participation 9,103 students in grades 3-12 took MCAS-Alt in at least one subject in 2014. ELA = 8,391 Math = 8,580 Sci & Tech/Eng = 3,308 Total has decreased by 127 since 2013 (9,230) In all, 48,060 strands were scored. Massachusetts Department of Elementary and Secondary Education
2012-2014 MCAS-Alt: Statewide Results (All Content Areas - All Grades) 80.12% 79.06% 2012 2013 2014 70.09% 17.09% 11.41% 8.53% 10.81% 10.41% 8.41% .92% 1.88% .15% .91% .09% .10% Incomplete Awareness Emerging Progressing Needs Improvement+ Massachusetts Department of Elementary and Secondary Education
MCAS-Alt Performance Levels Standard MCAS MCAS-Alt The first four performance levels are reported within Warning/Failing. Advanced Proficient Needs Improvement Warning (Failing at Grade 10) Massachusetts Department of Elementary and Secondary Education
MCAS-Alt for Accountability (PPI) 2015 Educator’s Manual for MCAS-Alt, Pp. 58-59 MCAS-Alt for Accountability (PPI) Schools and districts receive points toward their PPI, which is one part of the accountability system. But only one percent of students can receive 100 points, per the USDE. Composite Performance Index MCAS Performance Level MCAS Score Range MCAS-Alt Performance Level Points Per Student Proficient or Advanced 240 – 280 Progressing 75 or 100 * Needs Improvement (High) 230 – 238 Emerging 75 Needs Improvement (Low) 220 – 228 Awareness 50 Warning/Failing (High) 210 – 218 Portfolio Incomplete 25 Warning / Failing (Low) 200 – 208 Portfolio not submitted * Depending on “nature of disability” and “level of need” of each student Massachusetts Department of Elementary and Secondary Education 11 11
Earning Points for PPI… (Continued) Students will earn 100 CPI points for their school and district if they: Score Progressing Have one of the following disabilities: Intellectual or developmental disabilities Autism Multiple disabilities Deaf and blind Have a “high” level of need IMPORTANT: There is no cap or quota on number taking MCAS-Alt. MCAS-Alt is not limited to students with these disabilities. Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education
Transition to New Standards for the MCAS-Alt MCAS-Alt is now fully aligned with the Common Core State Standards in ELA/Literacy and Mathematics. High School Mathematics standards were added for 20142015. An updated Science and Tech/Eng Resource Guide is now available, based on the 2006 standards (not “Next Generation Science Standards,” which are still under review). All educator manuals and curriculum materials needed to conduct the MCAS-Alt are available at www.doe.mass.edu/mcas/alt/resources.html. Massachusetts Department of Elementary and Secondary Education
What Does MCAS-Alt Assess What Does MCAS-Alt Assess? One Observable, Measurable Outcome in Each Portfolio Strand ELA Reading: text comprehension, using the student’s primary mode of response. ELA Writing: expressive communication in grades 4, 7, and 10, using student’s primary mode of expression. ELA Language: word knowledge and meaning, making word choices, acquiring vocabulary Mathematics: two Math domains in each grade 38 (three conceptual categories in grade 10). Science and Tech/Eng Assesses three STE disciplines in grades 5 and 8; and one STE discipline in greater depth in either grade 9 or 10. Massachusetts Department of Elementary and Secondary Education
Progression from PreKHigh School Domain/Conceptual Category Mathematics Domains (Grades 38) and Conceptual Categories (High School) Progression from PreKHigh School Domain/Conceptual Category PK K 1 2 3 4 5 6 7 8 HS Counting and Cardinality Operations and Algebraic Thinking Number and Operations in Base Ten Number and Operations - Fractions Ratios and Proportional Relationships The Number System Expressions and Equations Functions Geometry Measurement and Data Statistics and Probability Algebra Number and Quantity KEY = Assessed by MCAS-Alt = Assess one entry point in any three Conceptual Categories
Access to the General Curriculum: Geometry Standards Less Complex More Complex Entry Points Access Skills “Essence” of standard: Solve mathematical problems involving 3-D shapes Visually track geometric shapes Match same shapes with different orientations Sort two-dimensional shapes by attribute (e.g., number of sides) Calculate the surface area of a cube Grade 8 Standard 8.G.A.9 Know the formulas for volume of cones, cylinders, and spheres, and use them to solve real-world and mathematical problems. Standard Massachusetts Department of Elementary and Secondary Education
2014 Resource Guide for Mathematics: Sample page — Grade 3 Standards Standard Number Standards Cluster Heading
2014 Resource Guide for Mathematics: Entry Points and Access Skills Cluster Heading Access Skills Entry Points
Creating a Measurable Outcome from the 2014 Resource Guide Review the grade-level standard in the required strand/domain. If standard is too challenging for student, select an entry point in the same grade, or earlier grade, that is challenging but attainable for the student. Create a measurable outcome based on the entry point. Example from Expressions and Equations: The following entry point: Compare number quantities using the symbols <,>,= …can be converted to the measurable outcome: “(Student) will compare number quantities using the correct symbol (<,>,=) with 80% accuracy and 100% independence.“ Massachusetts Department of Elementary and Secondary Education
A complete Portfolio Strand includes at least the following… Data Chart documenting performance of the measurable outcome on at least 8 different dates, with brief descriptions of each activity + First piece of primary evidence* showing performance of the measurable outcome Second piece of primary evidence* showing performance of the measurable outcome PRIMARY EVIDENCE: Work sample, video, or photograph (or series of photos) clearly showing an end product. All Evidence must assess the measurable outcome, and be labeled with name, date, percent accuracy, and percent independence. Massachusetts Department of Elementary and Secondary Education
Data Chart
Student Student Student
Primary Evidence #1
Primary Evidence #2
Student Student Reflection
Criteria for Scoring MCAS-Alt Portfolios Completeness — Is the minimum required evidence submitted? Level of Complexity — Is evidence aligned to the standards? At what level of complexity? Demonstration of Skills & Concepts — What is overall percent of correct responses (accuracy)? Independence — What is overall percent of independent student responses? Self-Evaluation — Does student show choice- and decision-making, and awareness of learning? Generalized Performance — Does student use different instructional approaches, presentation, materials, and methods of response? Massachusetts Department of Elementary and Secondary Education
Common Reasons for Scoring Incomplete Activities assessed different skills in the same portfolio strand. (4,139 strands) Assess what you say you will assess. At least 2 work samples did not relate to the measurable outcome listed by the teacher. (926 strands) Activity descriptions lacked clarity/specificity (1,272 strands) Scorer needs to know what the student did and how they did it. Missing titles of text for ELAReading (1,162 strands) Massachusetts Department of Elementary and Secondary Education Massachusetts Department of Elementary and Secondary Education
One Reason for Incomplete: Activities assessed more than one skill Sample Measurable Outcome: (Student) will read literary texts and identify the main characters with 85% accuracy and 85% independence.” Sample brief descriptions of activities: (Student) read the “Tortoise and the Hare” and identified the main characters and setting of the story.” (Student ) read short story “Scooter and Spot” and sequenced the events in the story. Massachusetts Department of Elementary and Secondary Education
One Reason for Incomplete: Descriptions lacked clarity and specificity What was the student asked to do? Must describe how student addressed the outcome. Pre-test Solve word problems worksheet Home-work Digestive system class assignment Answer vocabulary questions Label parts of digestive system Describe digestive system Chap. 2 test 1/7/13 1/8/13 1/9/13 2/1/13 2/4/13 2/5/13 2/7/13 2/8/13 Pre-test Answered math questions Home-work Short- answer word problems Put pictures on a story board Worked with partner Unit 2 math activity Chapter 3 at his computer 30 These do not adequately describe the activity or context. If you reviewed this portfolio, would you know what the student did? Massachusetts Department of Elementary and Secondary Education
Comments from Scorers For each score of Incomplete, one or more related comments are printed on the Portfolio Feedback Form (PFF). PFFs are posted online in June, and returned inside each portfolio in September. Administrators can print out and share these with teachers once they are available in June. Comments are selected from a list by scorers; they are not personal notes to teachers. Massachusetts Department of Elementary and Secondary Education
MCAS-Alt Score Appeals A process by which the Department addresses a claim of perceived scoring inaccuracy in a school's MCAS-Alt portfolio; or a miscalculation of a student's performance level on the MCAS-Alt. Submitted in late June by a school, educational collaborative, or approved private special education school requesting that the Department review the portfolio and, if warranted, recalculate the student's MCAS-Alt portfolio score. A copy of the portfolio must be kept at the school in order to file an appeal. Massachusetts Department of Elementary and Secondary Education
MCAS-Alt and Technology Digital resources, including this PowerPoint, at www.mcas-alt.org/materials Online Forms and Graphs available at: www.doe.mass.edu/mcas/alt/resources.html MCAS-Alt bimonthly newsletter Emailed during the school year Massachusetts Department of Elementary and Secondary Education
MCAS-Alt Security Requirements Principals ensure that: Only IEP teams (or 504 plans) designate students for MCAS-Alt. Portfolios are compiled appropriately and submitted on time. Evidence is authentic (i.e., not duplicated, altered, or fabricated), and portrays student’s performance accurately. ESE may request a fact-finding investigation, if irregularities are reported or found. Educators ensure that: Information in portfolio is complete and accurate. Student work and other evidence are neither duplicated, altered, nor fabricated. Regardless of similarity of classroom instruction or activities, evidence for each student reflects student’s authentic abilities and performance. student portfolios are submitted on time and with all required forms and information. . Pp. 7—8, Educator’s Manual Massachusetts Department of Elementary and Secondary Education
Storage and Destruction of Portfolios Returned to Schools 2015 Educator’s Manual for MCAS-Alt, Pg. 57 Storage and Destruction of Portfolios Returned to Schools Portfolios returned to your school become part of the student’s temporary record. Temporary records must be destroyed within 7 years after student transfers or graduates/exits. However, temporary records may be destroyed whenever they become “dated, irrelevant, or misleading” (see suggested timetable) Parent must be notified and given opportunity to receive materials prior to their destruction. Massachusetts Department of Elementary and Secondary Education
Additional Resource: MCAS-Alt Training Specialists Teachers across the state who… Have experience compiling and submitting MCAS-Alt portfolios Volunteered to help teachers in their district Will lead Department-sponsored portfolio review sessions in January and March for your teachers Massachusetts Department of Elementary and Secondary Education
Supporting Teachers Who Conduct MCAS-Alt Principals can: Assess capacity to complete the required number of MCAS-Alt portfolios Which teachers need support? Who can give it? Appoint MCAS-Alt “lead educator” in school and/or district to find answers, give advice, be available to assist as needed. Register teachers to attend training and review sessions in Oct-Jan-March. Massachusetts Department of Elementary and Secondary Education
Supporting Teachers (continued…) Consider assisting teachers to compile portfolios, by providing: Flexible scheduling Planning time to work on portfolios Designating other staff to assist in collection of evidence and/or give classroom coverage Occasional sub coverage, if compiling >3 portfolios Encourage an early start; stay organized Review emerging portfolios Contact Department with questions Review MCAS-Alt newsletter and other available resources Retrieve results in mid-June and share with teachers. Massachusetts Department of Elementary and Secondary Education
What Principals Can Do Before Submitting Portfolios To ensure that complete and accurate portfolios are submitted, principals (or designee) can: Ask teachers if all dates on the data charts are accurate. No classroom work dated on holidays, etc.) Look at brief descriptions on the data chart Do they adequately describe the activities? Do they match the measurable outcome? If other teachers in the school conduct MCAS-Alt, ask them to review each other’s portfolios. Sign the MCAS-Alt PCPA (Principal only). Massachusetts Department of Elementary and Secondary Education
Student Assessment in 2014-2015 and Beyond
PARCC and MCAS in 2014-2015 Districts in MA will select either MCAS or PARCC to administer in spring 2015. Approved private schools and collaboratives will also have a choice to administer either MCAS or PARCC. Results on all tests will be reported. Districts taking PARCC will be “held harmless” for their accountability rating. Rating can stay the same or go up, but cannot go down. Board of ESE votes on adopting PARCC in Fall 2015. Massachusetts Department of Elementary and Secondary Education
The future of MCAS and MCAS-Alt, if PARCC is adopted… All Grade 10 students will continue to take all high school MCAS tests to meet CD requirement through class of 2019. All students in Grades 5, 8, and 9/10 will continue to take MCAS Science and Tech/Eng tests. All students designated for alternate assessments will continue to take MCAS-Alt. Massachusetts Department of Elementary and Secondary Education
20142015 MCAS and PARCC Testing Schedule MCAS ELA: March 23April 6, 2015 MCAS Mathematics and STE (g5 & 8): May 419 MCAS HS Science and Tech/Eng: June 25 PARCC Performance-based Assessments (PBA): Online ELA/Literacy and Mathematics: March 16April 10 Paper ELA/Literacy and Mathematics: March 23April 3 PARCC End-of-Year Assessments (EOY): Online ELA/Literacy and Mathematics: May 429 Paper ELA/Literacy and Mathematics: May 1122 PARCC test administration training available in January. Massachusetts Department of Elementary and Secondary Education
PARCC Accessibility and Accommodations Many MCAS accommodations will be built into computer-based PARCC tests and be available to all students as “accessibility features.” Other accommodations will be available only to students with disabilities and English learners. Most Massachusetts accommodations are available on PARCC, including nonstandard accommodations, but may be delivered differently on computer-based tests. Many test administration procedures are available to any student on PARCC, at the discretion of principal (e.g., small group, separate setting, frequent breaks) Massachusetts Department of Elementary and Secondary Education
Additional Information on PARCC Accessibility and Accommodations PARCC accommodations do not include: Untimed testing for all students Individualized ELA graphic organizers and Math reference sheets that are allowed for MCAS Overview of PARCC Accommodations and Accessibility for MA Educators is posted to www.doe.mass.edu/parcc/. PARCC Accommodations and Accessibility Manual available at www.parcconline.org/parcc- accessibility-features-and-accommodations-manual. Massachusetts Department of Elementary and Secondary Education
Dates to Remember Educator and Administrator Training Sessions: Sept. 29, 30, Oct. 1, 2, 7*, 8*, 15*,16*, 2014 “Portfolios in Progress” (half-day portfolio review sessions): January 12—15, 2015 Feb. 24, 25, March 10, 11 Order MCAS and MCAS-Alt materials: Jan. 5—16 Binders received in schools: last week in Feb. Portfolios due: Thursday, April 2, 2015 Preliminary results: posted mid-June MCAS-Alt Score Appeals deadline: June 26 * PM Administrator Overview Session
Contact Information MA Department of Elementary and Secondary Education – Student Assessment (781-338-3625) Dan Wiener – dwiener@doe.mass.edu Debra Hand – dhand@doe.mass.edu General Inquiries – mcas@doe.mass.edu Measured Progress MCAS Service Center – 800-737-5103 Kevin Froton – froton.kevin@measuredprogress.org Terri Rippett – rippett.therasa@measuredprogress.org Massachusetts Department of Elementary and Secondary Education
? Questions