MTLE Passing Score-Setting Board of Teaching October 15, 2010

Passing Score-Setting Agenda Regulatory overview Review of test development activities Standards & Rules input Score-setting discussion and action ▫Panel-based passing score recommendations ▫Additional considerations: all fields ▫Field-specific considerations

Minnesota Law 122A.09, Subdivision 4(b) and (e) – requirements for basic skills testing in reading, writing, mathematics; requirement for pedagogy and content knowledge tests; requirement for reading standards 122A.18, Subdivision 2(b), (c), (d) – requirement for basic skills tests; requirement for remediation; requirement for legislative reports; requirements /conditions for issuing one-year licenses to candidates who have not passed the basic skills tests; requirement for making the basic skills a requirement to earn a first continuing license ▫Note: The legislative history suggests that this does not mean that all teachers must take the basic skills, but rather that people renewing a license under Subdivision 2(c) would not be able to earn a continuing license until they had passed the basic skills. (Session Law 1994: Chapter 647, Article 6, Section 14) 122A.23, Subdivision 2 (b), (d), (e) – requires out-of-state applicants to pass Minnesota’s licensure examinations

Minnesota Law 122A.18, Subdivision 2a.Reading strategies. (a) All colleges and universities approved by the Board of Teaching to prepare persons for classroom teacher licensure must include in their teacher preparation programs research-based best practices in reading, consistent with section 122A.06, subdivision 4, that enable the licensure candidate to know how to teach reading in the candidate's content areas. These colleges and universities also must prepare candidates for initial licenses to teach prekindergarten or elementary students for the assessment of reading instruction portion of the examination of licensure-specific teaching skills under section 122A.09, subdivision 4, paragraph (e).122A.06, subdivision 4122A.09, subdivision 4

Minnesota Law 122A.09, Subdivision 4(e) The board must adopt rules requiring candidates for initial licenses to successfully complete an examination of general pedagogical knowledge and examinations of licensure- specific teaching skills. The rules shall be effective by September 1, The rules under this paragraph also must require candidates for initial licenses to teach prekindergarten or elementary students to successfully complete, as part of the examination of licensure-specific teaching skills, test items assessing the candidates' knowledge, skill, and ability in comprehensive, scientifically based reading instruction under section 122A.06, subdivision 4, and their knowledge and understanding of the foundations of reading development, the development of reading comprehension, and reading assessment and instruction, and their ability to integrate that knowledge and understanding.122A.06 NOTE: The amendment to subdivision 4, by Laws 2009, chapter 96, article 2, section 17, is effective May 17, 2009, and applies to teacher candidates beginning February 1, Laws 2009, chapter 96, article 2, section 17, the effective date.

Minnesota Rule : Examinations for Teacher Licenses Subpart 1A – requirement for basic skills testing in reading, writing, mathematics Subpart 1B – requirement for pedagogy and content knowledge tests; requires tests to be adopted by the BOT; requires content test for additional fields of licensure, but new fields do not require another pedagogy test Subpart 2 – requirements for selection and adoption of a testing system Subpart 3(1) – requires BOT to set passing scores; requires candidates to achieve passing scores Subpart 4 – requires passing scores to be published in the State Register and BOT to notify MN colleges and universities of passing scores

Minnesota Rule : Examinations for Teacher Licenses Subpart 5 – allows MN approved programs to attest to the passing status of applicants ▫Translation: The folks in Educator Licensing do not receive score reports to verify passing the tests for MN completers; the recommendation from an approved program includes this as a requirement. Subpart 6 – provides guidance on test administration, scoring, and reporting Subpart 7 – allows candidates to be charged a fee for testing

Minnesota Rule : Examinations for Teacher Licenses Subpart 8 – requires candidates to attempt the basic skills tests prior to beginning their teacher preparation program Subpart 10 – allows for retakes of tests Subpart 11 – requires out-of-state applicants to meet MN’s testing requirements Subpart 12 – requires BOT to periodically review the testing system and to publish changes in the State Register

MTLE Design: RIGOR History: Social Studies broad-field licensure test MTLE sub-test model inherently increases the rigor and accountability for more narrow bands of content within a licensure field For example: ▫Elementary content knowledgeElementary content knowledge ▫5-12 Mathematics content knowledge5-12 Mathematics content knowledge

Test Development Activities Summer 2009: Program planning Fall 2009 ▫Advisory Committees established  Bias Review Committee (BRC) – 35 participants  37 Content Advisory Committees (CAC) participants  Targeted a 70/30 split: licensed teachers/higher education faculty members ▫Test frameworks developed  Reading stakeholder meeting  Content Advisory Committee preliminary online framework review  Bias Review Conference for framework review

Test Development Activities Fall-Winter : Item banks developed February 2010: Test Materials Review Conference ▫Bias Review Committee (4 days)  24 participants  Item review  9,200+ items reviewed!! ▫Content Advisory Committees (8 days)  Framework review AND item review  260 participants; 22 institutions represented Spring 2010: Content Validation Surveys

Test Development Activities April 2010 ▫Content Validation online surveys  200 randomly selected teachers per licensure field  100 randomly selected faculty members per licensure field  Survey analysis ▫Field Testing  22 institutions participating ▫Statistical analysis of results to finalize item banks  Out of 5,550 items field tested, 22 items were deleted

June Passing Score Activities Composition of panels ▫Content-specific ▫Targeted 70/30 split of teachers and faculty members ▫Average of 10 members per panel ▫Many, but not all, had been part of earlier test development work Types of panels ▫Multiple choice item banks ▫Marker response selection for constructed response items

June Passing Score-Setting Activities Training provided to panel members ▫Recognition of performance: “would” vs. “should”  Reasonable expectation to begin practice  Reasonable expectation to maintain public trust 2 rounds of passing score judgments: “Imagine a hypothetical group of individuals who are just at the level of knowledge and skills required to perform the job of an educator receiving an initial license in Minnesota. What percent of this group would answer the item correctly?”

June Passing Score-Setting Activities The result: ▫An item-by-item review by Minnesota experts ▫Typically, panelist item judgments fell within a range of 3-5 rating scale points (on a scale of 1-10) ▫A median of all ratings computed to provide a measure of expected performance of acceptably qualified for each item ▫A formula-based computation applied to the item ratings to generate a recommended score

BOT Passing Score-Setting Information available by subtest * ▫# of scorable items ▫Standard Error of Measurement ▫Panel-based passing score recommendation ▫Score points at -2 SEM, -1 SEM, +1 SEM, +2 SEM ▫Relationship between recommended raw score performance and # of scorable items * Remember: The subtest model inherently increases rigor and accountability for content.

BOT Passing Score-Setting Additional information / considerations ▫Impact data where n>10 in the September administration ▫Test objectives aligned to BOT standards ▫New test structure, format for candidates ▫Expanded study guides available for K-6, Early Childhood, Basic Skills; limited preparation materials in other fields ▫Structure of multiple subtests increases content accountability

BOT Passing Score-Setting Field-specific considerations ▫Basic Skills ▫New reading standards  Elementary and Early Childhood  19 content areas ▫Revised standards  Reading Teacher  Library Media  Computers, Keyboarding, and Related Technology Applications ▫Vocal and Instrumental Music test ▫World Language and Culture tests

MTLE Test Fields Basic Skills ▫Reading Subtest ▫Writing Subtest ▫Mathematics Subtest Pedagogy Tests: ▫Early Childhood (Birth to Grade 3) ▫Elementary (Grades K-6) ▫Secondary (Grades 5-12)

MTLE Test Fields Content Area Tests: Agricultural Education (Grades 5-12) Business (Grades 5-12) Chemistry (Grades 9-12) Communication Arts/Literature (Grades 5-12) NEW Computer, Keyboarding, and Related Technology Endorsement (Grades K-12) Early Childhood Education (Birth to Grade 3) Earth and Space Science (Grades 9-12) Elementary Education (Grades K-6) English as a Second Language (Grades K-12) Dance (Grades K-12) Family and Consumer Sciences (Grades 5-12) Health (Grades 5-12) Instrumental and Vocal Classroom Music (Grades K-12) Library Media Specialist (Grades K-12) Life Science (Grades 9-12)

MTLE Test Fields Content Area Tests (continued): Mathematics (Grades 5-12) Middle Level Endorsements (Grades 5-8) Communication Arts/Literature Mathematics Science Social Studies Physical Education (Grades K-12) Physics (Grades 9-12) NEW PrePrimary Endorsement (Age 3 to Prekindergarten) Reading Teacher (Grades K-12) Social Studies (Grades 5-12) Special Education Core Skills (Birth to Age 21) Technology (Grades 5-12) Theatre (Grades K-12) Visual Arts (Grades K-12) World Language and Culture (Grades K-12): NEW Chinese (Mandarin), French, German, Spanish