Tools for Schools: Developing End- of-Course Assessments Statewide Scott Cook, ISAT Coordinator Rob Sauer, Deputy Superintendent of Innovation and Choice Carissa Miller, PhD, Deputy Superintendent of Assessment

Outline A. Purpose, goals and uses of End-of-Course Assessments (EOCs) B. EOC test development plan C. Mentor/mentee model of development D. Other tools available for districts E. Creation logistics, getting involved F. Questions and comments

Assessment Purposes Variety of uses –Classroom tests/quizzes –Interim/benchmark assessments –Summative assessments –EOCs in classrooms (15% of grade) –Mastery tests –Graduation exams

Good Testing Practices* –Alignment to standards –Validity Are we testing what we intend to measure? –Reliability Can the measure be repeated with confidence? Pearson correlation, Cronbach’s alpha Differential Item Functioning (DIF) – performance among subgroups –Rigor Sufficient rigor for the course/purpose? –Comparability –Scoring and Reporting *American Educational Research Association, (1999) “Standards for Educational and Psychological Testing,” Washington, DC: AERA.

State EOC Uses Used as a tool to inform instruction Potentially for alignment of curriculum –Textbooks and course outlines vary Augmenting assessments within district/schools Item banks

EOC Test Development Plan Provide Sample End-of-Course examinations in: –Biology, Physical Science and Earth Science –Geometry, Algebra 1, Pre-Algebra Provide an Open source Item bank in these course areas- For local assessment For formative assessment to take place

Test Development: ISAT/EOC Comparison  ISAT Start to finish: roughly a 2-year process  Trained item writers follow standards, content limits; depth and range of knowledge, balance of representation  Items are reviewed by Idaho teachers before piloting  Piloted items are reviewed again for statistical analysis and alignment to standards and content limits

ISAT Test Development [cont.]  Bias review  Item correlation  DIF analysis  Distractor analysis  % correct values  Building test forms  Forms review  Census test ready

EOC Test Development Plan  Multiple choice format  Combine item writing and training with mentor/mentee model  Using experienced EOC item writers and inviting teachers from schools in “Needs Improvement Status”--access SI dollars  Professional development a positive by-product of process  Building item bank with existing items from Boise, Meridian, and Twin Falls [thanks!]

Tasks ahead………….  SDE must finalize or write standards which inform sample Test Blueprints- Geometry and Algebra course outlines exist  Create individual online item 1,800 for 6 courses, load in SchoolNet  Create list of focus words that inform search function

EOC Workshop, Nov  Experienced item writers are paired with novice items writers  10 individuals per course group  Whole group trains in basics of item writing  Groups begin to code items to test blueprints

Workshop Week [cont]  Groups select existing items that have worked well in districts for sample test forms  Groups write new items to flesh out test forms  Test forms are created  Test forms are reviewed  Items not used in forms revert to bank

Housing of EOCs  SchoolNet will house test forms and item bank  Educators will access through Single sign in portal and will need permission to use EOC application  Ability to create formative assessment, local EOCs, unit tests by searching standards or focus words--- ex., “ Distributive property.”

SCHOOLNET Demo Chapter 4: Creating Tests The Create page is where you create new tests and new test items. The test creation process likely began with the test blueprints and high-level benchmarking goals being set prior to the actual test creation process in Assess. Creating the test using the form below creates a test shell where items are then added from Item Central, or are authored right from the test details page. After the test content is entered, Assess supports a common workflow that takes the test into various stages, including a Public Draft that allows content and assessment specialists to review the items, alignments, metadata, and online testing display collaboratively. To generate a test, see: Step 1: Getting Started Step 2: Choosing an Item Type Step 3: Working within Item Types See also: Test Stages Test Detail Customizing Tests Modifying Tests Deleting Tests Creating Passages: Getting Started Creating Passages: Adding Content

Other Tools Available Course Outlines Courses of Study Sample Lesson Plans linked to Standards Formative Assessments Identify and Share Links to Sites and Best Practices

Creation Logistics When? - November Where? - Boise How will participants be invited? - Federal Requirements - We will be contacting districts that are eligible - Brief application will be required - Mentors will be selected All districts will benefit

Final thoughts  Teachers access items and test forms on SchoolNet platform  Local access, local control  Finalizing course test forms, QA items, training and rollout Winter 2011

Questions?

Contact Us If your district is interested in participating: Rob Sauer (208)