1. Tools for Schools: Developing End- of-Course Assessments Statewide Scott Cook, ISAT Coordinator Rob Sauer, Deputy Superintendent of Innovation and Choice Carissa Miller, PhD, Deputy Superintendent of Assessment

2. Outline A. Purpose, goals and uses of End-of-Course Assessments (EOCs) B. EOC test development plan C. Mentor/mentee model of development D. Other tools available for districts E. Creation logistics, getting involved F. Questions and comments

3. Assessment Purposes Variety of uses –Classroom tests/quizzes –Interim/benchmark assessments –Summative assessments –EOCs in classrooms (15% of grade) –Mastery tests –Graduation exams

4. Good Testing Practices* –Alignment to standards –Validity Are we testing what we intend to measure? –Reliability Can the measure be repeated with confidence? Pearson correlation, Cronbach’s alpha Differential Item Functioning (DIF) – performance among subgroups –Rigor Sufficient rigor for the course/purpose? –Comparability –Scoring and Reporting *American Educational Research Association, (1999) “Standards for Educational and Psychological Testing,” Washington, DC: AERA.

5. State EOC Uses Used as a tool to inform instruction Potentially for alignment of curriculum –Textbooks and course outlines vary Augmenting assessments within district/schools Item banks

6. EOC Test Development Plan Provide Sample End-of-Course examinations in: –Biology, Physical Science and Earth Science –Geometry, Algebra 1, Pre-Algebra Provide an Open source Item bank in these course areas- For local assessment For formative assessment to take place

7. Test Development: ISAT/EOC Comparison  ISAT Start to finish: roughly a 2-year process  Trained item writers follow standards, content limits; depth and range of knowledge, balance of representation  Items are reviewed by Idaho teachers before piloting  Piloted items are reviewed again for statistical analysis and alignment to standards and content limits

8. ISAT Test Development [cont.]  Bias review  Item correlation  DIF analysis  Distractor analysis  % correct values  Building test forms  Forms review  Census test ready

9. EOC Test Development Plan  Multiple choice format  Combine item writing and training with mentor/mentee model  Using experienced EOC item writers and inviting teachers from schools in “Needs Improvement Status”--access SI dollars  Professional development a positive by-product of process  Building item bank with existing items from Boise, Meridian, and Twin Falls [thanks!]

10. Tasks ahead………….  SDE must finalize or write standards which inform sample Test Blueprints- Geometry and Algebra course outlines exist  Create individual online item cards- @ 1,800 for 6 courses, load in SchoolNet  Create list of focus words that inform search function

11. EOC Workshop, Nov. 2010  Experienced item writers are paired with novice items writers  10 individuals per course group  Whole group trains in basics of item writing  Groups begin to code items to test blueprints

12. Workshop Week [cont]  Groups select existing items that have worked well in districts for sample test forms  Groups write new items to flesh out test forms  Test forms are created  Test forms are reviewed  Items not used in forms revert to bank

13. Housing of EOCs  SchoolNet will house test forms and item bank  Educators will access through Single sign in portal and will need permission to use EOC application  Ability to create formative assessment, local EOCs, unit tests by searching standards or focus words--- ex., “ Distributive property.”

14. SCHOOLNET Demo Chapter 4: Creating Tests The Create page is where you create new tests and new test items. The test creation process likely began with the test blueprints and high-level benchmarking goals being set prior to the actual test creation process in Assess. Creating the test using the form below creates a test shell where items are then added from Item Central, or are authored right from the test details page. After the test content is entered, Assess supports a common workflow that takes the test into various stages, including a Public Draft that allows content and assessment specialists to review the items, alignments, metadata, and online testing display collaboratively. To generate a test, see: Step 1: Getting Started Step 2: Choosing an Item Type Step 3: Working within Item Types See also: Test Stages Test Detail Customizing Tests Modifying Tests Deleting Tests Creating Passages: Getting Started Creating Passages: Adding Content

15. Other Tools Available Course Outlines Courses of Study Sample Lesson Plans linked to Standards Formative Assessments Identify and Share Links to Sites and Best Practices

16. Creation Logistics When? - November Where? - Boise How will participants be invited? - Federal Requirements - We will be contacting districts that are eligible - Brief application will be required - Mentors will be selected All districts will benefit

17. Final thoughts  Teachers access items and test forms on SchoolNet platform  Local access, local control  Finalizing course test forms, QA items, training and rollout Winter 2011

18. Questions?

19. Contact Us If your district is interested in participating: Rob Sauer rcsauer@sde.idaho.gov (208) 332-6934