2/15/2006

Online Testing Initiatives in North Carolina Presenters: Randy Craven Director for Information Technology Technical Outreach for Public Schools (TOPS), NC State University Scott Ragsdale Project Manager, North Carolina Computer Skills Assessments + Technical Outreach for Public Schools (TOPS), NC State University

2/15/2006 Modern science has been a voyage into the unknown, with a lesson in humility waiting at every stop. Many passengers would rather have stayed home. Carl Sagan

2/15/2006 The purpose of this session is to update the audience on NCDesk, the Online Test of Computer Skills, and other online assessments used in North Carolina testing.

2/15/2006 What Initiatives? Online Test of Computer SkillsOnline Test of Computer Skills End-of-Course Online Physics Field Test (mandatory)End-of-Course Online Physics Field Test (mandatory) End-of-Course Tests (optional 06-07)End-of-Course Tests (optional 06-07) Writing Online Special Study (Grades 7 and 10)Writing Online Special Study (Grades 7 and 10) Online formative assessments (i.e., ClassScape)Online formative assessments (i.e., ClassScape) Other online resourced assessmentsOther online resourced assessments –NCEXTEND1 –NCCLAS Other web applications supporting test development/maintenance:Other web applications supporting test development/maintenance: –Online Item Writing System –Experimental Section Review (ESR)

2/15/2006 Online Assessments Success The Online Test of Computer Skills was successful in its initial administration year of and continues to be successful this year.The Online Test of Computer Skills was successful in its initial administration year of and continues to be successful this year. NCCLAS online data collection was successful in and continues to work this year.NCCLAS online data collection was successful in and continues to work this year. Online Physics Field Test was administered to over 3,700 students during the fall administration with no significant issues.Online Physics Field Test was administered to over 3,700 students during the fall administration with no significant issues. For , NCEXTEND1 students have been matched to their assessors and have had their Decision Tree processes completed.For , NCEXTEND1 students have been matched to their assessors and have had their Decision Tree processes completed.

2/15/2006 Elements of Online Assessments More interactiveMore interactive Test Simulations required to be availableTest Simulations required to be available More data can be collectedMore data can be collected Opportunity to expand the use of innovative itemsOpportunity to expand the use of innovative items –Virtual/performance-based items –Short and/or extended response Quicker data processing and reportingQuicker data processing and reporting Reduction of materials managementReduction of materials management Students have a positive attitude for online testingStudents have a positive attitude for online testing

2/15/2006 Elements of Online Assessments (continued) TechnologyTechnology –Internet aware applications (NCDesk) Java versionJava version –Browser based (EOC & Physics) Session State CookiesSession State Cookies JavascriptJavascript Flashpaper / Interactive Flash Items / Flash playerFlashpaper / Interactive Flash Items / Flash player VersionsVersions

2/15/2006 Performance Components Online Test of Computer SkillsOnline Test of Computer Skills –50% multiple choice –50% performance Physics OnlinePhysics Online –Mostly multiple choice –Small number of performance-based items EOC OnlineEOC Online –All multiple-choice Online Writing Special StudyOnline Writing Special Study –Single prompt with essay response

2/15/2006 Preservation of State Students expect responses to items to retain state as they move from one item to another and if they return to an item, it will be as they had left it before.Students expect responses to items to retain state as they move from one item to another and if they return to an item, it will be as they had left it before. Preservation of state is not always easy since online assessment systems must be able to recover from disconnection.Preservation of state is not always easy since online assessment systems must be able to recover from disconnection. At this time for the Online Test of Computer Skills and the EOC Online Tests we are able to manage state for all items except the item the student is currently working on.At this time for the Online Test of Computer Skills and the EOC Online Tests we are able to manage state for all items except the item the student is currently working on.

2/15/2006 Problems/Opportunities Rate of technology changeRate of technology change Window of opportunity for testingWindow of opportunity for testing SchedulingScheduling Having enough…Having enough… –Bandwidth –Computers –Space –Time

2/15/2006 Managing Things NCRegistrationNCRegistration –User Accounts –Student Registrations –Scheduling Sessions –Receiving Reports

2/15/2006 "There is no reason anyone would want a computer in their home." Ken Olson President, chairman and founder of Digital Equipment Corporation (DEC), maker of business minicomputers, arguing against the PC in 1977.

2/15/2006 Online Test of Computer Skills Notable Preliminary/Unofficial Numbers from Fall students tested in one day at one school (for this was 139)205 students tested in one day at one school (for this was 139) 52 schools tested 100 or more students in one day (for this was 7)52 schools tested 100 or more students in one day (for this was 7) 5,068 maximum testing sessions used in a single day (for this was 3,063)5,068 maximum testing sessions used in a single day (for this was 3,063) 23 days of testing in which volume of used sessions was 3,000 or more23 days of testing in which volume of used sessions was 3,000 or more ~133,980 total students tested~133,980 total students tested ~106,035 8 th grade students tested~106,035 8 th grade students tested ~2,228 different test administrators with 52 logging in 300 or more students~2,228 different test administrators with 52 logging in 300 or more students ~3,446 students had the Computer Reads Aloud option~3,446 students had the Computer Reads Aloud option

2/15/2006 Online Progress Indicators

2/15/2006 Platform Shifting

2/15/2006 Online Computer Skills Items for Spring 2007 Testing Window 3/12/07 – 6/8/07Testing Window 3/12/07 – 6/8/07 NCRegistration accessible on 2/5/07 for scheduling, etc.NCRegistration accessible on 2/5/07 for scheduling, etc. Reports available to LEA Test Coordinators through NCRegistrationReports available to LEA Test Coordinators through NCRegistration JAWS compatibility not tested/approved – also newer versions of JAWS released since last tested – research/development ongoing processJAWS compatibility not tested/approved – also newer versions of JAWS released since last tested – research/development ongoing process

2/15/2006 Even if you're on the right track you'll get run over if you just sit there. Will Rogers

2/15/2006 NCDesk Moving Forward Adjustments for accommodating delivery of Writing assessments onlineAdjustments for accommodating delivery of Writing assessments online –Assessment selection on login page –Programmatically remove spell check functions –Programmatically remove image insertion functions –Other features deemed necessary Additional features for testing audio functionalityAdditional features for testing audio functionality Additional features for viewing correct responses for the Test Simulation possibleAdditional features for viewing correct responses for the Test Simulation possible

2/15/2006 EOC Online (Optional) Fall 2006Fall 2006 –First time EOCs offered online as an option –One school participated in Algebra I No problems reportedNo problems reported Students “enjoyed” itStudents “enjoyed” it Spring 2007Spring 2007 –Registration & Scheduling now available via NCRegistration Not included:Not included: –Chemistry Field Test –Physical Science Field Test –Biology Field Test

2/15/2006 EOCs Online Moving Forward (Next Year) Making the answer choices clickableMaking the answer choices clickable Improvement of the calculatorImprovement of the calculator –Moveable/draggable on screen (within test interface) –Add graphing capability Improvement of online reference contentImprovement of online reference content –Moveable/draggable on screen (within test interface) Attempt to improve item view state preservationAttempt to improve item view state preservation –Position –Zoom percentage Optional to continue until such time as it is no longer feasible to maintain paper-and-pencil testing.Optional to continue until such time as it is no longer feasible to maintain paper-and-pencil testing.

2/15/2006 NCEXTEND2 Online Process under way to present all NCEXTEND2 in an online formatProcess under way to present all NCEXTEND2 in an online format Format similar to EOC OnlineFormat similar to EOC Online Format for Writing would be delivered using NCDesk with a single editor item.Format for Writing would be delivered using NCDesk with a single editor item.

2/15/2006 Computers are useless. They can only give you answers. Pablo Picasso