Colorado Online Testing Site Readiness Fall Field Test 2013

Attendance Please take a moment and submit a quick attendance form http://goo.gl/15G73 You may also send an email to Collin Bonner at bonner_c@cde.state.co.us include: Full Name District Title or role Training session attended A record of who attends will help Pearson & CDE identify and reach out to districts who have not been trained

Agenda Site Readiness Training Introduction Online Testing Components Site Readiness Checklist Preparing for Testing Firewalls & Content Filters Proctor Caching System Check TestNav Configuring Proctor Caching in PearsonAccess Providing Support During Testing Early Warning System New Technologies Virtualization & Online Testing Emerging Devices & Online Testing Resources Rework order of agenda

Site Readiness Training Introduction

Testing Overview This presentation covers TestNav 7.5 for Fall 2013. An updated presentation will be available for TestNav 8 for Spring 2014. Most of the concepts regarding site readiness will carry over from TestNav 7.5 so little additional configuration may be necessary.

Site Readiness - High Level Checklist DTC identify School Technology Coordinators STC view Site Readiness Technical Training Configure Firewall/Content filter Identify testing labs and Proctor Caching Machines Install Proctor Caching Complete System Check verification on workstations and wireless testing labs Update workstations to comply with TestNav hardware/software requirements Verify Assessment Environments in the PearsonAccess Training Center. Enter TestNav Configuration(s) in PearsonAccess Production. Notify CDE’s Collin Bonner at bonner_c@cde.state.co.us of the results of step 8 for each district testing site.

Configure Firewall & Content Filter

Configure Firewall/Content Filter Takeaways How should I configure our Firewalls & Content Filters? Section Resources Hardware and Software Requirements link for TestNav 7 TestNav 7 Technology Guidelines Step 3 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Firewalls, Content Filters & Spam Filters The following domains and ports must be allowed to pass through your Internet firewalls, content filters or spam filters. *testnav.com 80, 443 *pearsonaccess.com 80, 443 Note: Districts using IP-based content filters must update the allowed IP addresses as part of this change. These districts will need to contact our Technical Support team to receive the allowed IP addresses and change a setting in PearsonAccess so that the correct URL prints on the Student Authorization Tickets. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Preparing for Testing: Proctor Caching for TestNav Overview

Proctor Caching Takeaways What is Proctor Caching? Where is Proctor Caching placed within the Network? What are the Proctor Caching Requirements? What are the steps to setup Proctor Caching? What information is provided on the Proctor Caching Diagnostics screens? Section Resources Hardware and Software Requirements link for Proctor Caching TestNav 7 Proctor Caching User Guide Proctor Caching media training Steps 4 & 5 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Proctor Caching Proctor Caching is Pearson-supplied software that is used in conjunction with TestNav to reduce bandwidth requirements and accelerate the delivery of test content. Allows you to pre-cache test content to your local network before a test Reduces the burden on your ISP by eliminating redundancy in requests for test content Stores an encrypted local copy of all pre-cached tests Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Diagram of How Proctor Caching Works Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Proctor Caching Requirements Runs on Windows, Mac OS X Does not require an underlying server-based operating system Proctor Caching hardware requirements can be found at http://www.pearsononlinetesting.com/TestNav/7/requirements_proctorcaching_2013_1.html TCP Ports: 80 (Internet), 4480, & 4481 (Local Network) Proctor Caching requires a fixed internal IP address We added Linux here per our contract Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Proctor Caching Setup Download Proctor Caching installer from PearsonAccess. Log into PearsonAccess and click on Support->Downloads Run the installer and Start Proctor Caching if not already started by the installer. Use System Check from client computer to verify that Proctor Caching is functional. Create a Proctor Caching setup or add Proctor Machine within PearsonAccess at District or School level. Could Brian do a demo or a film of these steps? Could people do something with their laptops here? Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Proctor Caching Diagnostics Contents tab for content visibility Pearson On-Line Testing integration Client List tab for monitoring client computer connectivity Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Contents Tab Content is displayed at the Test and Form level Functionality is available to Refresh, Reload or Purge selected test content Clicking on a test name will display the Content Details screen which displays individual items in a test Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Contents Tab – Status Ok – Content is successfully cached Not Loaded – Content is not cached Waiting… - Content is waiting to be loaded Loading… - Content is currently loading Failed to load content – There was a failure to load content MD5 Check Invalid – MD5 comparison could not be completed MD5 Mismatch – MD5 comparison was completed but files did not match In general, any status in red is a critical problem. Any content with a red status should be checked and Reloaded. If the problem is not resolved contact Pearson Support [give an example and what you do?] Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Client List Tab Clients which have connected to TestNav are listed by IP address Clicking on a computer will display the Client Details screen which displays information regarding a particular computer connecting to Proctor Caching Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Client tab – Status Ok – Client is active Idle – 5 to 30 minutes since client was active Long Idle – 30 minutes to 12 hours since client was active Clients are removed after 12 hours of inactivity Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Additional Technical Notes For setting up an upstream proxy refer to the Windows or Mac “Tips on Proctor Caching” section in the TestNav 7 Proctor Caching User Guide Use System Check for TestNav capacity planning in your network environment Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Preparing for Testing: System Check

System Check Takeaways What is System Check? How do I load System Check? What functionality is provided by System Check? How do I use the TestNav 7 tab? How do I use the Network User Capacity tab? How do I use the Testing Volume Calculator tab? Section Resources System Check User Guide Step 6 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

System Check for TestNav Overview System Check was developed as a tool for customers to use to validate their computers to ensure that TestNav software could run on their computers. Support can also use System Check to validate the computer for any technical problems. To gather realistic results System Check must be run from a student testing machine on the same day of week/time of day when you anticipate performing online testing. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Loading System Check System Check can be loaded from: http://www.pearsononlinetesting.com/Colorado and clicking on the link for the System Check tool System Check is Java based and requires that you run and accept an applet Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

System Check Tabs Functionality TestNav 7 Tab Validates testing workstations for TestNav Network User Capacity Tab Bandwidth testing for Internet and Proctor Caching connections Testing Volume Calculator Testing Volume Calculator for capacity planning Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav 7 Tab Validates computer workstation to ensure that minimum requirements are met Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Network User Capacity Tab Please specifically say what the Akamai Network is Internet bandwidth testing to the Pearson & Akamai distributed content network Click Add Caching Computer to add your potential Proctor Caching machine for a local network bandwidth test Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Network User Capacity Tab: Add Caching Computer Add a name for the caching computer, the IP address and port (4480) Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Network User Capacity Tab: Results Click on Start to begin the test Based on the bandwidth test System Check can provide the number of students that can test concurrently. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Testing Volume Calculator Tab Input number of students testing concurrently. How many testing labs and student testing workstations are available? Modify network links and network utilization sliders as necessary. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Testing Volume Calculator Tab - Interpretation Planned testing is feasible with available network capacity Planned testing is dangerously close to available network capacity Planned testing exceeds the available network capacity The graph will display the results based on your inputs. The background should remain white to ensure your planned testing volume can be supported by your available network capacity. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Preparing for Testing: TestNav Online Testing Platform

TestNav Takeaways Step 7 What are the TestNav Technology Dependencies? What are TestNav Best Practices and Recommendations? What do I need to know about wireless testing? What are Student Response Files (SRFs)? What directory permissions are needed for TestNav? Section Resources Hardware and Software Requirements link for TestNav 7 TestNav 7 Technology Guidelines Step 7 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav Technology Dependencies Firewalls, content filters and spam filters must allow the Pearson domains Minimum Screen Resolution of 1024 x 768 is required Browser Dependencies: Adobe Flash Player version 10 and higher Java runtime plugin version 1.5 and higher Windows Firewall configured to allow javaw.exe to communicate Allow pop-ups for Pearson sites Accept Java applet Allow Local File access to home directory Complete requirements can be found at http://pearsononlinetesting.com/TestNav/7/ Note: Pearson continually monitors and evaluates the hardware and software requirements. As vendors release newer versions of their products, we update the recommended requirements accordingly. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Best Practices and Recommendations Performance: Use Proctor Caching for best network performance. Use System Check extensively to test Proctor Caching and network environment before live testing. Use System Check to test wireless network capacity if using wireless clients. Testing Computers: Use System Check (TestNav Verification Test) to validate that TestNav will load on machines. Freeze browser, Adobe Flash Player, & Java environment if possible. Energy Saving features can interrupt testing, so the recommendation is to set those to not trigger during the length of the test. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Best Practices and Recommendations Testing Computers: Applications that launch automatically Any applications that may automatically launch on a computer should be configured not to launch during testing sessions. Common applications that may launch automatically include: Anti-virus software performing automatic updates Power management software on laptops warning of low batteries Screen Savers Email with Auto message notification Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Best Practices and Recommendations For wireless testing: Wireless access points – more is better Distance – closer is better Limit the number of computers per wireless access point Try a small number first for proof of concept Do not roll out a few hundred on the first attempt (baby steps) Limit obstructions Use the System Check Utility to verify that you have the appropriate amount of bandwidth for online testing Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav Directories TestNav creates a Pearson directory in the home directory of the testing computer: Windows XP C:\Documents and Settings\username\Pearson\... Windows Vista and 7 C:\Users\username\Pearson\... Mac //Users/username/pearson/... Student Log files: Pearson/student/logs SRF file default location: Pearson/student/srf TestNav and test content is temporarily stored locally: Pearson/temp Pearson recommends that the Pearson directory be excluded if desktop restoration tools such as DeepFreeze are used Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Student Responses TestNav will save student responses to an encrypted backup file so the student can either continue testing or exit the system without losing response data. TestNav requires at least one working save location TestNav by default saves student response data to: <home dir>/Pearson/student/srf Create a TestNav Config to specify custom Primary and Alternate file locations Pearson recommends designating two Response File Save Locations Best Practice: Primary Response File Location should be on the student workstation. The alternate Response File Location should be on a shared network location Students must have full access to designated Response File Locations Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Student Response Files (SRF) SRF file configured for network and local save locations Response file will be saved to both the individual student’s testing workstation and a network location Student may resume on any workstation that has access to the shared network save location Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Student Response Files (SRF) SRF file configured for local save locations only All response files will save only to each individual student’s testing workstation Student must resume on the same workstation or SRF files must be moved to the workstation where the student will resume the test If the SRF files are not placed in the same location on the new workstation TestNav will prompt the student to navigate to where the SRF files are saved when they resume testing Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Student Response Files (SRF) SRF file configured for network save location(s) only Response files will save to one or more network locations Student may resume on any workstation that has access to the shared network save location(s) Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Using PearsonAccess to Configure Proctor Caching

Proctor Caching Takeaways How do I use PearsonAccess to Configure TestNav for Proctor Caching Setup? Section Resources TestNav 7 Proctor Caching User Guide Proctor Caching media training Steps 9 Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Using PearsonAccess to Configure Proctor Caching Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Providing Support during Testing: Early Warning System & Quick Reference Guide

Early Warning System Takeaways What is the Early Warning System? What are the TestNav Early Warning System Triggers? How do I resolve Early Warning System alerts? Section Resources Early Warning System User Guide Please make this relevant to DTCs. They need to know this because they may be providing support during testing. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Early Warning System (EWS) The Early Warning System (EWS) is integrated functionality that gives TestNav a high degree of fault tolerance and provides additional fail-safes in the event of unexpected network disruptions during online testing. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Early Warning System Design Minimal interruptions for students Multiple physical file backups to safeguard against data loss Notification Screens designed for Test Proctors Early Warning System notifications often involve IT staff so an understanding is necessary to correct any issues during testing Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Student Responses – SRF Files The EWS writes continuously in the background to the student response files. Both the Primary and Alternate files are written to at the same time. The SRF file has a response data threshold that once reached will trigger TestNav to send response data to Pearson. Uploading of response data is continuous. If an upload fails TestNav will cycle and attempt another upload. If the response data upload is successful, TestNav will create a new empty SRF file and begin the process again. TestNav will only delete an SRF file once it is successfully uploaded. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Student Responses - Identification A combination of the test ticket ID combined with the session token is used to uniquely identify a Student Response File (SRF). TestNav can identify the correct student response file if a test is successfully resumed. The SRF can only be used the next time the student resumes a test. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav Early Warning System - Triggers Scenario 1: TestNav determines that neither Saving Response File Location is viable. Will result in immediate Test Proctor Click Here notification. Scenario 2: TestNav is unable to download portions of the test. Scenario 3: TestNav is unable to upload student responses to Pearson. Will result in Test Proctor Click Here notification at the time of test submission. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav EWS – Test Proctor Click Here Screens One of the following screens will appear when scenario 1, 2 or 3 occurs. Students should ALWAYS raise their hand when presented with either of the Test Proctor Click Here screens. They should NEVER click the Test Proctor-Click Here button. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav EWS – Scenario 1 In this scenario there is an issue with both of the Response File Locations. Either the student does not have full access to the location(s), OR the location(s) are temporarily unavailable. Resolve by entering and saving a viable Response File Location. If the issue is temporary, reselect the original location. Then, click the Save and Continue button. Best Practice: Contact your local Technology Coordinator to determine the appropriate course of action. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav EWS – Scenario 2 In this scenario, TestNav cannot access the test content on the student work station. If Proctor Caching is being utilized, either the Proctor Caching Server is not functioning properly, OR the local network and Pearson are not communicating. The ONLY way to resolve this situation is to click the “Close TestNav” button. The student’s testing status will remain as STARTED in PearsonAccess™. Once communication is restored, Resume the student in PearsonAccess and have the student log back into TestNav. Best Practice: Your local Technology Coordinator should be contacted to determine the appropriate course of action. TestNav makes several attempts to access the test content. This message indicates all attempts to retrieve test content have failed. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav EWS – Scenario 3 In this scenario not all of the student responses have been received by the Pearson testing server. However, responses not received by Pearson have been saved locally to the Primary Response File Location. You may try to resolve this scenario by clicking the Retry button until the submission of responses is successful or click on Close TestNav Best Practice: If clicking the Retry button does not resolve the situation, contact your local Technology Coordinator to determine the appropriate course of action. TestNav makes several attempts to communicate with the Pearson Testing Server. This message indicates attempts to submit some or all of the student responses have failed. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Quick Reference Guide The Quick Reference Guide should be available to all test proctors to help support students as they test. The Guide provides a list of all the warning messages that TestNav may show a student and how to resolve them. Available through PearsonAccess.com->Colorado->Support->Reference Guides->Online Test Administrator Quick Reference Guide Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

New Technologies: Virtualization & Online Testing

Virtualization Takeaways What is virtualization? What are the concerns with virtualization and online testing? Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Virtualization What is it? Using your monitor/keyboard/mouse to look at and run software on some other computer “Thin Clients” are one type of virtualization VDI (Virtual Desktop Infrastructure) is becoming popular as a way of turning aging computers into terminals, instead of retiring them There are almost as many different solutions as there are solutions providers

Virtualization Virtualization often appeals to schools looking to lower the student to computer ratio within limited IT budgets Virtualization presents unique security considerations when used for high-stakes online testing Schools & districts may already be using virtualization technology without Department of Education knowledge

Virtualization Network Virtualized environment runs on a central server and streams the user interface to workstations Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Virtualization Security Concerns Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Virtualization Concerns Virtualization security settings on the server are typically unable to be accessed or controlled by the test delivery engine Streamed data between the virtualization server and workstations may not be encrypted and could be intercepted by “man in the middle” exploits Secure test content Student keystrokes and interactions with TestNav Student responses Virtualization on student workstations may not properly lock down the environment and provide a secure online testing environment Unable to prevent student use of other applications Unable to prevent capturing screen and test content Lack of TestNav visibility to virtualization “modal” windows Virtualization performance issues could negatively impact students’ ability to test Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

VDI – Keeping Aging Computers In Service Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

A Worst-Case VDI Scenario Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

TestNav and Virtualization Pearson is working with individual virtualization vendors to qualify their solutions for use with TestNav Approved virtualization solutions will be listed on: www.TestNavQualified.com Non-approved virtualization solutions may expose online testing to exposure of secure content Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

New Technologies: Emerging Devices & Online Testing

Takeaways What technology changes are taking place in schools? What does comparability mean in online testing? Agenda Slide. Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Online Testing & Technology Innovation Technology Markets – Always Changing Innovation drives continual change in technology markets As technology evolves, the testing environment also changes Incremental updates: OS’s, browsers, Flash, Java, HTML New hardware & devices: netbooks, tablets, smartphones New network architectures: thin clients, virtualization, cloud computing New human/computer interfaces: mouse, touch, speech recognition, motion-controlled interaction (eg, Wii, Xbox, Kinect) A transition to technology-based assessments is not a one-time change

Diversity in the School Technology Environment

Comparability & “Standardized” Testing Comparability between Paper and Online Tests Decades of research looking at differences in experience — and impact to student performance — has resulted in: Scaled score adjustments Content & design changes Certain technology requirements Typically has treated online as the same experience for all students Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

New Comparability Research Focus Comparability between Devices Differences between devices are becoming more pronounced New research is beginning to look at differences within the “online” condition Areas of Research Display size Text input – keyboards vs. on-screen typing Interface – mouse vs. touch Cursor-based feedback Control & fine-point precision Ability to see through/around cursor Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Display Size Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Text Input

Mouse- & Touch-based Interfaces

Managing “Ongoing Change” in Testing Supporting Research Only way to know comparability impact is through research studies and field tests Expect that technology adoption will always be slowest in the realm of “high-stakes” testing Communication Always Helps Don’t hesitate to contact Pearson with questions Please share what is happening with technology in your schools Help educate your local technology staff about the critical connection between technology at the student level and statewide comparability Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Resources

Documents and Links DTC Academy: https://sites.google.com/a/elearningbydesign.net/cde-dtc-field-test/home or http://tinyurl.com/goDTCs CDE DTC webpage: http://goo.gl/1sM6QC PearsonAccess Colorado Support Page System Check User Guide System Check tool link Hardware and Software Requirements link for TestNav 7 Hardware and Software Requirements link for Proctor Caching TestNav 7 Technology Guidelines TestNav 7 Proctor Caching User Guide Proctor Caching media training Early Warning System User Guide Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Support Contacts Collin Bonner CDE Assessment Technology Specialist 303-866-6752 bonner_c@cde.state.co.us Pearson Support 888-687-4759 COHelp@support.pearson.com Districts/Schools should use Pearson as the first contact for support Copyright © 2013 Pearson Education, Inc. or its affiliates. All rights reserved.

Questions & Answers?

CDE Site Readiness Training Evaluation 2013 Please take a moment to complete the evaluation of this training: http://goo.gl/e74GLS or http://tinyurl.com/CDEeval