2009 NCTA Conference  San Antonio The Online Assessment Advantage at Tyler Junior College Ken Craver Director of Distance Education

2009 NCTA Conference  San Antonio Slide 2 What We Will Cover  Who we are  Why Questionmark  What we do Our Server Configuration Assessment Creation and Scheduling Assessment Delivery  Download-ables ( These slides Questionmark Bridge Questionmark Bridge User’s Guide More documents available upon request

2009 NCTA Conference  San Antonio Slide 3 Who We Are  Multi-campus junior college  students  3700 / semester in distance education  60+ faculty use Questionmark Perception in 90 different courses  4 support persons  Proctored testing in TJC Testing Center

2009 NCTA Conference  San Antonio Slide 4 Why Questionmark Our Students…  are computer learners  desire flexibility in schedules  can benefit from more and varied assessment opportunities  like immediate feedback on their grades and test questions missed  can be anywhere in the world

2009 NCTA Conference  San Antonio Slide 5 Why Questionmark Our Faculty (why they like it) …  do not lose valuable class time to giving assessments (4- 6 class periods / semester)  can provide more effective assessments using a variety of question types  can categorize and tag questions to allow more intelligent question selection  can give proctored exams in our testing center  can give practice exams students can take from anywhere  can use assessments as learning tools  can obtain a wide variety of statistical reports on assessment results.

2009 NCTA Conference  San Antonio Why Questionmark Some History  Originally championed by our Director of Testing  I was paid a stipend to administer the server along with training and supporting faculty  Also paid faculty member a stipend to train and support faculty  computer testing stations  Tables for paper tests Slide 6

2009 NCTA Conference  San Antonio Slide 7 Why Questionmark We Use Perception for…  Pre-course Assessments  Practice Quizzes  Exams  Surveys

2009 NCTA Conference  San Antonio Why Questionmark Question Types Slide 8  Adobe Captivate  Adobe Flash  Drag and Drop  Essay  Explanation  File Upload  Fill in Blanks  Hotspot  Likert  Matching  Matrix  Multiple Choice  Multiple Response  Numeric  Pull-down List  Ranking  Select a Blank  Spoken Response  Text Match  True/False  Yes/No

2009 NCTA Conference  San Antonio Slide 9 Server Configuration Questionmark Version 4.4

2009 NCTA Conference  San Antonio Slide 10 Assessment Process Our Process 1  Item bank creation Create topic structure Import multiple-choice questions using Questionmark Bridge (in-house program)  Assessment Creation  Assessment Publishing (just a copy from development repository to production repository)  Group Creation

2009 NCTA Conference  San Antonio Slide 11 Assessment Process Our Process 2  Participant Creation Import into group using Questionmark Bridge All class rolls are sent as text files to a secure FTP site Rolls are downloaded and participants are imported and linked to groups  Exam scheduling form completed Online form completed by each instructor that includes course, Exams to be given, deadlines  Assessments scheduled according to exam scheduling form  Results checked by instructor

2009 NCTA Conference  San Antonio Slide 12 Assessment Process Repository Structure TJC.qmr Shared Production repository TJC-Dev.qmr Shared Development repository Local Development repositories (on faculty computers)

2009 NCTA Conference  San Antonio Slide 13 Assessment Process Supporting Faculty  Support Team 2 instructors, 1 staff, 1 part-time employee Semester stipend equal to 3 hour overload contract Responsibilities  Create question topic structure and import/create questions sent by faculty  Create assessments  Create groups and import participants  Schedule assessments  Provide support  Faculty Designers (Do-it-Yourself-ers) Do some or all of their own design, implementation and scheduling work  Faculty Users Deliver questions to the support team and specify the content of their exams Complete online scheduling form for the exams they want scheduled Check student scores using Enterprise Manager web-based tool

2009 NCTA Conference  San Antonio Slide 14 Assessment Process Item Bank Creation  We import multiple choice questions 1. Format questions in a text file as follows: 1. Question stem containing the text of the question. a. Answer 1 b. Answer 2 =c. Answer 3 (correct answer) d. Answer 4 2. Use the Questionmark Bridge program to create an import file. 3. Import the “import file” to an Authoring Manager topic.

2009 NCTA Conference  San Antonio Slide 15 Assessment Process Questionmark Bridge Text file answers marked Prepare text file Questionmark Bridge (“import.txt”) Import “import.txt” imported into Authoring Manager

2009 NCTA Conference  San Antonio Assessment Process Authoring Manager Slide 16

2009 NCTA Conference  San Antonio Slide 17 Assessment Process Assessment Creation  Created in TJC-Dev development shared repository  Copied to TJC production repository when ready to be scheduled  All assessments use same custom template No changes allowed

2009 NCTA Conference  San Antonio Slide 18 Assessment Process Assessment Publishing

2009 NCTA Conference  San Antonio Slide 19 Assessment Process Group Creation  Group typically = class section  Use naming standard to make groups easy to find  Delete old groups at start of each semester

2009 NCTA Conference  San Antonio Slide 20 Assessment Process Participant Creation 1  Download all class rolls as text files from internal FTP site that is populated automatically from our Banner system Done second week of semester  Support team or faculty developer downloads roll from FTP site  Roll is prepared for import using Questionmark Bridge

2009 NCTA Conference  San Antonio Slide 21 Assessment Process Participant Creation 2  “Import.txt” file is imported into Questionmark and participants linked to a group.

2009 NCTA Conference  San Antonio Slide 22 Assessment Process Exam Scheduling Form and Database Zoho Creator form - (Free!)

2009 NCTA Conference  San Antonio Slide 23 Assessment Process Exam Scheduling Database  Available to all of our support team and our Testing Center personnel  If deadline for an exam changes faculty will submit the change via the scheduling form

2009 NCTA Conference  San Antonio Slide 24 Assessment Process Assessment Scheduling  Scheduled according to faculty input on scheduling form  Scheduled to groups  Deadlines may or may not be set on the schedule.

2009 NCTA Conference  San Antonio Slide 25 Assessment Process Results Checking  Instructors check their own results using Enterprise Reporter.  Common Reports Score List with Coaching Report Question Statistics Report Gradebook Report Transcript Report Test Analysis Report used by Nursing Instructors

2009 NCTA Conference  San Antonio Slide 26 Delivering Assessments Our Testing Center  50 testing stations  Test over 33 subjects  Average 3000 tests / month (long semester)  Peak of 600 tests / day  4 Full time staff  12 Part time staff

2009 NCTA Conference  San Antonio Slide 27 Delivering Assessments Our Testing Center Entry Point

2009 NCTA Conference  San Antonio Delivering Assessments Early Testing Center Slide 28

2009 NCTA Conference  San Antonio Slide 29 Delivering Assessments Current Testing Center Two types of Nova desk testing stations

2009 NCTA Conference  San Antonio Slide 30 Delivering Assessments Our Testing Center Nova desks with hoods contain computer with a monitor that is viewable through the glass top

2009 NCTA Conference  San Antonio Slide 31 Delivering Assessments Our Testing Center Security Cameras

2009 NCTA Conference  San Antonio Slide 32 Delivering Assessments Our Testing Center Security Monitor at front desk

2009 NCTA Conference  San Antonio Slide 33 Delivering Assessments Testing Center Procedures 1  Check scheduling database (Zoho) report for instructors using QuestionMark, special instructions, courses and their deadlines  Notify front desk at testing center of courses, instructors, and special instructions for QuestionMark users  Students must be on the classroll, on the QuestionMark roster with their personal identification number to be able to test  Students must show a valid photo ID and sign in before testing  All electronic devices must be turned off before entering the testing center  A proctor will take the student’s ID, select a computer, and enter the student into their test.  During the testing, the proctors monitor the testing area looking for any unusual activities.

2009 NCTA Conference  San Antonio Slide 34 Delivering Assessments Testing Center Procedures 2  Student will print a copy of their results if feature enabled by instructor  Constant visual monitoring by inside proctors and video cameras  Able to view any activities after the fact and download them if needed.  Give computer based test for any type of class  Paper based tests are allowed (but not encouraged) for online classes only  Also, use Blackboard, SAMS, Word, and Excel for testing purposes  Give Virtual College of Texas (VCT) exams and correspondence exams. Fee based  Off-site students - send Questionmark Secure (browser add-on) to be installed on test station.

2009 NCTA Conference  San Antonio Slide 35 Delivering Assessments QM Skills for Testing Center Manager  Can add participants and link them to a group  Can re-activate an expired exam.  Can check exam scores

2009 NCTA Conference  San Antonio Slide 36 Delivering Assessments Future  Satellite Testing Centers  Software to enable remote access to testing computers.

2009 NCTA Conference  San Antonio Slide 37 Delivering Assessments Security Options  Open access for low stakes  Secure access for high stakes Cameras in operation User name and password validated for monitored assessments:  Against Perception database  Only proctor knows password QuestionMark Secure browser add-in to prevent printing and browsing  Cannot access any other program once browser started  Cannot close browser  Can only take assessment

2009 NCTA Conference  San Antonio Slide 38 Thank You!  Ken Craver  Downloads Web Site