1. Perception Orientation
Presented By:
Wes Cloven

2. Overview of Questionmark
Focused on providing software to create, deliver and report on assessments
More than 2,000 customers have purchased Perception
More than 14,000 authoring systems installed
Millions and millions of candidates configured within the servers
Millions and millions of assessments conducted
Offices in US, UK, Belgium
Translated into Spanish, German, French, Dutch and Japanese
More than 70 man years of development in Perception

3. Why Computerized Assessments?
It helps students by:
Correcting misconceptions
Raising confidence levels
Directing them to valuable learning materials
Helping them know which subjects to focus on
It helps instructors and others by:
Helping them know where to focus
Quickly measuring knowledge, skills and abilities
Change learning environments based on feedback .

4. Architecture

5. Architecture

6. Architecture

7. Architecture

8. Architecture

9. The Big Picture

10. The Big Picture
Create questions and group into quizzes, tests, exams and surveys using a web browser or Windows PC

11. The Big Picture
Add, import, change, and delete participants, groups and their schedules

12. The Big Picture
Deliver assessment using a standard or secure browser, PDA’s, or using a Windows PC

13. The Big Picture
Report on the results. Reports include coaching reports, item analysis, gap analysis, and others.

14. Applications in the Big Picture

15. Authoring Manager Desktop

16. Authoring Manager Desktop
Repository Bar

17. Authoring Manager Desktop
Navigation
Pane

18. Authoring Manager Desktop
Display
Pane

19. Authoring Manager Desktop
Preview/Properties
Pane

20. Highlighting Topics Topic folders allow you to:
Logically organize questions
Provide topic outcomes
Prescriptive feedback by topic
Set prerequisites at topic level for passing grades

21. Highlighting Assessment Folders
Assessment folders allow you to:
Logically organize assessments

22. Choices for Creating Questions
Question Wizard
Copying an existing question (copy/paste)
Question Editor
Importing from file:
ASCII
QML
IMS QTI

23. Importing from ASCII

24. Select the Question type

25. Question Wording

26. Enter Choices, Feedback, and Set Scores for Each Choice

27. Set Shuffling
guessing.

28. Item options

29. Importing Questions ASCII Import IMS QTI Qpacks
Save word processing files as a text file with tags to show “question”, “choices”, “right answer”, feedback”, etc
Sample tags: <Q> Question text or wording; <C+> Question choice, the “correct” one scores 1; <C> Question choice scores 0.
IMS QTI
Industry standard for saving questions in an XML format
Qpacks
Pre-made assessments bought, sold, and exchanged between publishers and users

30. Question Types Drag and Drop Essay Explanation Fill in Blank Hot Spot
Java (not via wizard)
Likert Scale
Macromedia Flash
Matching
Matrix
Multiple choice
Multiple response
Numeric
Pull-Down List
Ranking
RoboDemo
Select a Blank
Text Match
True/False
Wimba spoken voice recorder
Yes/No
Enable authors to create more interesting and probing questions
Reveals more information during an assessment in a shorter time
Not offered by simpler assessment tools in LMS’s

31. Multiple Choice Question Type
A Multiple Choice item allows only one choice to be selected from several.

32. True/False Question Type
A true/false item is a Multiple Choice question type with only true and false as the possible choices.

33. Multiple Choice Question Type -- Likert Scale
A Likert Scale item is a Multiple Choice question that surveys opinions.

34. Multiple Response Question Type
A Multiple Response item allows several choices to be selected.
Several scoring scenarios can be defined.

35. Numeric Question Type
A Numeric item allows only digits to be submitted.

36. Text Match Question Type
A single word or multiple words can be submitted.
Outcomes and scoring
Searches for keywords or absence of keywords
Checks for misspellings
Can look for multiple possibilities

37. Essay Question Type Long open-ended text answers
Manually graded through a browser

38. Select-a-Blank Question
See the answer in the context of the question.
Multiple pull-down boxes.

39. Fill in Blanks Question Type
See the answer in the context of the question.
Allow for misspelling or case-sensitivity

40. Pull-Down List Question Type
Allows selection from a pull down list of choices
Outcomes are defined by matching the choice with the selection

41. Matching Question Type
Drop-down lists
Forces only unique choices

42. Matrix Question Type Not recommended as challenging to report on
Allows selection from a matrix list of choices across the page
Similar to selection question but selections go across the screen

43. Ranking Question Type Allows numeric ordering or rating.
All ranks must be selected and none can be repeated.

44. Hotspot Question Type -- Single Marker
The choice is selected by virtue of its position.
This can be selected by using a mouse or touch screen.

45. Drag and Drop Question --Multiple Markers
Click and drag each marker to the correct area

46. Managing Likert Scales

47. Creating an Assessment

48. Assessment Manager Wizard -- Step 1

49. Assessment Manager Wizard -- Step 2

50. Assessment Manager Wizard -- Step 3

51. Assessment Manager Wizard -- Adding Questions

52. Assessment Manager Wizard -- Step 4

53. Email at end of assessment

54. Local repository
Store items/questions, assessments, content, etc., locally

55. Remote repository
Store items/questions, assessments, content, etc., remotely

56. Local & Remote repositories
Store items/questions, assessments, content, etc., remotely
Store items/questions, assessments, content, etc., locally

57. Copy and Paste Between Repositories

58. Ways to publish with v3

59. Manage remote repositories

60. Workflow

61. Managing Schedules
Add, import, change, and delete participants, groups and their schedules

62. Enterprise Manager

63. Enterprise Manager - administration

64. Enterprise Manager - Administration

65. Enterprise Manager - Administration
Providing admin rights to other users means you can share your successes with senior management!

66. System Administration

67. Scheduling a Participant

68. The Big Picture
Deliver assessment using a standard or secure browser, PDA’s, or using a Windows PC

69. Security Options in Perception
Open Access for Low Stakes
Secure access for High Stakes
User name and password validated:
By Perception’s database
By Portals (when integrated)
By Management Systems (when integrated)
By LDAP
Time windows for access
Monitored/Proctored or not
Questionmark Secure to prevent printing and task switching

70. Launching Assessments from other systems
Assessments can be launched and tracked by Perception itself or by management systems
Questionmark Perception is:
AICC certified
ADL SCORM compliant
.NET certified
QTI compliant

71. Blended Delivery Options
Assessments can be taken using:
Live via web browsers, for example:
Microsoft Internet Explorer, AOL, Netscape, etc.
Questionmark Secure
Asynchronously over the web or intranet (P2G)
Windows run-time
32 bit Windows PC
Printed (for later scanning)

72. Text, HTML mark up, and software components
Look and Feel is Managed by Templates - Merging Item/Questions with Templates at delivery time
Assessments
Questions
Graphics, Video
Resources
Item data in database
Merged
Templates
Text, HTML mark up, and software components

73. Look and Feel can also be Managed by Style Sheets
Assessments
Questions
Graphics, Video
Resources
Item data in database
Merged
Templates
Style Sheets
Text and software components

74. Questionmark Secure Points invisibly to the web site No menu or icons
Cannot switch to another program
Disables printing
Exit only with password

75. Questionmark Secure
Normal browser allows menus, printing, task switching, exit, etc.
Questionmark Secure doesn’t

76. Perception To Go A module that enables participants to: Ideal for:
Synchronize from their PCs to their web server to receive quizzes, tests and surveys
Take the assessment off-line
Synchronize the results back to the server
Ideal for:
Mobile users
Internet charges are priced by the minute and you have long assessments

77. Step 1 - Participant logs on to server

78. Step 2 - Participant syncs with server

79. Step 3 - Participant disconnects from server

80. Step 4 - Participant selects and answers assessment

81. Step 5 - Participant reconnects and syncs back answers

82. Integration Learning Management Systems
To link to an LMS:
Schedule a survey, test or quiz
Student logs into the LMS and clicks through to assessment
Perception delivers the assessment
Results passed back to the LMS
Reports available in Perception and/or the LMS

83. Current LMS Integrations
SumTotal
Ingenium
Aspen
Blackboard
DKSystems OnTrack
Docent
Generation21
GeoMetrix Data Systems
LearningSpace
Plateau
Pathlore
ReadyGo
Saba
vCampus
ViewCentral
WBT Manager
WebCT

84. The Big Picture
Report on the results. Reports include coaching reports, item analysis, gap analysis, and others.

85. Enterprise Reporter Run reports securely from any browser
Filtered data to retrieve information you need
Custom build your own reports
Open database access for system integration
Report on the results. Reports include coaching reports, item analysis, gap analysis, assessment overview, score list, transcript, grade book, survey, and question statistics.

86. Enterprise Reporter

87. Enterprise Reporter Enterprise Reporter enables you to:
Monitor participant progress
Measure overall training effectiveness
Identify trends
Improve future training programs
….More effectively than Crystal Reports or Excel

88. Enterprise Reporter – Sample Assessment Overview Report

89. Enterprise Reporter – Sample Assessment Overview Report
Enables us to understand assessment performance over a period of time

90. Enterprise Reporter – Sample Coaching Report

91. Enterprise Reporter – Sample Coaching Report
Shows information about a single result
Used for feedback purposes to participants
Print out as a permanent record

92. Enterprise Reporter – Sample Export to ASCII Report

93. Enterprise Reporter – Sample Export to ASCII Report
Download to:
SPSS
Excel

94. Enterprise Reporter – Sample Gap Report

95. Enterprise Reporter – Sample Gap Report
Compare results between groups
Highlights changes
Carry out Level 2 Analysis

96. Enterprise Reporter – Sample Grade Book Report

97. Enterprise Reporter – Sample Grade Book Report
Lists assessment scores that may contribute to an overall score

98. Enterprise Reporter – Sample Question Statistics Report

99. Enterprise Reporter – Sample Question Statistics Report
Allows you to:
Evaluate and improve the quality of your questions
Report at assessment or topic level
Analyze statistics for each question outcome

100. Enterprise Reporter – Sample Item Analysis Report

101. Enterprise Reporter – Sample Item Analysis Report
Similar to Question Statistics
Fixed format for professional test analysts

102. Enterprise Reporter – Sample Score List Report

103. Enterprise Reporter – Sample Score List Report
Produces a list of results for one assessment
Links to Coaching Report

104. Enterprise Reporter – Sample Survey Report
Click on blue bar
To filter report

105. Enterprise Reporter – Sample Survey Report
Uses frequency analysis graphs
Used for “Smile” sheets and Level 1 evaluation
Click on blue bar
To filter report

106. Enterprise Reporter – Transcript Report

107. Enterprise Reporter – Transcript Report
List of results for single participant
Links to Coaching Report
Can be viewed by participant

108. Survey Report

109. Coaching Report

110. Question Statistics Report

111. Item Analysis Report

112. Learning Opportunities
Download 30-day evaluation copy
Questionmark Communities
QM Perception knowledge base and manuals
Learning Cafe – including tutorials
2-day training course
About Perception
About creating great assessments
On-site training at your location

113. Questionmark Customer Assistance
Questionmark Customer Support
Commercial issues (quotes, licenses, etc.)
Phone:
Questionmark Technical Support
Technical issues
Phone:
Questionmark Consulting Services
Advice on deployment issues

114. Thank You for Your Attention
Please complete our level 1 assessment!
Technical Services and Support:
(800)