A Wizard for PADI Assessment Design AERA April 2005 A Wizard for PADI Assessment Design Larry Hamel, CodeGuild, Inc. Patricia Schank, SRI International

2 What is a Wizard? An interview that gathers information from users through a series of simple questions E.g., Tax forms via Intuit’s TurboTax

3 PADI Wizard Goals Easy assessment design, based on expertly-crafted foundations: scaling up the number of designers who use PADI Good interaction for interviewees (designers) Allow users to stop/restart, go back & forth Status of completion available at a glance Integrate well with PADI interface Wizard-creation system for experts Add or edit wizard steps Reorder steps Adaptive, dynamic wizard (phase 2)

1st Design: Editing Wizard 4 1st Design: Editing Wizard Provides context, but fairly complex Costly to implement: start simpler?

2nd Design: Sequential Wizard 5 2nd Design: Sequential Wizard Simple question-answer interview But how to handle constraints of early criteria that cascade to rest of interview? E.g.: If trying to measure proficiency in “planning investigation”, must include activity for it.

3rd Design: Selection Wizard 6 3rd Design: Selection Wizard User’s criteria selects preconfigured templates (one per combination of criteria) Selected templates can be arbitrarily complex Example: Three criteria What is the goal of the assessment? (2 choices) Grade level of the students? (3 ranges) What is the content area? (6 choices) Yields 2x3x6 combinations: 36 templates, each of which is customized with appropriate constraints that cascade from user’s answers

Example: Running a Wizard 7 Example: Running a Wizard Begin with a welcome page Next, exit buttons to navigate

Running a Wizard (cont.) 8 Running a Wizard (cont.) Example criteria question Ask about goal of assessment, explain choices and offer menu selection

Running a Wizard (cont.) 9 Running a Wizard (cont.) Finishing wizard If a template matching the criteria is found, it is cloned so user can customize it further

Wizard Creation Constructing a wizard is itself an interview 10 Wizard Creation Constructing a wizard is itself an interview Add and specify steps, decision matrix

Discussion Many lessons learned (see paper), e.g. Future work 11 Discussion Many lessons learned (see paper), e.g. Abstract way to define steps Tension between simplicity and detail Many combinations in Selection wizard Future work Revisit priority for dynamic, adaptive steps Completion wizard Fill in the template identified by the Selection wizard. Selection wizard handles the cascading constraints first; the rest is easy (?). Under development—more to come!