Product Development & User Testing Dr. Minjuan Wang Department of Educational Technology, SDSU belleminjuan@gmail.com

Somewhat time and labor intensive Been accused of being too linear It has been criticized for being time and labor intensive, and too linear. A more flexible ADDIE model?

The Successive Approximation Model: An agile design method Evaluate Design Develop Start End We’re going to look more closely at the successive approximation model, which has been getting more popular in recent years. It is considered a more iterative process that emphasizes prototyping more heavily than the other models.

Summative Evaluation Traditional Instructional Design Models Linear Tasks performed in sequential fashion Summative evaluation (ADDIE) Long development phases Time consuming and expensive to make changes

Formative Evaluation Simultaneous design and development Ongoing feedback from end-users Iterative design cycle User community Design community

Iterative Design concept prototype testing product revisions Formative Evaluation concept prototype testing product revisions

Designer Blindness

Prototype Testing Main Entry: pro·to·type 1 : an original model on which something is patterned : ARCHETYPE 3 : a standard or typical example 4 : a first full-scale and usually functional form of a new type or design of a construction (an airplane) http://www.m-w.com/cgi-bin/dictionary

Rapid Prototyping Early stages does the product meet the users' needs and expectations? Start with a crude prototype and refine Prototypes may be discarded rather than carried forward as a kernel for further development. Increased “editing power” makes it easier to alter designs; less need for elaborate specifications. requires good flow of testing data.

Formative Evaluation may… Be done very early in the design process when ~10% of project resources have been expended Provide solid measurements of task performance Help designers empathize with user’s needs and context Uncover problems help developers decide when the project can move on to the next stage that were not noticed in the design phase

Types of Prototypes Low fidelity Mid fidelity High fidelity paper and pencil sketching; rapid cut-and-paste. Mid fidelity screen printout. High fidelity working example, similar in appearance and functionality to the proposed final product.

When to Test? user testing assess needs define goals generate specs Create a product Identify a piece You may combine Alpha and Beta Test in 572 Fully develop it alpha test beta test (field test)

TIME Formative Evaluation Summative Evaluation design freeze needs Design Gel design freeze needs specs prototype Formative Evaluation Summative Evaluation TIME

Instructional Effectiveness What to test Instructional Effectiveness Is this proposed design effective in meeting the instructional goals? Comprehension Retention Transfer

Test on Usability Nielsen (1993) defines usability with the use of measurable components so that usability can be systematically be evaluated and improved. Learnability: How easy it is to learn to use the system when referring to a novice user’s experience at the initial part of the learning curve. Efficiency of Use: This attribute refers to expert users and the level of their performance. Memorability: How easy the system is to remember when casual users return to it in infrequent periods of time. Errors: How often do users make errors while using the system, how serious are these errors, and how do users recover from these errors Satisfaction: How pleasant the system is according to the the individual satisfaction of each user. Learnability Memorability Errors Satisfaction time on task Efficiency of Use

Formative Evaluation Questions Are the instructional activities clearly structured? Is the content perceived as useful and relevant by the learner? Are the practice exercises easy to understand and complete? Is your delivery system well organized (web navigation, blackboard setup, moodle layout) easy to use? Is the instructor guide easy to follow?

Abbreviated Formative Evaluation Focus on potential weaknesses in your design Formulate formative evaluation questions probe for and evaluate potential weaknesses What are you uncertain about? Me, how to teach statistics. Is it better to have them do draw a chart by hand; or it is better to work with software? Which concept to present first?

How many test subjects? A correlational graph. The more you test, the more problems you’ll detect. Since 572 testing is rather informal, you decide how many to test. The more the merrier. http://www.useit.com/papers/heuristic/heuristic_evaluation.html

How To Test? Here is one approach: A few at a time Revise Test it again Test more if you wish Your time is the limit!!! You can also do one big testing session (with all participants) I am sure reality will help you decide

T3R: Test, Tweak, Test, Report Here is the rigorous approach. You can only just do one test. As long as your goal is accomplished (you found answers to your questions). Make recommendations based on your tests

Thank You From crystaljingsr Flickr-CC 21