Evaluation Using Modeling

Testing Methods Same as Formative Surveys/questionnaires Interviews Observation Documentation Automatic data recording/tracking Artificial/controlled studies Heuristic Evaluation Cognitive Walkthrough Usability Study KSLM GOMS

Simple User Models Idea: If we can build a model of how a user “works”, then we can predict how s/he will interact with the interface/system –Predictive model  predictive evaluation No mock-ups or prototypes!

Two Types of User Modeling Stimulus-Response –Practice law –Hick’s law –Fitt’s law Cognitive – human as interperter/predictor – based on Model Human Processor (MHP) –Key-stroke Level Model Low-level, simple –GOMS (and similar) Models Higher-level (Goals, Operations, Methods, Selections)

Hick’s law Decision time to choose among n equally likely alternatives –T = I c log 2 (n+1) –I c ~ 150 msec How can we use this law?

Fitts’ Law Models movement times for selection (reaching) tasks in one dimension Basic idea: Movement time for a selection task –Increases as distance to target increases –Decreases as size of target increases

Movement time Run empirical tests to determine values for C & a Will get different values for different input devices and uses MT = C + a(log 2 (d/w + 1.0) width (tolerance) of target distance to move

Questions What do you do in 2D? –Rectangles, or circles? –For rectangles ID = log 2 (d/min(w, l) + 1) How can we use this law?

Power law of practice T n = T 1 n -a –T on the nth trial is T on the first trial times n to the power -a; a is about.4, between.2 and.6 –Skilled behavior - Stimulus-Response and routine But NOT learning How can we use this law?

Cognitive models

Keystroke-Level Model (KSLM) Developed by Card, Moran & Newell, see their book and CACM Models Skilled users performing routine tasks Assigns times to basic human operations - experimentally verified Based on MHP - Model Human Processor

Model Human Processor

KSLM Accounts for Keystroking T K Mouse button pressT B Pointing (typically with mouse) T P Hand movement between keyboard and mouse T H Drawing straight line segments T D “Mental preparation” T M System response time T R

14 KSLM Decompose task into sequence of operations - K, B, P, H, D, M, R Place M operators –In front of all K’s that are NOT part of argument strings (ie, not part of text or numbers) Example - select a word and type new text –Home on mouseH(mouse) –Point to wordP(word) –Select wordMBB(mouse button) –Home on keyboardH –Type new 5-letter wordM5K

15 KSLM Apply rules to example –Home on mouseH(mouse) –Point to wordP(word) –Select wordMBB(mouse button) –Home on keyboardH –Type new 5-letter wordM5K T = 5T K +2T B +T P +2T H +T M +T R

16 KSLM Plug in real numbers from experiments –K (Keystroking):.08 sec for best typists,.28 average, 1.2 if unfamiliar with keyboard –B (Button Press): down or up secs; click secs –P (Pointing): 1.1 secs –H (Homing/hand movement): 0.4 secs –M (mental preparation): 1.35 secs –R (System response): depends on system; often less than.05 secs T = 5T K +2T B +T P +2T H +T M +T R = 5(.28)+2(.2)+1.1+2(.4) = 5.1 secs

17 KSLM Example: Mouse for menu selection? What is the operator sequence? –H(mouse)PB(left-click)MPB(left-click) Complicated rules for placing M’s – but boils down to chunking (one M before each “chunk” of a task) Candidate M’s before each B, K, and P involved in specification or selection of a command; eliminate the M’s that are “fully anticipated” or in a “cognitive unit” Textbook timings (all in seconds) H = 0.40, P = 1.10, B = 0.20, M = 1.35 Total predicted time = 4.35 s

18 Problems/shortcomings of KSLM?