1. Evaluation Using Modeling

2. Testing Methods Same as Formative Surveys/questionnaires Interviews Observation Documentation Automatic data recording/tracking Artificial/controlled studies Heuristic Evaluation Cognitive Walkthrough Usability Study KSLM GOMS

3. Simple User Models Idea: If we can build a model of how a user “works”, then we can predict how s/he will interact with the interface/system –Predictive model  predictive evaluation No mock-ups or prototypes!

4. Two Types of User Modeling Stimulus-Response –Practice law –Hick’s law –Fitt’s law Cognitive – human as interperter/predictor – based on Model Human Processor (MHP) –Key-stroke Level Model Low-level, simple –GOMS (and similar) Models Higher-level (Goals, Operations, Methods, Selections)

5. Hick’s law Decision time to choose among n equally likely alternatives –T = I c log 2 (n+1) –I c ~ 150 msec How can we use this law?

6. Fitts’ Law Models movement times for selection (reaching) tasks in one dimension Basic idea: Movement time for a selection task –Increases as distance to target increases –Decreases as size of target increases

7. Movement time Run empirical tests to determine values for C & a Will get different values for different input devices and uses MT = C + a(log 2 (d/w + 1.0) width (tolerance) of target distance to move

8. Questions What do you do in 2D? –Rectangles, or circles? –For rectangles ID = log 2 (d/min(w, l) + 1) How can we use this law?

9. Power law of practice T n = T 1 n -a –T on the nth trial is T on the first trial times n to the power -a; a is about.4, between.2 and.6 –Skilled behavior - Stimulus-Response and routine But NOT learning How can we use this law?

10. Cognitive models

11. Keystroke-Level Model (KSLM) Developed by Card, Moran & Newell, see their book and CACM Models Skilled users performing routine tasks Assigns times to basic human operations - experimentally verified Based on MHP - Model Human Processor

12. Model Human Processor

13. KSLM Accounts for Keystroking T K Mouse button pressT B Pointing (typically with mouse) T P Hand movement between keyboard and mouse T H Drawing straight line segments T D “Mental preparation” T M System response time T R

14. 14 KSLM Decompose task into sequence of operations - K, B, P, H, D, M, R Place M operators –In front of all K’s that are NOT part of argument strings (ie, not part of text or numbers) Example - select a word and type new text –Home on mouseH(mouse) –Point to wordP(word) –Select wordMBB(mouse button) –Home on keyboardH –Type new 5-letter wordM5K

15. 15 KSLM Apply rules to example –Home on mouseH(mouse) –Point to wordP(word) –Select wordMBB(mouse button) –Home on keyboardH –Type new 5-letter wordM5K T = 5T K +2T B +T P +2T H +T M +T R

16. 16 KSLM Plug in real numbers from experiments –K (Keystroking):.08 sec for best typists,.28 average, 1.2 if unfamiliar with keyboard –B (Button Press): down or up - 0.1 secs; click - 0.2 secs –P (Pointing): 1.1 secs –H (Homing/hand movement): 0.4 secs –M (mental preparation): 1.35 secs –R (System response): depends on system; often less than.05 secs T = 5T K +2T B +T P +2T H +T M +T R = 5(.28)+2(.2)+1.1+2(.4)+1.35 +.05 = 5.1 secs

17. 17 KSLM Example: Mouse for menu selection? What is the operator sequence? –H(mouse)PB(left-click)MPB(left-click) Complicated rules for placing M’s – but boils down to chunking (one M before each “chunk” of a task) Candidate M’s before each B, K, and P involved in specification or selection of a command; eliminate the M’s that are “fully anticipated” or in a “cognitive unit” Textbook timings (all in seconds) H = 0.40, P = 1.10, B = 0.20, M = 1.35 Total predicted time = 4.35 s

18. 18 Problems/shortcomings of KSLM?