1. 12/6/2018 8:38:35 AM
An IMPRINT Model of the Digit Data Entry Task MURI Annual Meeting 9/27/06
Carolyn Buck-Gengler and William Raymond
Department of Psychology and
Center for Research on Training
University of Colorado at Boulder

2. 12/6/2018 8:38:35 AM
A basic data entry experiment: Healy, Kole, Buck-Gengler, & Bourne (2004)
10 blocks of 64 #s
Numeral format only
NO intertrial pause
Keypad only
Exp 1 (modeled in ACT-R)
Exp 2 (modeled in IMPRINT)
Repetition of numbers
each # repeated 5 times (different sets each half)
all unique #s
Use of non-dominant (left) hand
Left hand only
Both hands used
(R-R, R-L, L-R, L-L)
Break between halves
0s, 60s, 300s, self-paced
300s
Description of the study and the particular experiment we are currently modeling-- Coty is modeling Experiment 1; we are starting with Experiment 2 but plan to go to Experiment 1 after modeling all aspects of Experiment 2 needed to compare with Experiment 1
Describing both to highlight some of the manipulations that differ between the two
Fatigue: Two aspects of the basic DDE experiment were changed or manipulated in this experiment to try to induce fatigue:
1) No intertrial pause: Earlier experiments had an intertrial pause of 500 ms. In these experiment there was no intertrial pause
2) use of left (non-dominant) hand -- manipulated in E2; only left hand in E1. Earlier experiments used right (dominant) hand
Experiment 1: each of the 64 numbers was repeated 5 times; different set sof 64 numbers were used in each half
December 6, 2018

3. Results from Healy et al. (2004, Experiment 2) to be fit
12/6/2018 8:38:35 AM
Results from Healy et al. (2004, Experiment 2) to be fit
Speed improvement
Total RT (TRT) decreases across first half
RTs increase across second half (for some keystrokes)
Digit entry times: Keystroke 3 is slower than keystrokes 2 or 4 (and Enter Key)
Effect of hand
Left hand slower than right hand
Accuracy decline
Speed/accuracy tradeoff - errors increase across blocks
Types of errors (by output length)
We usually refer to K3 slower than K2, K4, enter as “chunking”
Also looking at serial position of errors, RT for error trials
December 6, 2018

4. access representation
12/6/2018 8:38:35 AM
A cognitive model of digit data entry 
1395
Language/reading
Symbolic representation
Motor planning
Fine motor manipulation
read digit
represent digit
access representation
plan response
access plan
execute plan 
1395
December 6, 2018

5. Modeling Approach Initial goal: fit results from this experiment
12/6/2018 8:38:35 AM
Modeling Approach
Initial goal: fit results from this experiment
i.e., effects of skill practice, hand manipulation
Both speed and accuracy effects for all response measures
Add one manipulation at a time to expand the model
From here move to other similar experiments
At each step, verify previous measures still replicated
Model the cognitive processes underlying the task
Decompose task into cognitive components (mappable to MURI task taxons)
Model individual variation (not aggregate data)
Granularity of processing is digit (not trial or block)
Probabilistic simulation
We started modeling with this experiment because it has the fewest manipulations -- model the very basic task
Next three slides will expand on these three points:
• high level summary of the results to be fitted by the model
• sequence of steps in expanding the model
• cognitive model of the task
December 6, 2018

6. Speed/Accuracy tradeoff
12/6/2018 8:38:35 AM
Modeling Sequence
Right hand only,
first 5 blocks,
No repetition
Speed/accuracy
tradeoff
Other
manipulations
predictions
Add Left hand,
first 5 blocks,
No repetition
Effect of hand
on speed
Left hand only,
Repetition
Repetitious
practice
ALL experiments currently modeled or in immediate future involve keypad only (typing location) and numeral presentation only. Perhaps further out we will look at row (typing location) and word/other presentation formats.
First effort: just RTs: Speed improvement due to skill practice only
Adding errors: Speed/accuracy tradeoff
Possible predictions:
R hand with repetition (keypad, numeral presentation)
Other manipulations:
Typing Location
Feedback
Presentation format (word/ math problems, etc)
Add 2nd half
No repetition
Speed/Accuracy tradeoff
and effect of hands
over longer time
December 6, 2018

7. The IMPRINT model Main network: the “computer”
12/6/2018 8:38:35 AM
The IMPRINT model
Main network: the “computer”
Bookkeeping
Set up subject-specific parameters and variables
“Present” each trial
Goal network: the “subject”
Set up trial-specific parameters and variables
Main subnetwork: one trial
Components (“tasks”) for each cognitive process that goes on in a trial
In Imprint, There are two networks running in parallel. One represents the “computer” presenting the data -- which really is used for bookkeeping at the experimental and subject level, and to keep the simulation running. Each “run” of the IMPRINT model corresponds to a unique subject. The other represents the subject. It is only the lowest level of the subject network that is of primary interest today. This network represents the simulation of ONE trial. It is executed over and over again to simulate an entire experimental run of one subject, and that larger network is, in turn, run over and over again to simulate multiple subjects. Subject characteristics, such as an average speed or hand used, are set at the outermost (experiment) level.
The basic unit in IMPRINT is the “task”. This sometimes corresponds 1 to 1 with the idea of component, and sometimes “component” encompasses more than one task. The term component will be used for both.
December 6, 2018

8. “Subject” Network: Do one trial
12/6/2018 8:38:35 AM
“Subject” Network: Do one trial
read
represent
access rep
plan resp
access plan
execute plan
read
plan motor
This network shows the embodiment of the cognitive model into the IMPRINT network.
At a high level, one trial of DDE involves the following:
** read/“represent” the 4-digit number,
** plan the motor response, and
** type it, followed by typing the enter key to end the trial.
In our model, we have broken these three steps into 2 or more tasks each, and each subset of tasks is executed for each digit (the network loops over the pair once for each digit) or chunk of digits in the typing part. A separate task is placed in the network for the enter key.
Flow -- based on “typical” trial with chunking into two 2-digit chunks:
** First, the trial is initialized in Initialize Trial (setting various counters to 0, updating the trial counters, determining error status of trial, and determining response times for each component).
** Next are the two subcomponents of reading a digit, ** Read 1 digit and ** Represent one digit. ** This pair is looped 4 times, as the enter key does not need to be read.
** After reading, the motor task is accessed for each digit. This is also broken down into two tasks, ** Access representation of 1 keystroke and ** Plan Motor response for 1 keystroke. This pair of tasks is looped 4 times.
** Finally, the digits are typed. Based on data collected in many experiments, we have assumed chunking, but have designed the network to allow for chunks of different sizes. A chunk size of two is the basic assumption. In general, the two steps of typing a key are ** Access Motor Plan for 1 chunk (executed once for each chunk, which is 4/chunksize) and ** Execute 1 keystroke, which is executed once for each digit. Type a chunk and Keystroke gatekeeper are there for bookkeeping and logic purposes. ** A final task Plan and Execute Enter embodies typing the enter key, followed by the computer/dummy task Cleanup after trial.
At this point, the “goal” is finished, and control returns to the computer’s network, which then presents the next trial.
type
December 6, 2018

9. Mapping tasks to keystrokes
12/6/2018 8:38:35 AM
Mapping tasks to keystrokes
Init (K1)
The time response data from the DDE experiments are as follows: Init time (= time since stimulus onset at which the first digit is typed), times for the 2nd, 3rd, and 4th digits (K2, K3, K4), and the time to the Enter key. [In some versions of the experiment, times for any keys beyond the 4th are also collected.]
** Init time is usually very long in comparison to the other digit typing times;
** K2 and K4 are very similar in time and shortest, and
** K3 is somewhat longer.
** Enter key time is also relatively very short.
It is assumed theoretically that all the reading and most of the encoding and motor planning is captured in the Init time, with some small amount of planning/re-accessing done before the 3rd keystroke is executed (and thus being captured in the time for K3), leading to the chunking concept, that the four digits are grouped into smaller chunks of two digits for the purpose of remembering or entering them better. Thus, the times for all aspects of reading/encoding the digits and accessing the motor plan, as well as typing the first digit, must be included in the init time, and the extra time for K3 must be expressed in the network as something that only happens for the 3rd digit (and 1st) and not for the 2nd and 4th digits (nor the enter key). Thus, in the model, any tasks outside of actual typing for reading/planning/encoding/accessing, except those that could also contribute to K3, must take place before the first time the Execute 1 keystroke task is encountered. K3
K2,K4
Enter
December 6, 2018

10. Implementing skill learning (RTs)
12/6/2018 8:38:35 AM
Implementing skill learning (RTs)
Main concept to model: Learning of the skill of typing 4-digit numbers, without repetition
Two types of skill learning:
Cognitive: Applying only to K1 and K3; independent of hand – applied in Access motor plan for 1 chunk
Physical: Applying only to Execute 1 keystroke and Plan and execute Enter; more learning possible for L hand than for R hand
Using power function
Assumption: Learning occurs only on correct trials
Here we are talking only about the response times, not accuracy, and the improvement of response times (learning) over time due only to skill/practice and not to repetition of numbers.
The Model does not actually LEARN. Learning is SIMULATED by using a multiplier derived from an exponent, a parameter set in the model, applied to a count of trials, either good trials so far for cognitive learning, or all trials so far for physical learning. Each type has its own exponent, and when the left hand is used, the physical learning exponent is adjusted to allow for more learning for the left hand than the right hand.
More learning for L than for R based on data
Power function: decreases, approaches asymptote, assumed for human learning
Assumption: Learning occurs only on correct trials (exponent applied only to a count of correct trials so far)
Cognitive learning occurs only on correct trials (exponent applied only to a count of correct trials so far)
Physical learning occurs on all trials
December 6, 2018

11. Caveat Not yet modeling fatigue
12/6/2018 8:38:35 AM
Caveat
Not yet modeling fatigue
Seems to counteract learning – at least for K1 – in second half
Modulated by repetition as well, which will be addressed when we model Experiment 1
Role of fatigue in the first half, if it exists, is swamped by learning --> only evident in second half
December 6, 2018

12. RT Assumptions Subjects’ TRTs from normal distribution
12/6/2018 8:38:35 AM
RT Assumptions
Subjects’ TRTs from normal distribution
A TRT chosen for each, then used as the base for each of the trials for that subject
A specific set of proportions of TRT assigned to each task
Independence of tasks – time for each drawn separately from a skewed distribution around the proportion of the total time allotted to that task
Penalty for the Left hand on the execute keystroke tasks (Execute 1 keystroke and Plan and execute Enter)
These assumptions are BASED ON published results and other analyses of experimental data
December 6, 2018

13. Total Response Time: Overall TRT
12/6/2018 8:38:35 AM
Total Response Time: Overall TRT
RESULTS compared to model, overall
Overall Improvement across session halves
Overall improvement across blocks in first half but not second
-- 2nd half part not yet modeled, because we don’t have something for “fatigue”
Note: whether overall mean increases or decreases between blocks 5 and 6 depends on how individual Ss randomly assigned to Hand and Switch groups
Improvement across session halves – depended on combination of hands used in each half – could be explained by L being slower than R
LR best improvement between halves
RL got worse
LL and RR intermediate improvement
However, improvement of no switch and switch groups similar, so improvement was cognitive, not just motoric
improvement of LL+RR similar to RL+LR, so improvement was cognitive, not motoric
Always faster on L than on R
Improvement is cognitive
December 6, 2018

14. Total Response Time by groups
12/6/2018 8:38:35 AM
Total Response Time by groups
Improvement across session halves – depended on combination of hands used in each half – could be explained by L being slower than R
LR best improvement between halves
RL got worse
LL and RR intermediate improvement
However, improvement of no switch and switch groups similar, so improvement was cognitive, not just motoric
improvement of LL+RR similar to RL+LR, so improvement was cognitive, not motoric
Always faster on R than on L
Improvement is cognitive
Notes:
Solid lines = Experimental data, dashed = model output
Each color represents a different hand/switch combination
December 6, 2018

15. 12/6/2018 8:38:35 AM
All keystrokes
December 6, 2018

16. Init (K1) Init (K1): decrease in first half, increase in second half
12/6/2018 8:38:35 AM
Init (K1)
Init (K1): decrease in first half, increase in second half
Cognitive learning; offset by fatigue in second half
Hand not a factor
Model: decrease in first half, fatigue not modeled yet so don’t have increase in second;
Relative position of data due to our statistical Ss being drawn from a similar “population” but not being identical with the experimental Ss
December 6, 2018

17. Execution time (average of K2, K3, K4): Overall RTs
12/6/2018 8:38:35 AM
Execution time (average of K2, K3, K4): Overall RTs
First, overall:
Looks like 2nd half lower than first, less change within block (just as with experiment data)
December 6, 2018

18. Execution time (average of K2, K3, K4): by groups
12/6/2018 8:38:35 AM
Execution time (average of K2, K3, K4): by groups
Improvement across session halves – depended on combination of hands used in each half – could be explained by L being slower than R
LR best improvement between halves
RL got worse
LL and RR intermediate improvement
However, improvement of no switch and switch groups similar, so improvement was cognitive, not just motoric
improvement of LL+RR similar to RL+LR, so improvement was cognitive, not motoric
Always faster on R than on L
Improvement is cognitive
Notes:
Solid lines = Experimental data, dashed = model output
Each color represents a different hand/switch combination
December 6, 2018

19. Conclusion time (Enter key)
12/6/2018 8:38:35 AM
Conclusion time (Enter key)
Model maybe not declining as much as experimental data in second half
December 6, 2018

20. Accuracy Overall accuracy: is trial correct or not
12/6/2018 8:38:35 AM
Accuracy
Overall accuracy: is trial correct or not
Output length analysis: how many digits typed in error trial?
Missed trial: responded with Enter key only
Missed digit: typed fewer than 4 digits
Wrong digit: typed 4 digits, but at least one wrong
Extra digit: typed more than 4 digits
Also will be looking at Location of error(s) (serial position)
Are also splitting errors into Planning vs. execution errors, per Bill Raymond’s separate analysis
December 6, 2018

21. Accuracy (errors): how modeled
12/6/2018 8:38:35 AM
Accuracy (errors): how modeled
At beginning of trial network, trial randomly assigned a status of either “correct” or “incorrect”
Proportional to data
Output length randomly selected from distribution
Gives missed trial, missed digit, wrong digit, and extra digit trials
Technical details of modeling
Chance of randomly being assigned to “incorrect” grows with block number to model decrease in accuracy over the course of the experiment.
Reflecting data from various experiments, the distribution of output length ensures most error trials have length 4; of the remainder, most are 3 or 5, and very very few are 0, 1, 2, or 6 or more. Errors are distributed between lengths 0 and 8.
December 6, 2018

22. Accuracy – overall Overall accuracy:
12/6/2018 8:38:35 AM
Accuracy – overall
Experiment Results:
Overall accuracy:
- Errors increased from first to second half (directional test)
• Errors modeled as increasing so that error rate increases overall when second half modeled
- Error rate increase did not depend on which hand was in use
• Must be some cognitive component, rather than limited to motoric processes
• Comparison to similar analysis in E1 suggests that motoric processes may also still be involved (Motoric also involved because increase in errors between halves significant only by directional test and because no effect of block )
Model follows data fairly well
December 6, 2018

23. Accuracy – output length
12/6/2018 8:38:35 AM
Accuracy – output length
RESULTS
Output length:
- Missed trial and missed digit trials very few, and not increasing across blocks:
• Subjects did not type fewer digits as the experiment progressed and any fatigue increased
MODEL
Pretty good match
Red: data from experiment
Blue: model
December 6, 2018

24. Insights gained in modeling process
12/6/2018 8:38:35 AM
Insights gained in modeling process
Where learning occurs
Breaking it up into physical and cognitive components, and where to put it
Hands not different, non-dominant just slower
More complex understanding of the chunking process
Very little skill learning?
December 6, 2018

25. Summary Modeled fairly well Taken into account
12/6/2018 8:38:35 AM
Summary
Modeled fairly well
RTs, both TRT and keystrokes, also chunking pattern
With exception of effects of fatigue
Accuracy findings
Taken into account
Learning/practice, both cognitive and physical
Hand and hand switch
December 6, 2018

26. Next on modeling agenda
12/6/2018 8:38:35 AM
Next on modeling agenda
Fatigue
Buildup, especially with no break and no change of hand
Dissipation: Effects of break and different length breaks
Relationship with repetition (compare E1, E2)
Learning through Repetition (Healy et al., 2004, E1)
Test (E1) (repetition priming/item transfer)
Feedback (Kole et al., 2006)
Presentation format, etc. (other experiments)
December 6, 2018

27. 12/6/2018 8:38:35 AM
Acknowledgments
Personnel at MA&D for IMPRINT assistance, access, and facilitation
Especially Bob Sargent
and Ron Laughery
December 6, 2018