Exploring Individual Variability Using ACT-R Christian Schunn George Mason University

Understanding Variability in Performance u Within and between subjects variability are important sources of information (beyond average performance) –Differentiate models –Indicate whether average is representative of any individual's behavior (both within and between subjects)

Sources of Variability: Naïve View u Random sampling with fixed probabilities –e.g., most mathematical models of memory –e.g., most mathematical models of choice

ACT-R view u “Noise” (variations on Naïve) –Expected gain noise –Activation noise (encoding and transient) –Perceptual noise u Parametric variation –Global architecture differences Ability (W, d) & motivational differences (G) –Experiential differences (expertise & luck) e.g., q&r, a&b, activation, strength, a ij, etc u Knowledge variation (expertise & luck) –Productions & Chunks

An example: Are there individual differences in adaptivity? u On average, people select choices according to base-rates of success –e.g., Probability matching u On average, people adapt (or change) strategies when base-rates change –Reder 82, Siegler 87, Lovett & Anderson 96 u Do people systematically differ in how much or how fast they adapt? –Also, is average meaningful?

A BST Experiment

Experiment Details Block = 10 trials

Aggregate performance

A mathematical model of aggregate performance r 2 =.93 (zero parameters) }

Variation in Adaptivity u Suggests more than just noise?

ACT-R BST model u Adapted from Lovett (1998) –Same productions & parameter values u Force-over, force-under, then finish task, retry with other strategy if can't solve u Can ACT-R provide better fit and more insights than the Naïve Monte Carlo simulation?

Fit to Data (default) u Luck/experiential differences plus noise u Less variability than humans r 2 =.77, RMS=.06

Sensitivity Analyses u What influences mean and variance in adaptivity? –EGS settings –Motivational levels –Learning –Prior experiences u Each model uses default settings and tweaks one feature

Noise settings (EGS) u Even at better settings, variability is low.84,.08.77,.06.72,.09

Motivation settings (G) u At high G settings, variability goes down.84,.11.77,.06.67,.13

Learning decay (d) u Decay makes model too sensitive u And variability still too low.75,.13.77,.06

Prior experiences u Apparently subjects have been playing BST previously? u Can get greater variability, but fit to mean becomes worse..78,.25.77,.06

Relationship to awareness data u Suggests more than just noise u ACT-R fits unaware best?

Conclusions/Questions u ACT-R gives more persuasive exploration of chance variability u Variability and mean affected differently –EGS, G affect means but not variablity levels For adaptivity only! (both affect block variability) –Amount of prior experiences affects both u Watch out for individual differences: –Evidence for parameter learning decay just a mixture of aware and unaware?