1. Christine Fry and Alex Park March 30th, 2004
A Statistical Approach to Predicting Switching Behavior from Reward Histories
Christine Fry and Alex Park
March 30th, 2004

2. The Experiment Subjects chooses between 2 buttons
Immediately see result of choice
Accumulate fractions of Hershey’s kisses on each experiment
3 tasks with different reward curves
240 trials (choices) for each task
Pilot vs. main experiment — motivation

3. Reward Curve Reward Allocation to A
Reward given depends on allocation A choices over last 40 presses
Middle is local optimum based on matching

4. Modeling User Behavior
After seeing reward at time t, subject can choose to
press same button (st=0) (NO SWITCH)
press other button (st=1) (SWITCH)
Action
Reward
Switch Decision
Assuming users are rational (big assumption)
We can model switching decision with information that subject acts on
What factors are important for decision making?

5. Factors involved in making switching decision
Possible factors involved in deciding
Change in most recent reward
Average change over last n rewards
Amount of reward accumulated so far
Amount of time elapsed in experiment
Each of the above is a linear function of a history vector
where the function is a weighted sum of the inputs

6. Training weights by discriminant analysis
Predicting switching decisions based on history vectors boils down to 2-way classification of vectors xt.
Classification by linear discriminant analysis
Separate xt into two classes (switch or stay)
Compute weight vector, w, which best separates two classes
Use w and decision boundary to determine relative probabilities of switching for a given test input
Switch Stay
Switch Stay
P(x|Sw)
P(Sw|x) =
P(x|St) + P(x|Sw)
Probability of
switching for
projected input, x
Example set of
history vectors
Distributions of
projected points

7. Evaluation paradigm 1) Final allocation resting point
Train weight vector on first 120 trials, play game for the next 120 actions
Train weight vector on first 120 trials, classify next 120 history vectors
# Trials
The above tasks can be evaluated as follows
1) Final allocation resting point
Predicted
Actual
Predicted
Switches
Actual
2) Error rate of switch classifications
Errors

8. Results
Evaluated on set of data from 6 users, using 3 different reward profiles
For classification, deterministic assignment used.
Reward history vector extends 5 observations into the past
Method 1
Method 2
Method 3
Overall final allocation prediction error** (1 trial each): 0.12
Overall switch classification error rate:
No Error