1. PSY 626: Bayesian Statistics for Psychological Science
4/13/2018
Binomial regression
Greg Francis
PSY 626: Bayesian Statistics for Psychological Science
Fall 2016
Purdue University
PSY200 Cognitive Psychology

2. Zenner Cards
Guess which card appears next:

3. Zenner Cards
Guess which card appears next:

4. Zenner Cards
Guess which card appears next:

5. Data Score indicates whether you predicted correctly (1) or not (0)
File ZennerCards.csv contains the data for 22 participants

6. Loading data # load full data file
ZCdata<-read.csv(file="ZennerCards.csv",header=TRUE,stringsAsFactors=FALSE)
# load the rethinking library
library(rethinking)
# Dummy variables to indicate actual card type and guessed card type
ZCdata$CardIndex <- coerce_index(ZCdata$ActualCard)
ZCdata$SelectedCardIndex <- coerce_index(ZCdata$GuessedCard)

7. Binomial model
yi is number of observed outcomes (e.g., correct responses) from n draws when the probability of a correct response is pi
We know n for any trial is 1
We estimate pi
It is convenient to actually estimate the logit of pi

8. Binomial regression We will model the logit as a linear equation
Among other things, this insures that our pi value is always 0 and 1 (as all probabilities must be)
Makes it easier to identify priors
Our MAP estimate will gives us logit(pi), to get back pi we have to invert

9. Model set up # All cards and subjects treated the same
ZCmodel1 <- map(
alist( Score ~ dbinom(1, p),
logit(p) <- a,
a ~ dnorm(0, 10)
), data= ZCdata )

10. Model results Maximum a posteriori (MAP) model fit Formula:
Score ~ dbinom(1, p)
logit(p) <- a
a ~ dnorm(0, 10)
MAP values: a
Log-likelihood: a is logit(p)
a=coef(ZCmodel1)["a”]
cat("Probability of correct response: ", logistic(a))
Probability of correct response:

11. Model results > precis(ZCmodel1) Mean StdDev 5.5% 94.5%
> logistic(-1.43)
[1]
> logistic(-1.2)
[1]

12. Differences across cards
# Different probabilities for different actual cards
ZCmodel2 <- map(
alist( Score ~ dbinom(1, p),
logit(p) <- a[CardIndex],
a[CardIndex] ~ dnorm(0, 10)
), data= ZCdata )
print(precis(ZCmodel2, depth=2))
aValues2<-c()
for(p in unique(ZCdata$CardIndex)){
code<-sprintf("a[%d]", p)
aValues2<- c(aValues2, coef(ZCmodel2)[code]) }
cat("Probability of correct response: ", logistic(aValues2))

13. Differences across cards
Maximum a posteriori (MAP) model fit
Formula:
Score ~ dbinom(1, p)
logit(p) <- a[CardIndex]
a[CardIndex] ~ dnorm(0, 10)
MAP values:
a[1] a[2] a[3] a[4] a[5]
Log-likelihood:
>
> plot(coeftab(ZCmodel2))
Probability of correct response:

14. Differences across cards
compare(ZCmodel1, ZCmodel2)
WAIC pWAIC dWAIC weight SE dSE
ZCmodel NA
ZCmodel
A model that uses different probabilities for different cards is expected to do better than a model that ignores card type.
Does this suggest that people have some kind of predictive power?

15. Differences in guesses
It might be better to see if participant’s guesses improve model fit
# Different probabilities for different selected cards
ZCmodel4 <- map(
alist( Score ~ dbinom(1, p),
logit(p) <- a[SelectedCardIndex],
a[SelectedCardIndex] ~ dnorm(0, 10)
), data= ZCdata )
Maximum a posteriori (MAP) model fit
Formula:
Score ~ dbinom(1, p)
logit(p) <- a[SelectedCardIndex]
MAP values:
a[1] a[2] a[3] a[4] a[5]
Log-likelihood:
Probability of correct response:

16. Null model Put a tight prior on a to be around logit(0.2)
ZCmodel0 <- map(
alist( Score ~ dbinom(1, p),
logit(p) <- a,
a ~ dnorm(logit(0.2), 0.001)
), data= ZCdata )
Maximum a posteriori (MAP) model fit
Formula:
Score ~ dbinom(1, p)
logit(p) <- a
MAP values: a
Log-likelihood: p=

17. Comparing models All models
> compare(ZCmodel0, ZCmodel1, ZCmodel2, ZCmodel4)
WAIC pWAIC dWAIC weight SE dSE
ZCmodel NA
ZCmodel
ZCmodel
ZCmodel
Model with different probabilities for different cards slightly beats null model
Indicates some response bias for some cards over other cards

18. Comparing models Among models that might indicate prediction ability
> compare(ZCmodel0, ZCmodel1, ZCmodel4)
WAIC pWAIC dWAIC weight SE dSE
ZCmodel NA
ZCmodel
ZCmodel
Null model expected to best predict future data
Its constraints (lack of flexibility) do not hurt much
Flexibility of other models are expected to fit noise in the data
Which hurts model prediction

19. Participants? Different probabilities for each participant
Probability of correct response:
Different probabilities for each participant
ZCmodel3 <- map(
alist( Score ~ dbinom(1, p),
logit(p) <- a[Participant],
a[Participant] ~ dnorm(0, 10)
), data= ZCdata )
Maximum a posteriori (MAP) model fit
Formula:
Score ~ dbinom(1, p)
logit(p) <- a[Participant]
MAP values:
a[1] a[2] a[3] a[4] a[5] a[6] a[7] a[8] a[9] a[10] a[11] a[12] a[13]
a[14] a[15] a[16] a[17] a[18] a[19] a[20] a[21] a[22]
Log-likelihood:

20. Comparing models > compare(ZCmodel0, ZCmodel1, ZCmodel4, ZCmodel3)
WAIC pWAIC dWAIC weight SE dSE
ZCmodel NA
ZCmodel
ZCmodel
ZCmodel
Little support for individual differences (at least relative to the other models)

21. Participants and cards?
ZCmodel5 <- map(
alist( Score ~ dbinom(1, p),
logit(p) <- a[SelectedCardIndex] + b[Participant],
a[SelectedCardIndex] ~ dnorm(0, 10),
b[Participant] ~ dnorm(0, 10)
), data= ZCdata )
Maximum a posteriori (MAP) model fit
Formula:
Score ~ dbinom(1, p)
logit(p) <- a[SelectedCardIndex] + b[Participant]
a[SelectedCardIndex] ~ dnorm(0, 10)
MAP values:
a[1] a[2] a[3] a[4] a[5] b[1] b[2] b[3] b[4] b[5] b[6]
b[7] b[8] b[9] b[10] b[11] b[12] b[13] b[14] b[15] b[16] b[17]
b[18] b[19] b[20] b[21] b[22]
Log-likelihood:

22. Looking at probabilities
aValues5<-c()
for(p in unique(ZCdata$Participant)){
for(p2 in unique(ZCdata$CardIndex)){
code<-sprintf("b[%d]", p)
code2<-sprintf("a[%d]", p2)
aValues5<- c(aValues5, coef(ZCmodel5)[code] + coef(ZCmodel5)[code2]) }
cat("Probability of correct response: ", logistic(aValues5))
Probability of correct response:

23. Comparing models
> compare(ZCmodel0, ZCmodel1, ZCmodel4, ZCmodel3, ZCmodel5)
WAIC pWAIC dWAIC weight SE dSE
ZCmodel NA
ZCmodel
ZCmodel
ZCmodel
ZCmodel

24. Trial effects Modify the code to look for a trial effect
Does a model that includes just trial effects (no card or participant effects) beat the null model?