1. Measuring Belief Bias with Ternary Response Sets
Samuel Winiger, Henrik Singmann, David Kellen

2. Valid Invalid Syllogism No oaks are jubs. Some trees are jubs.
Logical arguments: Premise: Putative Conclusion
No oaks are jubs.
Some trees are jubs.
Therefore, some trees are not oaks.
Believable and valid, the ultimate mind fuck.
Valid
Invalid

3. Belief Bias in Syllogistic Reasoning
Believability
Validity
Believable
Unbelievable
Valid
No oaks are jubs.
No trees are jubs.
Some trees are jubs.
Some oaks are jubs.
Therefore, some trees are not oaks.
Therefore, some oaks are not trees.
Invalid

4. Belief Bias in Syllogistic Reasoning
Believability
Validity
Believable
Unbelievable
Valid
No oaks are jubs.
No trees are jubs.
Some trees are jubs.
Some oaks are jubs.
Therefore, some trees are not oaks.
Therefore, some oaks are not trees.
Invalid
89%
56%
71%
10%
Data: Evans et al. (1983)

5. Belief Bias in Syllogistic Reasoning
Two possible explanations:
Believability affects reasoning processes (e.g., more effort for unbelievable syllogisms)
Believability affects response bias (e.g., higher propensity for accepting believable conclusions)
Belief Bias in Syllogistic Reasoning
Believability
Validity
Believable
Unbelievable
Valid
No oaks are jubs.
No trees are jubs.
Some trees are jubs.
Some oaks are jubs.
Therefore, some trees are not oaks.
Therefore, some oaks are not trees.
Invalid
89%
56%
71%
10%
Data: Evans et al. (1983)

6. Threshold-Model for Belief Bias
Klauer, Musch & Naumer, (2000)

7. Threshold-Model for Belief Bias
Problem:
Data provides 4 independent data points
Model has 6 free parameters (4 reasoning and 2 guessing parameters)
Model parameters not uniquely identified
Klauer et al.'s solution: response bias manipulation
Participants in one of three bias condition: 17% versus 50% versus 83% valid
Fix reasoning parameters across conditions, but allow for different response bias
Belief bias solely affected reasoning processes.
Klauer, Musch & Naumer, (2000)

8. Signal Detection Model of Belief Bias
Dube, Rotello, & Heit (2010)

9. Signal Detection Model of Belief Bias
Signal detection based analysis with confidence-rating:
Belief bias mainly a response bias effect! (also Trippas, Kellen, Singmann, et al. in press, PB&R)
Dube, Rotello, & Heit (2010)

10. Experiment Syllogism evaluation task with 3 response options: "valid"
"I don't know"
"invalid"
Belief Bias: Driven by reasoning processes or response processes?
Logical validity (valid vs. invalid)
Conclusion believability (believable vs. unbelievable)
354 Participants (online study)
8 syllogisms per participant
12/26/2018
Title of the presentation, Author

11. Results: Response Frequencies
We see a belief bias effect:
marginal proportion for “Yes” is higher for believable conclusions.
Larger effect of believability for invalid syllogisms.
12/26/2018
Title of the presentation, Author

12. Extended Threshold Model
rvb
valid
rib
invalid
valid and believable
invalid and believable nb
I don’t know nb
I don’t know
1-rvb gb
valid
1-rib gb
valid
1-nb
1-nb
1-gb
invalid
1-gb
invalid
rvu
valid
riu
invalid
valid and unbelievable
invalid and unbelievable nu
I don’t know nu
I don’t know
1-rvu gu
valid
1-riu gu
valid
1-nu
1-nu
1-gu
invalid
1-gu
invalid

13. Modeling Results: Extended belief bias MPT
Group- level posterior estimates of the difference parameters.
12/26/2018
Title of the presentation, Author

14. Modeling Results: Extended belief bias MPT
BF0: 1.4 – 2.0
BF0: 2.1 – 4.5
BF0: 0.8 – 1.6
BFAlt: 16 – 230
Group- level posterior estimates of the difference parameters.
12/26/2018
Title of the presentation, Author

15. Summary
Binary "Valid"/"Invalid" response format does not provide independent data points for comprehensive measurement model of belief bias
Klauer et al. (2000): threshold model
Response bias manipulation
Believability affects reasoning processes
Dube et al. (2010); Trippas et al. (in press); Stephens, Dunn, & Hayes (2017): signal-detection model
Confidence-ratings
Believability affects response processes
Our results: extended threshold model
Ternary response sets
Believability affects response processes (i.e., larger propensity for responding "valid" for believable syllogisms)
Differences in conclusions not dependent on model, but manipulation for achieving parameter identifiability.
Substantive theories of belief bias must include response bias!
12/26/2018
Title of the presentation, Author

16. Thank you for your attention. Questions?
12/26/2018
Title of the presentation, Author

17. Syllogisms Structures Complex structures (indeterminately invalid)
- Dube et al. (2010) experiments 1-3
- Klauer et al. (2000) experiments 3, 4, and 7.
This set includes Trippas et al. (2013) and Stephens et al. (2017)
Contents
Rated contents from all around the literature
Klauer et al., 2000
Dube et al., 2010
Ball, Phillips, Wade, & Quayle, 2006
Oakhill & Johnson-Laird, 1985
Quayle & Ball, 2000; Evans et al., 1983
12/26/2018
Title of the presentation, Author

18. Results: Bayes Factors
Bayes factors derived from the difference parameters between conditions * Bayes factors in favor of the null Hypothesis
Parameter rv ri n g
narrow prior
1.4*
1.2
2.1*
232.0
medium prior
1.5*
1.3*
3.1*
70.3
wide prior
2.0*
1.6*
4.5*
16.1
12/26/2018
Title of the presentation, Author