Measuring Belief Bias with Ternary Response Sets Samuel Winiger, Henrik Singmann, David Kellen

Valid Invalid Syllogism No oaks are jubs. Some trees are jubs. Logical arguments: Premise: Putative Conclusion No oaks are jubs. Some trees are jubs. Therefore, some trees are not oaks. Believable and valid, the ultimate mind fuck. Valid Invalid

Belief Bias in Syllogistic Reasoning Believability Validity Believable Unbelievable Valid No oaks are jubs. No trees are jubs. Some trees are jubs. Some oaks are jubs. Therefore, some trees are not oaks. Therefore, some oaks are not trees. Invalid

Belief Bias in Syllogistic Reasoning Believability Validity Believable Unbelievable Valid No oaks are jubs. No trees are jubs. Some trees are jubs. Some oaks are jubs. Therefore, some trees are not oaks. Therefore, some oaks are not trees. Invalid 89% 56% 71% 10% Data: Evans et al. (1983)

Belief Bias in Syllogistic Reasoning Two possible explanations: Believability affects reasoning processes (e.g., more effort for unbelievable syllogisms) Believability affects response bias (e.g., higher propensity for accepting believable conclusions) Belief Bias in Syllogistic Reasoning Believability Validity Believable Unbelievable Valid No oaks are jubs. No trees are jubs. Some trees are jubs. Some oaks are jubs. Therefore, some trees are not oaks. Therefore, some oaks are not trees. Invalid 89% 56% 71% 10% Data: Evans et al. (1983)

Threshold-Model for Belief Bias Klauer, Musch & Naumer, (2000)

Threshold-Model for Belief Bias Problem: Data provides 4 independent data points Model has 6 free parameters (4 reasoning and 2 guessing parameters) Model parameters not uniquely identified Klauer et al.'s solution: response bias manipulation Participants in one of three bias condition: 17% versus 50% versus 83% valid Fix reasoning parameters across conditions, but allow for different response bias Belief bias solely affected reasoning processes. Klauer, Musch & Naumer, (2000)

Signal Detection Model of Belief Bias Dube, Rotello, & Heit (2010)

Signal Detection Model of Belief Bias Signal detection based analysis with confidence-rating: Belief bias mainly a response bias effect! (also Trippas, Kellen, Singmann, et al. in press, PB&R) Dube, Rotello, & Heit (2010)

Experiment Syllogism evaluation task with 3 response options: "valid" "I don't know" "invalid" Belief Bias: Driven by reasoning processes or response processes? Logical validity (valid vs. invalid) Conclusion believability (believable vs. unbelievable) 354 Participants (online study) 8 syllogisms per participant 12/26/2018 Title of the presentation, Author

Results: Response Frequencies We see a belief bias effect: marginal proportion for “Yes” is higher for believable conclusions. Larger effect of believability for invalid syllogisms. 12/26/2018 Title of the presentation, Author

Extended Threshold Model rvb valid rib invalid valid and believable invalid and believable nb I don’t know nb I don’t know 1-rvb gb valid 1-rib gb valid 1-nb 1-nb 1-gb invalid 1-gb invalid rvu valid riu invalid valid and unbelievable invalid and unbelievable nu I don’t know nu I don’t know 1-rvu gu valid 1-riu gu valid 1-nu 1-nu 1-gu invalid 1-gu invalid

Modeling Results: Extended belief bias MPT Group- level posterior estimates of the difference parameters. 12/26/2018 Title of the presentation, Author

Modeling Results: Extended belief bias MPT BF0: 1.4 – 2.0 BF0: 2.1 – 4.5 BF0: 0.8 – 1.6 BFAlt: 16 – 230 Group- level posterior estimates of the difference parameters. 12/26/2018 Title of the presentation, Author

Summary Binary "Valid"/"Invalid" response format does not provide independent data points for comprehensive measurement model of belief bias Klauer et al. (2000): threshold model Response bias manipulation Believability affects reasoning processes Dube et al. (2010); Trippas et al. (in press); Stephens, Dunn, & Hayes (2017): signal-detection model Confidence-ratings Believability affects response processes Our results: extended threshold model Ternary response sets Believability affects response processes (i.e., larger propensity for responding "valid" for believable syllogisms) Differences in conclusions not dependent on model, but manipulation for achieving parameter identifiability. Substantive theories of belief bias must include response bias! 12/26/2018 Title of the presentation, Author

Thank you for your attention. Questions? 12/26/2018 Title of the presentation, Author

Syllogisms Structures Complex structures (indeterminately invalid) - Dube et al. (2010) experiments 1-3 - Klauer et al. (2000) experiments 3, 4, and 7. This set includes Trippas et al. (2013) and Stephens et al. (2017) Contents Rated contents from all around the literature Klauer et al., 2000 Dube et al., 2010 Ball, Phillips, Wade, & Quayle, 2006 Oakhill & Johnson-Laird, 1985 Quayle & Ball, 2000; Evans et al., 1983 12/26/2018 Title of the presentation, Author

Results: Bayes Factors Bayes factors derived from the difference parameters between conditions * Bayes factors in favor of the null Hypothesis Parameter rv ri n g narrow prior 1.4* 1.2 2.1* 232.0 medium prior 1.5* 1.3* 3.1* 70.3 wide prior 2.0* 1.6* 4.5* 16.1 12/26/2018 Title of the presentation, Author