PSYC512: Research Methods PSYC512: Research Methods Lecture 4 Brian P. Dyre University of Idaho

PSYC512: Research Methods Lecture 4 Outline Questions about material covered in Lecture 3 Questions about material covered in Lecture 3 Scientific Method Scientific Method Scope and assumptions Scope and assumptions Theories and hypothesis testing Theories and hypothesis testing Proof and disproof Proof and disproof Strong Inference Strong Inference

PSYC512: Research Methods Methods of Obtaining Knowledge (review) Peirce (1877) Peirce (1877) Authority (faith) Authority (faith) Tenacity (sticking your head in the sand) Tenacity (sticking your head in the sand) A priori (pragmatism) A priori (pragmatism) Scientific method Scientific method

PSYC512: Research Methods Features of Scientific Method Rational – based on logic Rational – based on logic Empirical – based on data Empirical – based on data Testable – rational theories and hypotheses are testable Testable – rational theories and hypotheses are testable Parsimonious – the simplest explanation is most likely true Parsimonious – the simplest explanation is most likely true General – theories should account for broad phenomena General – theories should account for broad phenomena Tentative – explanations are never proved, they may always be improved (skeptical) Tentative – explanations are never proved, they may always be improved (skeptical) Rigorously Evaluated – replicate, replicate, replicate! Rigorously Evaluated – replicate, replicate, replicate! Self-correcting – theories that are disproven are refined or abandoned Self-correcting – theories that are disproven are refined or abandoned

PSYC512: Research Methods The Scope of Science The scope of science is limited to questions that are tractable using the scientific method. The scope of science is limited to questions that are tractable using the scientific method. Some assumptions of science Some assumptions of science Objective reality exists Objective reality exists philosophy of materialism or physicalism – we can talk about reality in the third vs. first person philosophy of materialism or physicalism – we can talk about reality in the third vs. first person Reality is deterministic (causality exists, how else would scientific models make predictions?) Reality is deterministic (causality exists, how else would scientific models make predictions?) Reality is systematically observable Reality is systematically observable Questions or theories that violate these assumptions are beyond the scope of science, E.g., questions of “Does God Exist?” or theories of “Intelligent Design”, are metaphysical, not scientific. Questions or theories that violate these assumptions are beyond the scope of science, E.g., questions of “Does God Exist?” or theories of “Intelligent Design”, are metaphysical, not scientific. The precision of scientific “answers” (theories) depends on the precision of the question. Vague questions lead to vague answers. E.g. “Why do bad things happen to good people?” The precision of scientific “answers” (theories) depends on the precision of the question. Vague questions lead to vague answers. E.g. “Why do bad things happen to good people?”

PSYC512: Research Methods What do theories do? What purpose do they serve? What do theories do? What purpose do they serve? Understanding Understanding Prediction Prediction Organizing and Interpreting Results Organizing and Interpreting Results Generating Research Generating Research What makes a theory a good theory? What makes a theory a good theory? Broad explanatory power Broad explanatory power Defines logical links between variables Defines logical links between variables Predicts novel events precisely enough to be tested (confirmed) Predicts novel events precisely enough to be tested (confirmed) Predicts non-events precisely enough to be tested (disconfirmed) Predicts non-events precisely enough to be tested (disconfirmed) Parsimony Parsimony Theories and Hypothesis Testing Theory Data InductionDeduction

PSYC512: Research Methods To Prove or Disprove, That is the Question… Conditional Reasoning and the logic of falsification (Popper) Conditional Reasoning and the logic of falsification (Popper) Theories Predict Data Theories Predict Data Confirmational Strategy: trying to prove a theory Confirmational Strategy: trying to prove a theory “If theory A is correct, “If theory A is correct, then I will observe pattern of data A” Disconfirmational Strategy Disconfirmational Strategy “If theory A is correct, “If theory A is correct, then I will not observe pattern of data B” These are statements of conditional reasoning These are statements of conditional reasoning

PSYC512: Research Methods To Prove or Disprove, That is the Question… Conditional Reasoning: The Propositional Calculus Conditional Reasoning: The Propositional Calculus Two premises and a conclusion Two premises and a conclusion Premise 1) If then Premise 1) If then Premise 2) Affirm/deny Premise 2) Affirm/deny Conclusion) Therefore Conclusion) Therefore Four Possibilities for Premise 2 Four Possibilities for Premise 2 Affirm AntecedentDeny Antecedent Affirm ConsequentDeny Consequent

PSYC512: Research Methods To Prove or Disprove, That is the Question… Confirmational Reasoning Confirmational Reasoning Premise 1 Premise 1 If then If then Premise 2; Conclusion Premise 2; Conclusion AA) theory A is correct; therefore data A will be observed (Valid, but pointless) DA) theory A is incorrect; therefore data A will not be observed (Invalid and pointless) AC) data A observed; therefore theory A is correct (Invalid, but often used) DC) data A not observed; therefore theory A is incorrect (valid, but only if observations are exhaustive—accepting the null)

PSYC512: Research Methods To Prove or Disprove, That is the Question… Disconfirmational Reasoning Disconfirmational Reasoning Premise 1 Premise 1 If then If then Premise 2; Conclusion Premise 2; Conclusion AA) theory A is correct; therefore data B will not be observed (Valid, but pointless) DA) theory A is incorrect; therefore data B will be observed (Invalid and pointless) AC) data B not observed; therefore theory A is correct (Invalid) DC) data B observed; therefore theory A is incorrect (valid, most scientifically useful!)

PSYC512: Research Methods Confirmation and Disconfirmation of Theories: Summary Confirmation (Poor) Confirmation (Poor) if theory correct then observation will occur if theory correct then observation will occur Observation occurs  Support, but not proof Observation occurs  Support, but not proof Observation does not occur  disproof? NO! Observation does not occur  disproof? NO! Disconfirmation (OK) Disconfirmation (OK) if theory correct then observation will not occur if theory correct then observation will not occur Observation does not occur  Support, but not proof Observation does not occur  Support, but not proof Observation does occur  disproof Observation does occur  disproof Strong Inference (BEST!) Strong Inference (BEST!) PROOF?

PSYC512: Research Methods Strong Inference (Platt, 1964) Science is fundamentally based on disconfirmation (Popper) Science is fundamentally based on disconfirmation (Popper) Theories are not evaluated in isolation, rather they compete with one another (relativism) Theories are not evaluated in isolation, rather they compete with one another (relativism) Critical Experiments – results will disconfirm one (or more) theory (theories) while confirming one or more alternative theories Critical Experiments – results will disconfirm one (or more) theory (theories) while confirming one or more alternative theories Disconfirmed theories are discarded (or revised) like logical branches pruned from the tree of understanding, in which only one branch represents truth Disconfirmed theories are discarded (or revised) like logical branches pruned from the tree of understanding, in which only one branch represents truth

PSYC512: Research Methods Strong Inference (Platt, 1964) The Question to ask in your own mind on hearing The Question to ask in your own mind on hearing any scientific explanation or theory put forward: any scientific explanation or theory put forward: “What experiment could disprove your hypothesis?” or on hearing a scientific experiment described: on hearing a scientific experiment described: “What hypothesis does your experiment disprove?” Practicing explicit and formal analytical thinking Practicing explicit and formal analytical thinking the “notebook” containing the logical trees, alternative hypotheses, and crucial experiments the “notebook” containing the logical trees, alternative hypotheses, and crucial experiments include as an appendix to your annotated bibliography include as an appendix to your annotated bibliography

PSYC512: Research Methods Next Time… Topic: Measurement scales, descriptive stats, variables and sampling Topic: Measurement scales, descriptive stats, variables and sampling Be sure to: Be sure to: Read the assigned readings (Howell chapters 1-3) Read the assigned readings (Howell chapters 1-3) Continue searching and reading the scientific literature for your proposal Continue searching and reading the scientific literature for your proposal