Interpreting Empirical Research Results A Tutorial

Agenda  What is Research?  What is a Theory?  Reading Research Papers  Research Criteria  Sampling Considerations  Caveat Emptor

What is Research?  Directed  Purposive  Collection of Data  To shed light on  A theory Executed according to a plan Goal-oriented, with measurable objectives For empirical research, data are used to see if theory is “true” A sort of stress test for theories; if theory doesn’t hold water, then it’s false. A set of statements about reality, somewhat simplified, perhaps saying that X causes Y

Realm of Ideas and Thoughts Reality Hypothetico-Deductive Method TheoryHypotheses Implies Test Subjected to a real-world Data/ Experience From which are gathered… Analyzed and compared to Scientist Views or experiences phenomena in the real world and has Idea about Reality That are crafted into coherent Focus on the test Focus on the ideas

Reading Research Papers  What is the theory?  What is the model?  What are the data “about”?  Is the research method or procedure trustworthy?  Is the sample used representative?  How can results be applied?  Is the research ethical?

Realm of Ideas and Thoughts Reality Theorizing Theory Idea about Reality That are crafted into coherent A Theory is a set of statements that simplify reality while saying something about its structure. It is a link between the realm of ideas and reality. There are many kinds of theories…

Kinds of Theories  Pictorial: an image that stands for reality  Descriptive: a set of dimensions that distinguish various elements or phenomena  Associational: a set of statements that tell what phenomena appear or go together  Causal: a set of statements that say that some phenomena cause other phenomena  Implicational: a set of statements that say that some phenomena imply other phenomena  Operational: a set of conditions that make things happen or cause parts of reality to reappear.

Realm of Ideas and Thoughts Reality Crafting The Hypotheses TheoryHypotheses Implies Test Subjected to a real-world The research hypotheses are logical implications of the theory. If the theory is true, the hypotheses should be true. A real- world test of the hypotheses tries to see if the hypotheses are false in reality. If they turn out to BE false, then the theory they are based on must also be false.

Realm of Ideas and Thoughts Reality Conducting The Test Hypotheses Test Subjected to a real-world Data/ Experience From which are gathered… The real-world test should “stress” the hypotheses strongly. To the extent that the test tries to prove the hypotheses false, and fails to do so, then we retain our belief that the hypotheses (and hence the theory the hypotheses are based on) are true. If the test is TOO strong, then ANY hypothesis might seem false and we don’t trust the TEST.

Realm of Ideas and Thoughts Reality Interpreting the Data Theory Test Data From which are gathered… Scientist Data arising from the real- world test are analyzed and the researcher reviews experiences in the real world test and revises the ideas in the theory (or rejects the theory altogether)

Research Criteria  Validity: are the results really about the research question? Are the data about pheonomena the research question is about?  Reliability: are the results replicable? Are they generalizable to a larger set of situations?

Sampling Considerations  Is the sample representative?  How was the sample obtained?  What was the sampling frame?  Was the sample treated ethically?

Realm of Ideas and Thoughts Reality Hypothetico-Deductive Method TheoryHypotheses Implies Test Subjected to a real-world Data/ Experience From which are gathered… Analyzed and compared to The results are either (1) test failed to show theory is false or (2) test showed theory to be false. In the case of (1), there is a question of whether the test was strong enough. In the case of (2) the question is whether or not the test was TOO strong. An unwanted result is (3) test neither showed nor failed to show test to be false. NOTE: test cannot show theory to be true, in general.

Caveat Emptor