Methodology Matters: Doing Research in The Behavioral and Social Sciences Student:Way Chang Cai Scott Lippert

Introduction ► Doing Research simply means the systematic use of some set of theoretical and empirical tools to try to increasing our understanding of some set of phenomena or events.

Some Basic Features of The Research Process ► Some content that is of interest ► Some ideas that give meaning to that content ► Some techniques or procedures by means of which those ideas and contents can be studied.

Some Basic Features of The Research Process ► Substantive domain, which we draw contents that seem worthy of our study and attention.  Understand both “ actor ” and “ context ” here refer to human system ► Conceptual domain, which we draw ideas that seem likely to give meaning to our results.  The idea give the meaning that we study ► Methodological domain, which we draw techniques that seem useful in conducting that research.  Methods are the tools – the instruments, techniques and procedures – by which a science gathers and analyzes information

Methods are the tools The instruments, techniques and procedures. Ex. If you want to cut a apple, a hammer don ’ t help much. For that you need a saw. All research method help you gain knowledge. All method used to gather and analyze evidence

However ► All methods have inherent flaws, through each has certain potential advantages. You can ’ t avoid these flaws, but you can bring more than one method to bear on each of aspect of a problem.

Research Strategies: Choosing a Setting for a Study ► Research evidence in social and behavioral science always involves somebody doing something in some situation, which are who, what and where.

► Three criteria  Generalizability of evidence over the population of Actors.  Precision of measurement of the behaviors that are being studied.  Realism of situation or context within which the evidence is gathered, in relation to the contexts to which you want evidence to apply. ► You may always want to maximize these criteria, but increasing one of these three features reduces one or both of the other two.

Sample Survey Field Study Judgment Study Experimental Simulation Laboratory Experiment Computer Simulation Formal Theory Field Experiment Quadrant III Respondent Strategies Quadrant II Experimental Strategies Quadrant I Field Strategies Quadrant IV Theoretical Strategies Maxima for each criterion: A = Generalizability B = Precision C = Realism Obtrusive Unobtrusive Abstract Concrete

Quadrant I ► Field Study  The researcher sets out to make direct observations of “ natural ”, ongoing systems, while intruding on and disturbing those systems as little as possible. (case studies) ► Field experiment  Researcher gives up some of the unobtrusiveness of the plain field study, in interest of gaining more precision in the information resulting from the study.

Quadrant II ► Laboratory experiment  Researcher is able to study the behaviors of interest with considerable precision, and to do so under conditions where many extraneous factors have been eliminated or brought under experimental control. ► Experimental simulation  The researcher attempts to achieve much of the precision and control of the laboratory experiment but to gain some of the realism ( apparent realism ) of field studies.

Quadrant III ► Sample survey  The investigator tries to obtain evidence that will permit him or her to estimate the distribution of some variables, and/or some relationships among them, within a specified population. ► Judgment Study  Researcher concentrates on obtaining information about the properties of a certain set of stimulus materials, usually arranged so that they systematically reflect the properties of some broad stimulus domain.

Quadrant IV ► Formal theory  The researcher focuses on formulating general relations among a number of variables of interest. ► Computer Simula  Complete and closed system that models the operation of the concrete system without any behavior by any system participants.

Strategic Issues

Baserates ► Provide a basis for comparison  Eg: “ 3 out of 10 smokers develop lung cancer ”  Is this high, low or normal?  A baserate will give this information

Correlation ► Does X covary with Y?  E.g. “ Does happiness vary with age? ” ► Covariance can have high positive or negative value  Covariance can be linear or nonlinear ► Nonlinear is often overlooked  Covariance shows that X and Y have a relationship, but not necessarily a causal one

The Difference Question ► Does the presence of X alter the state or magnitude of Y?  If so, to what degree?

Randomization in Experiments ► Attempt to minimize the effect of extraneous factors ► Randomization doesn ’ t guarantee equal distribution of extraneous factors  Does make highly unequal distribution unlikely

Sampling, Allocation ► A large sample minimizes chance results ► Probability of chance causing results can be estimated  Accuracy limited by knowledge of extraneous factors ► Even if results are likely not caused by chance, results don ’ t prove X caused Y  Could be that ignored variable caused Y  This can be prevented with careful design

Validity of Findings ► Internal Validity  Degree the results allow causal relations to be determined ► Construct Validity  How well defined is the theory behind the study ► External Validity  Concerned with the limits under which the results are valid

Classes of Measures and Manipulation

Potential Classes of Measures ► Self report  Questionnaire, diary ► Observations  Visible or Invisible ► Refers to whether subject knows he is being observed ► Archival records  Records usually made for non-research purposes ► Trace measures

Self Reports ► Strengths  Inexpensive, Easy to make, low rate of information discarded ► Weaknesses  Reactivity ► This means that people respond as they think they should, rather than how they normally would

Observations ► Strengths  Large amounts of information created ► Weaknesses  Large amounts of information created, reactivity, high cost, ethics issues

Trace Measures ► Strengths  Unobtrusive, non-reactive ► Weaknesses  Information lacks context, information is not specific enough

Archival ► Strengths  Sometimes only option, inexpensive ► Weaknesses  Reactive, loose link between the information desired and that archived

Techniques for Manipulating Variables ► Selection ► Direct intervention ► Induction

Selection ► Group based on one criteria  E.g. Age, sex ► Lose random distribution of the variable you are manipulating

Direct Intervention ► Set up the desired situation  E.g. creating juries of 12, 6  Inexpensive, gives reliable information  Can still distribute randomly ► Only works for tangible, superficial variables ► Some reactivity exists

Induction ► Different types  Misleading instructions ► Lie to subjects about what is being measured  False feedback ► Give feedback based on the needs of the experiments, not the results obtained by subject  Experimenters pretend to be subjects ► Carry out pre-determined activities to measure results