1. Copyright © Allyn & Bacon (2007) Final Preparations Before Data Collection Graziano and Raulin Research Methods: Chapter 14 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.

2. Copyright © Allyn & Bacon (2007) Selecting Statistical Procedures Selecting statistical analysis procedures is part of the design process Selecting statistical analysis procedures is part of the design process Based on the characteristics of the study Based on the characteristics of the study Use the information about the study to follow the flowcharts to arrive at decisions about statistics to use Use the information about the study to follow the flowcharts to arrive at decisions about statistics to use Need not memorize the flowcharts Need not memorize the flowcharts

3. Copyright © Allyn & Bacon (2007) Describing Your Research (1/2) 1. What is its level of constraint? 2. What are the independent variables? 3. What are the levels of the independent variables? 4. What type of design is the research? 5. What are the dependent variables?

4. Copyright © Allyn & Bacon (2007) Describing Your Research (2/2) 6. What is (are) the dependent measure(s)? 7. What is the level of measurement of each dependent measure? 8. What type of data is generated for each dependent measure? 9. List the research hypothesis (hypotheses)? 10. What kind of test is needed?

5. Copyright © Allyn & Bacon (2007) Structure of the Flowcharts Initial Flowchart (14.1) Initial Flowchart (14.1) –Prompts for the details of the study –Sends to other flowcharts based on data type Secondary Flowcharts Secondary Flowcharts –Score Data Flowchart (14.2) –ANOVA Flowchart (14.3) –Ordered Data Flowchart (14.4) –Nominal Data Flowchart (14.5)

6. Copyright © Allyn & Bacon (2007) Initial Flowchart

7. Copyright © Allyn & Bacon (2007) Score Data Flowchart

8. Copyright © Allyn & Bacon (2007) ANOVA Flowchart

9. Copyright © Allyn & Bacon (2007) Ordered Data Flowchart

10. Copyright © Allyn & Bacon (2007) Nominal Data Flowchart

11. Copyright © Allyn & Bacon (2007) Functional Flowchart (Student Resource Website) Functional flowchart walks you through the process of selecting a statistical test Functional flowchart walks you through the process of selecting a statistical test Links to the manual and computerized computation tutorials Links to the manual and computerized computation tutorials

12. Copyright © Allyn & Bacon (2007) Secondary Analyses Defined as analyses that do not directly test the hypothesis of the study Defined as analyses that do not directly test the hypothesis of the study Three types of secondary analyses Three types of secondary analyses –Post hoc analyses: Such as specific means comparisons in ANOVA –Secondary analyses to explain results –Data snooping: Exploring the data for clues that might form hypotheses for later research

13. Copyright © Allyn & Bacon (2007) Caveats and Disclaimers Flowcharts are a teaching device only Flowcharts are a teaching device only NOT every statistical procedure is included NOT every statistical procedure is included –Only the most common (e.g., testing mean differences) Procedures suggested by flowcharts, although the correct procedures to use, are NOT the only correct procedures for that situation Procedures suggested by flowcharts, although the correct procedures to use, are NOT the only correct procedures for that situation

14. Copyright © Allyn & Bacon (2007) Robust Statistical Procedures A statistical procedure is robust to an assumption IF the assumption can be violated without distorting results A statistical procedure is robust to an assumption IF the assumption can be violated without distorting results Evaluated with Monte Carlo procedures Evaluated with Monte Carlo procedures –Computer simulations of the effects of various situations on the effectiveness of a statistic Evidence suggests that assumption violations have little effect IF your sample sizes about equal in each group Evidence suggests that assumption violations have little effect IF your sample sizes about equal in each group

15. Copyright © Allyn & Bacon (2007) Pre-Data Check A final check before beginning the research A final check before beginning the research Also serves as an excellent check during the planning for the study Also serves as an excellent check during the planning for the study Covers Covers –Study definition and design –Selection of all procedures –Participant selection and assignment –Preparation for running the study

16. Copyright © Allyn & Bacon (2007) Checklist (1/8) Initial Problem Definition Initial Problem Definition –Literature reviewed? –Problem statement developed? –Variables operationally defined? Research Hypotheses Research Hypotheses –Clear hypotheses stating the direction and type of relationship expected?

17. Copyright © Allyn & Bacon (2007) Checklist (2/8) Statistical Analyses Statistical Analyses –Descriptive statistics planned? –Inferential statistics planned? –Post hoc and secondary analyses planned? Theory and operational definitions Theory and operational definitions –Theoretical base for study clear? –Do hypotheses and procedures address the issues?

18. Copyright © Allyn & Bacon (2007) Checklist (3/8) Independent Variable Manipulation Independent Variable Manipulation –Manipulation carefully planned? –Manipulation pretested? –Manipulation check planned? Dependent Measures Dependent Measures –Dependent measures operationally defined? –Measures pretested or piloted? –Measures reliable and valid? –Procedures to measure reliability available?

19. Copyright © Allyn & Bacon (2007) Checklist (4/8) Controls in place Controls in place –For threats to internal validity –For threats to external validity Participants Participants –Adequately sampling target population? –Descriptive demographics collected? –Sample large enough? –Participants properly assigned to conditions?

20. Copyright © Allyn & Bacon (2007) Checklist (5/8) Participants (continued) Participants (continued) –Groups carefully defined (differential research)? –Matching information preserved (matched- subjects designs)? –Participants available? –Scheduling procedure ready? –Participant incentive procedures in place (e.g., fees, credit)?

21. Copyright © Allyn & Bacon (2007) Checklist (6/8) Research ethics Research ethics –IRB approval obtained? –Informed consent forms ready (for human research)? –Debriefing and/or feedback procedures in place? –Ethical guidelines checked (for animal research)?

22. Copyright © Allyn & Bacon (2007) Checklist (7/8) Preparation of setting (continued) Preparation of setting (continued) –Adequate space available? –Space free of distractions? –Equipment checked out? –Enough research help available? –Assistants adequately trained for emergencies? –Assistants trained on research procedures? –Blind procedures in place (if necessary)?

23. Copyright © Allyn & Bacon (2007) Checklist (8/8) Participant Preparation, Instruction, and Procedures Participant Preparation, Instruction, and Procedures –Instructions to participants clear? –Instructions and procedures pilot tested?

24. Copyright © Allyn & Bacon (2007) Checklist Complete? Then Go for it! Then Go for it!

25. Copyright © Allyn & Bacon (2007) Summary Select statistical procedure as part of design process Select statistical procedure as part of design process –Statistical procedures selected before data collection –Selection based on study characteristics Pre-data checklist Pre-data checklist –A final check before beginning research –Covers design, ethical, and practical issues