1. PSYC512: Research Methods PSYC512: Research Methods Lecture 18 Brian P. Dyre University of Idaho

2. PSYC512: Research Methods Lecture 18 Outline Research Proposal Information Research Proposal Information Writing Research Proposal Writing Research Proposal The Art of Peer Review The Art of Peer Review Addressing Reviewer’s Comments Addressing Reviewer’s Comments Inferential Statistics Inferential Statistics Testing for differences Testing for differences Determining if a sample represents a population Determining if a sample represents a population Determining if two or more samples differ Determining if two or more samples differ Testing for relationships Testing for relationships Form of relationship Form of relationship Strength of relationship Strength of relationship

3. PSYC512: Research Methods Writing a Research Proposal Purpose of Research Proposal Present a literature review defining concepts and intervening variables pertinent to research question Present a specific research question with explicit hypotheses to be tested Develop a plan for addressing that research question empirically, including descriptions of Target population and subject sampling The research design, including explicit definition of independent and dependent variables and how the stimuli and procedures implement these variables Aspects of stimuli and procedures that control for extraneous and confounding variables Types of analyses to be used, predicted results, and how these predictions relate to the hypotheses

4. PSYC512: Research Methods Research Proposals: Global Concerns Scientific Writing Style Precision more important than entertainment, but… Research is part science and part advertising – not only do you need to develop good ideas but you must be able to sell your ideas Proposal is often the basis of first impression for the quality of the research project Proposal must be clear on ALL levels of analysis Words Sentences Paragraphs OUTLINE! Sections

5. PSYC512: Research Methods Research Proposals: Global Concerns APA guidelines Orderly expression of ideas - organization Smoothness of expression - transitions Economy of expression - concise language Precision and clarity – use scientific vocabulary “jargon” correctly and insure that all terms are defined the first time they are used

6. PSYC512: Research Methods Organization of a Research Proposal: APA STYLE Title Page – title should specifically describe what the paper is about so that it is useful information for other researchers’ literature searches Abstract essentially a mini-paper for lit. searches be extremely CONCISE! (< 150 words) introduce specific topic discuss variables, etc. present major results (no statistics!) discuss important conclusions

7. PSYC512: Research Methods Organization of a Research Proposal : APA STYLE Introduction Purpose Demonstrate knowledge of relevant research Define intervening variables and their relation to manipulations and measurements used in previous research Present and justify research question and hypotheses Present and justify the general method to be used Organization Start broad then narrow to your general purpose Discuss only relevant research in a logical flow Near the end provide an explicit statement of hypotheses and an overview of the general research design

8. PSYC512: Research Methods Organization of a Research Proposal: APA STYLE Method explicitly state how variables are manipulated define in separate sub-sections Participants (subjects) Design Stimuli/Apparatus/Materials Procedures Results Describe the Scales of IVs and DVs Describe Transformations of DVs Summarized planned statistical analyses Summarize predictions

9. PSYC512: Research Methods Important Elements of Style (Brian’s pet peeves) Define scientific terms (“jargon”) and abbreviations at first use, then use terms consistently Do not use informal or colloquial language e.g., you don’t “run” subjects, you test subjects Ambiguous pronouns: if you use the word “it” make sure the surrounding context makes the meaning of the word “it” obvious, otherwise avoid using it (as in “it”) Sexist pronouns (he vs. she) – word sentences to avoid having to use these if at all possible, or use “he or she”

10. PSYC512: Research Methods Important Elements of Style (Brian’s pet peeves) Verb tense Introduction Section: Discussion of specific previous research  past tense, Discussion of ideas or concepts that are general for all time and not linked to specific moment in the past  present tense Method and results Proposal: future tense research report: past tense

11. PSYC512: Research Methods Important Elements of Style (Brian’s pet peeves) Plural/singular mismatches Passive voice – word sentences in active voice Superfluous, imprecise language (e.g., avoid vague adverbs—search for all words ending in “ly” and consider eliminating them) Style issues like “that” vs. “which” For more information on writing style I recommend: Strunk, W., jr., and White, E.B. (2000) The Elements of Style, 4 th Ed. Boston: Allyn and Bacon.

12. PSYC512: Research Methods The Art of Peer Review Example review available online in lecture schedule Example review available online in lecture schedule Goal: Assist author in improving the clarity and impact of the paper (or proposal) by offering specific constructive criticism (no name calling!) and complements (where appropriate) Goal: Assist author in improving the clarity and impact of the paper (or proposal) by offering specific constructive criticism (no name calling!) and complements (where appropriate) Structure Structure Summary Summary Major (or general) criticisms (typically <=3) Major (or general) criticisms (typically <=3) Minor (specific) criticisms (any number, specifically listed by page and line number) Minor (specific) criticisms (any number, specifically listed by page and line number)

13. PSYC512: Research Methods Addressing Reviewer’s Comments Never blame a reviewer for a negative review Never blame a reviewer for a negative review Consider all criticism as constructive—the reviewer is trying to help you! Consider all criticism as constructive—the reviewer is trying to help you! Assume misunderstandings are your fault, not the reviewers. All misunderstandings occur because you did not write clearly enough Assume misunderstandings are your fault, not the reviewers. All misunderstandings occur because you did not write clearly enough The cover letter (required with submission of revised proposal) The cover letter (required with submission of revised proposal) Specifically responds to the reviewers comments by Specifically responds to the reviewers comments by describing specific changes made in the paper to address criticisms describing specific changes made in the paper to address criticisms Presenting a rationale for why a reviewer’s criticism was not addressed Presenting a rationale for why a reviewer’s criticism was not addressed Must be worded very diplomatically! Must be worded very diplomatically!

14. PSYC512: Research Methods Presenting Research Time-allotted: 20 minutes  strictly enforced! Time-allotted: 20 minutes  strictly enforced! Use same general format as the written report Use same general format as the written report introduction (roughly 10 minutes) introduction (roughly 10 minutes) method (roughly 8 minutes) method (roughly 8 minutes) Summary (2 minutes or so) Summary (2 minutes or so) Given time allotted you cannot go into the same level of detail as your written report Given time allotted you cannot go into the same level of detail as your written report Materials Materials Talk from an outline of points you wish to make Talk from an outline of points you wish to make Visual Aids  power point Visual Aids  power point Elements of Style Elements of Style PRACTICE YOUR PRESENTATION! PRACTICE YOUR PRESENTATION! Anticipate questions and how you will answer them Anticipate questions and how you will answer them

15. PSYC512: Research Methods Using Inferential Statistics Which Statistic? Which Statistic? The statistical decision tree Howell Figure 1.1 The statistical decision tree Howell Figure 1.1 Testing for relationships vs. differences (a false distinction) Testing for relationships vs. differences (a false distinction) Relationships: assessing the strength of relationship between measured (dependent) variables Relationships: assessing the strength of relationship between measured (dependent) variables Differences: comparing different groups or treatments on some measurement Differences: comparing different groups or treatments on some measurement But what causes those differences? The relationship between the independent variable defining the groups or treatment and the dependent variable But what causes those differences? The relationship between the independent variable defining the groups or treatment and the dependent variable Hence, testing for differences is really testing the relationship between the IV and DV Hence, testing for differences is really testing the relationship between the IV and DV

16. PSYC512: Research Methods Analyzing Frequencies (Howell, Chapter 5) Bernoulli Trials: series of independent trials that result in one of two mutually exclusive outcomes Bernoulli Trials: series of independent trials that result in one of two mutually exclusive outcomes E.g. coin flips, gender of babies born, increase of decrease in a measure after application of a treatment E.g. coin flips, gender of babies born, increase of decrease in a measure after application of a treatment The Binomial Distribution The Binomial Distribution

17. PSYC512: Research Methods Analyzing Frequencies (Howell, Chapter 5) Using the binomial distribution Using the binomial distribution Mean number of successes = Np Mean number of successes = Np Variance in number of successes = Npq Variance in number of successes = Npq Testing Hypotheses using the binomial distribution: The Sign Test Testing Hypotheses using the binomial distribution: The Sign Test Ho is typically p= q =.50 (50-50 chance of success of failure), but that doesn’t have to be the case Ho is typically p= q =.50 (50-50 chance of success of failure), but that doesn’t have to be the case H1 is typically p ≠q H1 is typically p ≠q Plug in values for N, X, p, and q and p(X) directly provides the probability that the pattern of data could result given the null hypothesis is true Plug in values for N, X, p, and q and p(X) directly provides the probability that the pattern of data could result given the null hypothesis is true Sum the probabilities p(X) for all number >= X to get the total probability of finding p(>=X) Sum the probabilities p(X) for all number >= X to get the total probability of finding p(>=X) Important: The sign test takes into account direction of differences but not magnitude Important: The sign test takes into account direction of differences but not magnitude

18. PSYC512: Research Methods Analyzing Frequencies (Howell, Chapter 5) What about multiple (more than 2) possible outcomes? What about multiple (more than 2) possible outcomes? Multinomial distribution Multinomial distribution

19. PSYC512: Research Methods Analyzing Frequencies (Howell, Chapter 5) Using the multinomial distribution Using the multinomial distribution Mean X k = Np Xk Mean X k = Np Xk Variance in X k = Np Xk (1-p Xk ) Variance in X k = Np Xk (1-p Xk ) Testing Hypotheses using the multinomial distribution: Testing Hypotheses using the multinomial distribution: Ho is typically p X1 = p X2 … = p Xk = 1/k (each outcome has the same chance), but that doesn’t have to be the case Ho is typically p X1 = p X2 … = p Xk = 1/k (each outcome has the same chance), but that doesn’t have to be the case H1 is typically p X1 ≠ p X2 … ≠ p Xk H1 is typically p X1 ≠ p X2 … ≠ p Xk Plug in values for N, X, and p X, and p(X 1, X 2 …X k ) directly provides the probability that this particular pattern of data could result given the null hypothesis is true Plug in values for N, X, and p X, and p(X 1, X 2 …X k ) directly provides the probability that this particular pattern of data could result given the null hypothesis is true Must sum the probabilities for all patterns that deviate equal to or more to get the total probability – time consuming! Must sum the probabilities for all patterns that deviate equal to or more to get the total probability – time consuming!

20. PSYC512: Research Methods Analyzing Frequencies (Howell, Chapter 6) Easier Alternative to Multinomial distribution: Chi-square (    test Easier Alternative to Multinomial distribution: Chi-square (    test Compare computed value of   to value of   distribution with df=k-1 Compare computed value of   to value of   distribution with df=k-1 Expected frequencies for the null hypothesis typically = N/k, where N is the total number of observations Expected frequencies for the null hypothesis typically = N/k, where N is the total number of observations k is the number of categories in the variable O is the observed frequency for each category E is the expected frequency for each category i is the category index

21. PSYC512: Research Methods Analyzing Frequencies (Howell, Chapter 6) Using   with multiple dimensions: contingency tables—frequencies of one dimension are contingent on the other dimension Using   with multiple dimensions: contingency tables—frequencies of one dimension are contingent on the other dimension E ij = R i C j /N E ij = R i C j /N N is the total number of observations N is the total number of observations Compare computed value of   to value of   distribution with df=(R-1)(C-1) Compare computed value of   to value of   distribution with df=(R-1)(C-1) R is the number of categories in the dimension defined by the rows of the table C is the number of categories in the dimension defined by the columns of the table O is the observed frequency for each category E is the expected frequency for each category i and j are category indices

22. PSYC512: Research Methods Assumptions of the    test Assumptions of the    test Each observation is independent Each observation is independent Inclusion of non-occurrences Inclusion of non-occurrences Analyzing Frequencies (Howell, Chapter 6)