Copyright, 2005, Prentice Hall, Sarafino CHAPTER 10 Preparing to Conduct a Study

Copyright, 2005, Prentice Hall, Sarafino The Developmental Phase The easiest part of any research is the actual gathering of data. –It is the easiest part because of all the planning and effort you have put into in it to ensure that things go smoothly. –The slides that follow will highlight the nine – yes 9 – phases that researchers must go through in order to ensure an efficient and true outcome of their research effort.

Copyright, 2005, Prentice Hall, Sarafino Phase 1: Identify the Research Topic and Hypothesis Things to consider: –Have you reviewed the ideas of other on this topic? Do your ideas mesh with those others? Why or why not? –Are there any theories relating to your topic? –Is your hypothesis testable and falsifiable? –Is your hypothesis theoretically supported by previous literature?

Copyright, 2005, Prentice Hall, Sarafino Phase 2:Determine Participants and Their Availability Do you need human or animal participants and how many do you need? Do your participants need to have unique qualities (e.g., have a phobia, or prone to develop a disease)? –If so, can you find enough participants? Will you be able to generate equal groups? –If not, what are the implications?

Copyright, 2005, Prentice Hall, Sarafino Phase 3: Choose Your Variables Are the variables you selected consistent with previous research? –How do I find reliable and valid measurement instruments to measure my variables? Mental Measures Yearbook; Test Critiques Are you going to manipulate your IVs? –If so, what levels will you select to maximize variance? Will your dependent variable(s) produce nominal, ordinal, interval, or ratio data?

Copyright, 2005, Prentice Hall, Sarafino What is your hypothesis asking? –Will an experimental, quasi-experimental, correlational, or descriptive strategy work best to support your hypothesis. Will you need a control group? If so, what kind? Will the participants be repeatedly measured? What statistical analyses will need to be performed? –Are there assumptions associated with running these analysis, are there limits to the interpretive power of these statistics? Phase 4: Choosing a Design

Copyright, 2005, Prentice Hall, Sarafino Try to predict unwanted sources of variance in your study and then implement control procedures. –If you think participant or experimental bias will influence your result then implement a double blind procedure. –If there may be order effects, make sure you counterbalance. How will you establish and maintain equivalent groups? Is it possible to hold the conditions of the experiment constant. –Will the different groups of participants really be treated identically except for the IV manipulation? Phase 5: Identify Sources of Error, and Controls

Copyright, 2005, Prentice Hall, Sarafino Phase 6: Identify Limitations on Generalization Will the types of participants used (e.g., students) limit the study’s external validity? Does the environment in which the study is run (e.g., laboratory) limit external validity? How far will your statistical analyses allow you to generalization?

Copyright, 2005, Prentice Hall, Sarafino Phase 7: Ethical Risks and Debriefing of Participants Ask yourself the following questions: Will or might your study: –Physically or psychologically harm? –Use deception? Violate privacy? –Use coercion? Fail to use informed consent? If you answer yes to any of these – ask yourself if they can be avoided, if not, be sure to include measures in your debriefing to address and/or minimize any possible lasting effects.

Copyright, 2005, Prentice Hall, Sarafino Phase 8: Design An Informed Consent Form Informed consent is usually required in any study with human participants. –Informed consent involves providing participants with the relevant details of the study, what they might experience, and that they can withdraw from the experiment at any time. Once participants are informed, then their written consent is obtained to participant in the research. –See example in your textbook.

Copyright, 2005, Prentice Hall, Sarafino Phase 9: Submit a Proposal for IRB Approval IRB = Institutional Review Board The proposal often includes: –Detailed descriptions of the purpose, methods to be use, the expected results, and the implications of the results. The previous 8 phases should provide you will enough information to submit a good proposal.

Copyright, 2005, Prentice Hall, Sarafino Pilot or Trail Runs Once you have received all the necessary approvals, it is time to run your study. Many studies include a consolidation phase to debug the study before the actual data collection begins. –This is often called a pilot study or a trail run of the proposed research.

Copyright, 2005, Prentice Hall, Sarafino Pilot Study A pilot study is a trail run of your entire experiment with participants – the participants in your pilot usually are not included in the actual study. The pilot study will help you: –Isolate unforeseen problems and fix them –Figure out how long the testing takes –Identify problems with test materials and/or variables. Any of the above may lead to a redesign of the study, a slight change in the operational definitions of variables, or even an altering of the hypotheses.

Copyright, 2005, Prentice Hall, Sarafino A Crucial Problem: Sampling After debugging your study in the pilot phase, it is now time to collect data. One of the more important aspects you’ll need to consider is how you will recruit and select your participants. This is called sampling. –Sampling, if done correctly, will enhance external validity. But if done incorrectly will significantly decrease your study’s external validity. –The goal of sampling is to generate a representative sample of your target population.

Copyright, 2005, Prentice Hall, Sarafino Types of Sampling There are two main categories of sampling: –Probability Sampling The odds of being selected from a population to a research study can be calculated. –Nonprobability Sampling The odds of being selected from a population to a research study cannot be calculated

Copyright, 2005, Prentice Hall, Sarafino Probability Sampling Probability samples almost always start with a “sampling frame” – a list of all individuals from which the sample will be selected. In a best scenario, all members of the target population will be on the list. There are several types of probability sampling: –Simple Random Sample Each individual in the population has an equal chance of being selected (e.g., names out of a hat). –Stratified Random Sample The sample frame is partitioned in some manner and simple random sampling is done on each partition. Why? –Cluster Sample Units or clusters of similar individuals are identified. Some clusters are randomly selected, and each individual in each of the selected clusters is measured.

Copyright, 2005, Prentice Hall, Sarafino Nonprobability Sample In nonprobability sampling a sampling frame is not present. –As a result, nonprobability samples may not be representative of the target population. The most common type of nonprobability sample (and the most common in psychology) is called a “convenience sample” Convenience samples are qualified volunteers who are easily available. You were likely part of a convenience sample if you participated in research while enroll in your first year Introduction to Psychology course. –If a convenience sample is used, what should researchers report regarding the convenience sample? –How will a convenience sample affect external validity?

Copyright, 2005, Prentice Hall, Sarafino Descriptive Statistics Descriptive stats provide information about the central tendencies of a group of data. Importance terms include: –Mean: the arithmetic average in a data set –Median: the middlemost score in a data set –Mode: the most frequent score in a data set –Variance: the degree to which scores in a data set deviate from the mean. There are various ways to measure variance or variability. Range: measure of variability in a data set Standard Deviation: most commonly used measure of variability.

Copyright, 2005, Prentice Hall, Sarafino Inferential Statistics Inferential statistics are mathematical / statistical procedures for determining the probability that the relationships or differences we observe in our data actually occur in the population. Inferential statistics also tell us whether the differences we see in our data occurred by chance or not. There are 2 types of inferential statistics: –Parametric statistics –Nonparametric statistics

Copyright, 2005, Prentice Hall, Sarafino Parametric and Nonparametric Statistics Parametric Statistics test hypotheses that are based on data that allow us to estimate parameters (e.g., means and standard deviations). –In other words, parametric statistics are used with interval or ratio data. –E.g. Pearson r, multiple regression, t-test, ANOVA Nonparametric Statistics test hypotheses that do not involve parameters (e.g., when the data are nominal or ordinal, or not normally distributed). –E.g., Spearman rank correlation, Chi-square.

Copyright, 2005, Prentice Hall, Sarafino Statistical Significance What do we mean when we say something is statistically significant? –We are simply saying that there is only a small probability that what we found was due solely to chance. –That small “chance” probability goes by various names: Type I Error Alpha level And is often symbolized by an italicized p “p” The alpha level is predetermine before the study begins and in psychology the level is usual set to 5% or –This means that if the results we gather cannot be obtained by chance more that 5 times in 100 random trials we would say that our results are statistically significant. It is unlikely they occurred solely by change. So can we ever be wrong?