1. Biases in Experimental Design: Validity, Reliability, and Other Issues

2. TAKE NOTE
Research studies with small sample sizes, high variability, and sampling bias are usually not representative of the general population.
Sampling bias is when the sample in question is not representative of the general population.
Selection bias occurs when the participants in the sample are not equally and fairly selected for both the experimental and control groups; this renders any results from the experiment meaningless.

3. Take Note
Response bias is when only highly motivated people return a survey. When this occurs, the resulting data is biased toward those with the motivation to answer and submit the survey, and is therefore not representative of the population as a whole.
External validity is the ability to apply conclusions gathered from the results of an experiment to the general population.
Data sets with little variability have values that are similar to each other; data sets with high variability have values that are more spread out.

4. Key Terms Reliability: overall consistency of a measure
External Validity: whether or not study findings can be generalized to real world scenarios
Bias: inclination, predisposition, or prejudice toward something
Law of Diminishing Returns: the tendency for a continuing effort toward a particular goal to decline in efficacy after a certain amount of success has been achieved
Key Terms

5. Overarching Ideas in Research
Research studies often fall prey to experimental bias, in which the results are not representative of what they are supposed to measure. This limits the applicability of the results to anything beyond the experiment itself, which decreases or eliminates the value of those results.

6. External Validity
A study that is externally valid is one in which the data and conclusions gathered from the results of an experiment can be applied to the general population outside of the experiment itself. If the study's data and conclusions cannot be applied to the general population, including general events or scenarios, then the experiment's results are only relevant to that experiment, and nothing more. A study's external validity can be threatened by such factors as small sample sizes, high variability, and sampling bias.

7. Small Sample Sizes
The smaller the sample size for an experiment, the less applicable the results will be to the general population. The world has some 7 billion individuals, and thus a representative sample in any experiment would have to be very large to be applied to this general population. Nonetheless, the larger the sample group is in relation to the general population to whom the results are to be applied, the more likely it is to be applicable.

8. Small Sample Sizes
This premise, however, can be negatively impacted by the law of diminishing returns, which states that effectiveness will decline after a certain amount of success has been achieved. This means that after a certain point, including more individuals in a study would gradually have less value to researchers. This could be caused by a multitude of factors, including cost and time put into the research. Generally it is best to attain a reasonable sample size that is representative of the population being studied.

9. High Variability
Variability, also known as dispersion or spread, refers to how spread out a group of data is, or how much the measures differ from each other. Data sets with similar values are considered to have little variability because the values are within a smaller spread, whereas data sets with values that are spread out have high variability because the values are within a larger spread. In many instances of high variability there are outliers, which are values that exist far outside of the area where the majority of values are found. In many cases these outliers, which increase the variability of the data set, are removed when conducting statistical analysis of the data.

10. Sampling Bias
Sampling bias occurs when the sample participating in the study is not representative of the general population. This may be the result of purposeful selection of participants by the researcher, but there are many other factors that can create sampling bias. One example is surveys taken during a presidential election. The results of the surveys often depend on the city, state, or area being surveyed. For example, people in cities tend to vote one way, while people in rural environments often vote another. Similarly, one's geographic location (the Northeast, South, Midwest, etc.) can have an impact on who is being surveyed. If there is a high saturation of a given political party in an area surveyed, then the results will be skewed in the direction of the political party, and not be representative of the general population.

11. Selection Bias
Selection bias happens when the comparisons in data from the sample population have no meaning or value because the participants in the sample were not equally and fairly selected for both the experimental and control groups. Both the experimental and control groups should be representative of the general population, as well as representative of each other. One group should not show substantially higher characteristics of a given variable than the other, as this can distort the findings.

12. Response Bias
Response bias (also known as "self-selection bias") occurs when only certain types of people respond to a survey or study. When this occurs, the resulting data is biased towards those with the motivation to answer and submit the survey or participate in the study. The resulting data, however, is not representative of the desired sample, nor the population at large. This is because only a select few have answered the survey and participated in the experiment. This data requires a disclaimer saying that out of all respondents, a certain characteristic is found. Regardless of a disclaimer, the results cannot be applied to the general population, nor the entire desired sample group.