Research Design ED 592A Fall 2011

Research Concepts 1. Quantitative vs. Qualitative & Mixed Methods 2. Sampling 3. Instrumentation 4. Validity and Reliability 5. Internal Validity 6. External Validity 7. Design – Experimental & Correlational

Qualitative vs. Quantitative Qualitative: Investigate the quality of relationships, activities, situations, or materials. Quantitative: The systematic empirical investigation of social phenomena via statistical, mathematical or computational techniques.

Descriptive Statistics Mean - The mean can be defined as the arithmetic average of all values. The mean measures the central tendency of a variable. Significant Difference - In statistics, a result is called statistically significant if it is unlikely to have occurred by chance.

Even When you are Right you may be Wrong… The risk of not being 100% confident that what you observe in an experiment is due to the treatment or what is being tested. p <.05 (1 in 20 that any differences where not due to the treatment)

Sampling Sampling: Selecting those who will participate Population All members of a particular group to whom the researcher would like to generalize about Representative sample Sample that is similar to the population on all characteristics

Random Sampling Random Simple Random All members have an equal chance of being selected Stratified RS Certain characteristics are represented in the sample in the same proportion as they occur in the pop. Cluster RS Using groups as the sampling unit rather than individuals Two-Stage RS Select randomly and then choose individuals randomly from the groups

Purposive Sampling Purposive sampling targets a particular group of people. “Quasi” - design

Instrumentation The entire process of collecting data in a research investigation Data Kinds of information researchers obtain on the subjects of their research Classify Who will provide the data? What is your Method of data collection? Who will collect the data? What kind of responses are required? Self-report data (provided by the subjects of the study) Informant data (provided by other people about the subjects of the study)

Validity & Reliability Validity Appropriateness, meaningfulness, correctness, and usefulness of the inferences researchers make Reliability Consistency of scores or answers from one administration of an instrument to another and from one set of items to another

Internal Validity Any relationship observed between two or more variables should be unambiguous as to what it means rather than being due to “something else”. Selection Bias Individuals or groups differing from one another in unintended ways that are related to the variables to be studied Mortality Loss of someone as the study progresses

Internal Validity (continued) Instrumentation: The way instruments are used Decay – nature of the instrument, (including scoring procedure) is changed in some way or another Data collection Bias – unconsciously distort the data to make certain outcomes more likely

Internal Validity (continued) Location The location creates alternative explanations for the results Testing The impact of the pretest on the post-test results History Unanticipated and unplanned for events may occur during the course of a study that can affect the responses of subjects

Maturation Change during an intervention may be due to the factors associated with the passing of time rather than to the intervention itself Attitudes of Subjects Performing better because of the novelty of the treatment rather than the specific nature of the treatment. (i.e. Hawthorne effect) Implementation The experimental group is treated in ways that are unintended and not necessarily part of the method, yet give them an advantage of one sort or another

How to Reduce Threats Standardize conditions Obtain as much information as possible on the subjects Organize information on the details of the study Choose an appropriate design! Planning!

External Validity Can you make generalizations based on your findings…? External validity is the degree to which the conclusions in your study would hold for other persons in other places and at other times.

Experimental Design One-shot case study X 0 One-group pretest-posttest 0 X 0

Experimental Static-Group Comparison X Static-Group pretest-posttest 0 X

Correlational Research It is a measure of the association, or co- variation of two or more dependent variables. elational.html