Choosing Analyses and Reviewing for Exam HDFS 8200: Research Methods in HDFS (MANFRA) FALL 2011

Choosing Analyses Are you exploring parametric (normal distribution, homoscedasticity) or nonparametric (count/frequency; non-normal distribution) data? Understand your variables –How many independent and dependent variables do you have? –Are your dependent variables discrete or continuous? –Are your independent variables discrete or continuous? Don’t forget other factors/variables you need to control for—they should be included as IVs

Common Analyses IVDVAnalysis --1 continuousOne-sample t-test 1 discrete (2 levels)1 continuousIndependent samples t-test 1+ discrete (2+ levels)1 continuousANOVA 1+ discrete (2+ levels) & 1+ continuous 1 continuousANCOVA 1 continuous Pearson correlation 1+ continuous or dichotomous (discrete) 1 continuousRegression 1+ discrete (2+ levels)2+ continuousMANOVA 1+ discrete (2+ levels) & 1+ continuous 2+ continuousMANCOVA 1 discrete (2+ levels)Count/frequencyChi-square 2 discrete (2+ levels)Count/frequencyTwo-way chi-square

Comparative vs. Associational In cases where you are comparing groups—or levels of the IV—you are conducting what can be referred to as group differences analyses –A t-test has two levels of the IV (therefore two groups) and compares the group mean differences of the DV for each of these two levels of the IV –An ANOVA has two-plus levels of the IV and compares the group means of the DV between several groups (2 or more) In cases where you are making an association between variables—typically when you have two continuous variables—you are conducting a correlational analysis –A Pearson correlation explores the magnitude of the relation between two continuous variables

Example 1 A restaurant manager wants to know if dinners spend more money per month on special dishes or regular menu items. –Parametric data – not looking at count/frequency data –1 DV: $$/mos Continuous variable –1 IV: Type of Dish (Special vs. Regular) Discrete variable with 2 levels No other IV needed for statistical control Most parsimonious analysis: independent sample t-test

Example 2 Do more people shop at the Mall (vs. any other shopping store front) during the Holiday season compared to summer? –Non-parametric data – looking at count/frequency of shoppers (not distributed with variance around mean) –DV: No real DV Count/frequency data –2 IVs: Location (Mall vs. Other) and Season (Holiday vs. Summer) Both discrete variable with 2 levels No other IV needed for statistical control Most parsimonious analysis: two-way Chi- Square (Χ 2 )

Example 3 A school is interested in knowing if children in team- taught classes score better on the reading test compared to children with one teacher. It is believed that the age of the child will influence the reading test scores, so they wish to take age into account. –Parametric data – not looking at count/frequency data –1 DV: reading test scores Continuous variable –2 IVs: (1) main IV of focus and (2) control factor (1) Team-Taught (Yes vs. No) –Discrete variable with 2 levels (2) Age of child –Continuous variable Most parsimonious analysis: ANCOVA (Analysis of Covariance)

Degrees of Freedom Definition: number of sample data points free to vary (take on any value) when estimating a population parameter from that sample Calculation: DF = # of knowns - # of estimates Example: –40 individuals take a stress test; 20 males, 20 females. You want to compare the stress levels of males to females. –Calculate a mean for males and a separate mean for females to test the null hypothesis (H0: μ males – μ females = 0). Note that the null hypothesis is written about the population parameters—i.e., the parameters you are estimating in the population –Calculate degrees of freedom. DF = 40 – 2 = 38 What are the # of knowns? 40 What are the # of estimates? 2 –What type of statistical test could you use to analyze these data?

Importance of Variable Type How do discrete vs. continuous variables affect the degrees of freedom? Consider the following example: –A researcher is interested in knowing if divorced men have less disposable income than married men. The research asks the men in his sample to report the amount of money (to the nearest dollar) that they have in disposable income (after all debt paid) per month. –The IV (attribute or active?...thus experiment or nonexperiment??) is relationship status (married vs. divorced). This is discrete with 2 levels. A comparison between these two groups will be made. –The DV?? If the research leaves the values as real numbers, continuous.

Importance of Variable Type What if 12 participants were asked to select an income range from a survey? 0-$500 (0)$500-$750 (1)$750-$1000 (2) $1000-$1250 (3)$1250-$1500 (4)>$1500 (5) Type of variable now? Ordinal, 6 levels Is it discrete or continuous? –If continuous, then the researcher would estimate one mean per relationship status group based on the average of the 6 categories—represented by 0, 1, 2, 3, 4, 5 DF = 12 – 2 = 10 –If discrete, then the income variable would be conceived as 5 separate variables ($500-$750, Y/N; $750-$1000, Y/N; $1000-$1250, Y/N; $1250-$1500, Y/N; >$1500, Y/N—when all are N, then $0-$500) –The research would need to estimate one mean per each of the 5 separate variables—for each of the two relationship status groups DF = 12 – (5*2) = 12 – 10 = 2

Making the Statistical Decision When to reject the null hypothesis vs. fail to reject null hypothesis? In statistic programs, simply look at calculated p-value p-value calculation is based on comparison of the calculated statistic value with a known distribution—could be the F distribution or the t distribution (or other) These distributions vary in a known way depending on the degrees of freedom –A p-value of.05 is reached with 2 degrees of freedom when the t-value is 4.30 –A p-value of.05 is reached with 10 degrees of freedom when the t-value is 2.23 This is a HUGE difference in terms of t-values! –The equivalent value for a normal distribution (when DF = ∞) is 1.96 DF = 30 :: 2.04ORDF = 120 :: 1.98 (pretty close) If the calculated t-value is greater (in either direction if nondirectional) than these critical t-values, the research rejects the null hypothesis

Test Your Knowledge: Together For the following scenario, address the following: –Research question, hypothesis (substantive and symbolic for H0 and H1), variables needed to be measured, method of measuring variables, variable types (discrete, continuous), population (target, accessible), sample (realistic), study design, parameter estimates, degrees of freedom, statistical procedure, statement if null hypothesis is rejected –Don’t forget to take ethical considerations into account You are interested in knowing the effects of preschool quality (high vs. low) on learning for children from high SES homes and low SES homes

Test Your Knowledge: Independent For the following scenario, address the following: –Research question, hypothesis (substantive and symbolic for H0 and H1), variables needed to be measured, method of measuring variables, variable types (discrete, continuous), population (target, accessible), sample (realistic), study design, parameter estimates, degrees of freedom, statistical procedure, statement if null hypothesis is rejected –Don’t forget to take ethical considerations into account You are interested in knowing if rates of depression increase as disposable income decreases in the elderly

EXAMINATION NOTES Online (hopefully) or ed to you and posted Two parts: (1) out of class and (2) in class Out of class (11/11): –Time limit of 3 hours to complete (between Fri at Noon and Saturday at Noon) –Fill in, short answer, short essay, essay –Typed –Open book, open notes In class (11/18): –Time limit 30 minutes to complete (during class time only) –Multiple choice, true/false, identification, short answer –Closed book, closed notes

Q and A Time NOTE: you should be thinking of all of these components in relation to the other. For example, the stated hypothesis impacts the design and the statistical analysis.

Review Questions (OR Sample Test Questions) Why is measurement reliability important for Type II error (β)? How does limiting your accessible population to a small city in the mid-west affect a studies external validity? How do symbolically written hypotheses indicate what statistical procedure to run? Why can’t constructs be measured in a research study? Can a statistical analysis have zero (0) degrees of freedom? How can the design of a study improve internal validity? How does the appropriate ethical practice of having people volunteer for studies negatively impact the field? What issues are there with using null hypothesis testing to understand the “truth”? Can a researcher merely re-conceive a ratio measured variable as an interval, ordinal, or dichotomous variable? If so, why might s/he want to do this?