1. Correlation & Causal Comparative Research Class 6

2. This Week’s Schedule Today – Review and continue w/ statistical analysis Tuesday – 3 individual meetings 9-10am – Full class (Stats & Method) 10-11am – 3 individual meeting 11am-12noon Wednesday – 9am-10am – Music ed history (Skype w/ Eastman Class) Show and tell!! – 10:00am-11:00am – Qualitative Research-Full class – 11:00am-12noon-3 individual meetings

3. This Week’s Schedule Thursday – 4 Project Presentations (20 minutes) – 2 qualitative/historical 5ish minute presentations (in pairs & a trio) – Disseminating research Friday – 5 Project Presentations – 2 qualitative/historical 5ish minute presentations (in pairs & a trio)

4. Assignments Tuesday – Work on presentations, projects, etc. Wednesday – Read Queen Bees and Wanna Bees chapt. 1 OR 6 – Read one historical article from the Journal of Historical Research in Music Education. Be prepared to write or discuss – Chapter 3 – Method Thursday & Friday – Project Presentations (20 minute w/ 5 minutes for questions & discussion) – Informal presentation in pairs of a qualitative or an historical article Monday, July 22 by 5pm – Final Project Proposal

5. Who & When Tuesday Meetings – 9-10 (3) – 11-noon (3) Weds Meetings – 11-noon (3) Thursday – Project Presentations (4) – Qual./Hist. presentations (2 pairs) Friday – 5 presentations – Qual./Hist. trio presentation

6. APA Format Headers – Chapter title = Level 1 – Others = Level 2 (Flush left) Remember title page, page #s Running Head – < 50 characters total. Goes in the header flush left Research Question after purpose statement & before need for study (Header?) Commas = …apples, oranges, and grapes.

7. Types of Data – Revised simple to complex; lowest to highest Nominal/Categorical = numbers as labels – Male/female (1 or 2) – Sop/Alto/tenor/bass (1, 2, 3, 4) Ordinal = ranks – Contest ratings Interval = Scale (equal distance b/w each number) – Contest scores (1-100) – Lack of meaningful zero (0 on test = no knowledge?, 0 temperature = arbitrary) or meaningful ratios (2x as smart?) Ratio = – Equal interval data – True zero possible (0 decibels, 0 money) – Ratios can be calculated in a meaningful way [2x as loud, ½ money, height, weight, depth (a lake can dry up) (?), etc.]

8. Terms Inferential statistics Parametric vs. non-parametric Assumptions/Parameters – Variances? – Randomization Mean vs. variance Used to compare 2 groups and no more? Independent vs. dependent (paired or correlated) One tail vs. Two tail tests?

9. Terms What is I have more than 2 groups? I need a…? If there is a significant difference in the test above, then what do I need to do? Why do we test the significance of the difference in variances? What if the variances are sig. different?

10. Statistical Significance Probability that result happened by chance and not due to treatment – Expressed as p – p <.1 = less than 10% (1/10) probability… – p <.05 = less than 5% (1/20) probability… – p <.01 – less than 1% (1/100) probability… – p <.001 – less than.1% (1/1000) probability… Computer software reports actual p alpha level = probability level to be accepted as significant set b/f study begins Statistical significance does not equal practical significance

11. Statistical Power Likelihood that a particular test of statistical significance will lead to the rejection of null hypothesis – Parametric tests more powerful than nonparametric. (Par. more likely to discover differences b/w groups. Choice depend on type of data) The larger the sample size, the more likely you will be to find statistically significant effects. The less stringent your criteria (e.g.,.05 vs. 01 vs. 001), the easier it is to find statistical significance

12. Statistical Tests http://pspp.awardspace.com/http://pspp.awardspace.com/ (Windows) http://bmi.cchmc.org/resources/software/pspphttp://bmi.cchmc.org/resources/software/pspp (Mac) http://vassarstats.net/

13. See Handout from Friday Awareness of non-parametric tests 3 groups, ordinal data? 2 groups, interval data? 2 groups, nominal/categorical data? Relationship b/w two groups, ordinal data?

14. Independent Samples t-test Used to determine whether differences between two independent group means are statistically significant n = < 30 for each group. Though many researchers have used the t test with larger groups. Groups do not have to be even. Only concerned with overall group differences w/o considering pairs – [A robust statistical technique is one that performs well even if its assumptions are somewhat violated by the true model from which the data were generated. Unequal variances = alternative t test or better Mann-Whitney U] Application: Explore Data – Compare science tests of inst & non-inst. students

15. Correlated (paired, dependent) Samples t- test Used to determine differences between two means taken from the same group, or from two groups with matched pairs are statistically significant – e.g., pre-test achievement scores for the whole song group vs. post-test achievement scores for the whole song group Group size must be even (paired) N = < 30 for each group Application: Compare Reading & Math test scores of Instrumental Students

16. Compare 2 means Need sample of at least 10 Work like Independent and dependent t tests Independent – Mann Whitney U Application: Data set #3. Is there a sig. diff. b/w Final ratings at Site 1 vs. site 2? Pairs or dependent samples – Wilcoxon signed ranks Application: Data set #2. Is there a sig. difference b/w rating of judges 1 & 2?

17. ANOVA Analyze means of 2+ groups Homogeneity of variance Independent or correlated (paired) groups More rigorous than t-test (b/w group & w/i group variance). Often used today instead of T test. F statistic One-Way = 1 independent variable Two-Way/Three-Way = 2-3 independent variables (one active & one or two an attribute)

18. One-Way ANOVA Calculate a One-Way ANOVA for data-set 1 – All non- instrumental tests Post Hoc tests – Used to find differences b/w groups using one test. You could compare all pairs w/ individual t tests or ANOVA, but leads to problems w/ multiple comparisons on same data – Tukey – Equal Sample Sizes (though can be used for unequal sample sizes as well) – Sheffe – Unequal Sample Sizes (though can be used for equal sample sizes as well)

19. ANCOVA – Analysis of Covariance Statistical control for unequal groups Adjusts posttest means based on pretest means. [example] http://faculty.vassar.edu/lowry/VassarStats.ht ml http://faculty.vassar.edu/lowry/VassarStats.ht ml [The homogeneity of regression assumption is met if within each of the groups there is an linear correlation between the dependent variable and the covariate and the correlations are similar b/w groups]

20. Effect Size (Cohen’s d) http://www.uccs.edu/~faculty/lbecker/es.htm http://www.uccs.edu/~lbecker/ http://www.uccs.edu/~faculty/lbecker/es.htmhttp://www.uccs.edu/~lbecker/ [Mean of Experimental group – Mean of Control group/average SD] The average percentile standing of the average treated (or experimental) participant relative to the average untreated (or control) participant. Use table to find where someone ranked in the 50 th percentile in the experimental group would be in the control group Good for showing practical significance – When test in non-significant – When both groups got significantly better (really effective vs. really really effective! Calculate effect size: – Treatment group: M=24.6; SD=10.7 – Control Group: M=10.8; SD=7.77

21. Cohen's StandardEffect SizePercentile StandingPercent of Nonoverlap 2.097.781.1% 1.997.179.4% 1.896.477.4% 1.795.575.4% 1.694.573.1% 1.593.370.7% 1.491.968.1% 1.39065.3% 1.28862.2% 1.18658.9% 1.08455.4% 0.98251.6% LARGE0.87947.4% 0.77643.0% 0.67338.2% MEDIUM0.56933.0% 0.46627.4% 0.36221.3% SMALL0.25814.7% 0.1547.7% 0.0500%

22. Chi-Squared Measure statistical significance b/w frequency counts (nominal/categorical data) http://www.quantpsy.org/chisq/chisq.htm Test for independence: Compare 2 or more proportions Goodness of Fit: compare w/ you have with what is expected – Proportions of contest ratings (I, II, III or I & non Is) – Agree vs. Disagree Weak statistical test

23. Correlation Pearson Spearman Cronbach’s alpha (α)

24. Correlational Research Basics Relationships among two or more variables are investigated The researcher does not manipulate the variables Direction (positive [+] or negative [-]) and degree (how strong) in which two or more variables are related

25. Uses of Correlational Research Clarifying and understanding important phenomena (relationship b/w variables— e.g., height and voice range in MS boys) Explaining human behaviors (class periods per weeks correlated to practice time) Predicting likely outcomes (one test predicts another)

26. Uses of Correlation Research Particularly beneficial when experimental studies are difficult or impossible to design Allows for examinations of relationships among variables measured in different units (decibels, pitch; retention numbers and test scores, etc.) DOES NOT indicate causation – Reciprocal effect (a change in weight may affect body image, but body image does not cause a change in weight) – Third (other) variable actually responsible for difference (Tendency of smart kids to persist in music is cause of higher SATs among HS music students rather than music study itself)

27. Interpreting Correlations – r Correlation coefficient (Pearson, Spearman) Can range from -1.00 to +1.00 – Direction Positive – As X increases, so does Y and vice versa Negative – As X decreases, Y increases and vice versa – Degree or Strength (rough indicators) < +.30; small < +.65; moderate > +.65; strong > +.85; very strong – r 2 (% of shared variance) % of overlap b/w two variables percent of the variation in one variable that is related to the variation in the other. Example: Correlation b/w musical achievement and minutes of instruction is r =.86. What is the % of shared variance (r 2 )? – Easy to obtain significant results w/ correlation. Strength is most important

28. Application Rate your principal & school quality on a scale of 1-7 Principal: (1=highly ineffective; 2=ineffective; 3=somewhat ineffective; 4=neither effective nor ineffective; 5=somewhat effective; 6=effective; 7=highly effective School cleanliness: (1=very dirty; 2=dirty; 3=somewhat dirty; 4=neither dirty or clean; 5=somewhat clean; 6=clean; 7=very clean) Type of data? Calculation (Pearson or Spearman?) Reliability (Cronbach’s alpha) www.gifted.uconn.edu/siegle/research/.../reliabilitycalculator2.xls www.gifted.uconn.edu/siegle/research/.../reliabilitycalculator2.xls

29. Interpreting Correlations (cont.) Words typically used to describe correlations – Direct (Large values w/ large values or small values w/ small values. Moving parallel. 0 to +1 – Indirect or inverse (Large values w/small values. Moving in opposite directions. 0 to -1 – Perfect (exactly 1 or -1) – Strong, weak – High, moderate, low – Positive, Negative Correlations vs. Mean Differences – Groups of scores that are correlated will not necessarily have similar means (e.g., pretest/posttest). Correlation also works w/ different units of measurement. 50 75 9 40 62 14 35 53 20 24 35 45 15 21 58

30. Statistical Assumptions The mathematical equations used to determine various correlation coefficients carry with them certain assumptions about the nature of the data used… – Level of data (types of correlation for different levels) – Normal curve (Pearson, if not-Spearman) – Linearity (relationships move parallel or inverse) Non linear relationship of # of performances & anxiety scores = Young students initially have a low level of performance anxiety, but it rises with each performance as they realize the pressure and potential rewards that come with performance. However, once they have several performances under their belts, the anxiety subsides. ( – Presence of outliers (all) – Ho/mo/sce/da/sci/ty – relationship consistent throughout Performance anxiety levels off after several performances and remains static (relationship lacks Homoscedascity) – Subjects have only one score for each variable

31. Correlational Approaches for Assessing Measurement Reliability Consistency over time – test-retest (Pearson, Spearman) Consistency within the measure – internal consistency (split-half, KR-20, Cronbach ’ s alpha) – Spearman Brown Prophecy formula 2r/(1 + r) Among judges – Interjudge (Cronbach ’ s Alpha) Consistency b/w one measure and another – (Pearson, Spearman)

32. Reliability of Survey What broad single dimension is being studied? – e.g. = attitudes towards elementary music – Preference for Western art music – “People who answered a on #3 answered c on #5” Use Cronbach’s alpha – Measure of internal consistency – Extent to which responses on individual items correspond to each other

33. Spearman Brown Prophesy Formula Reliability = ___n x r___ 1+(n-1)r n=number of times we multiply items to get new test length (10 item to 20 item – n=2) For a test of 10 items w/ reliability (α) of.60 – (15 items) 1.5 x.60/1+(1.5 - 1).60 = reliability for test 1.5x size – (20 items) 2 x.60/1+(2-1).60 = reliability for a test 2x size – (25 items) 2.5 x.60/1+(2.5 – 1).60 = reliability for test 2.5x size – (5 items).5 x.60/1+(.5 – 1).60 = reliability for test.5 size