1. R.G. Bias | rbias@ischool.utexas.edu | Name that tune. Song title? Performer(s)? 1

2. R.G. Bias | rbias@ischool.utexas.edu | Scientific Method (continued) “Finding New Information” 3/29/2010 2

3. R.G. Bias | rbias@ischool.utexas.edu | Objectives  I want to arm you with a scientist’s skepticism, and a scientist’s tools to conduct research and evaluate others’ research.  Swoopin’ out of “scientific method” and “experimental design” and into “statistics.” -Randolph – remember to take roll.

4. R.G. Bias | rbias@ischool.utexas.edu | 4 Critical Skepticism  Remember the Rabbit Pie example from earlier?  The “critical consumer” of statistics asked “what do you mean by ’50/50’”?  Let’s look at some other situations and claims.

5. R.G. Bias | rbias@ischool.utexas.edu | 5 Company is hurting.  We’d like to ask you to take a 50% cut in pay.  But if you do, we’ll give you a 60% raise next month. OK?  Problem: Base rate.

6. R.G. Bias | rbias@ischool.utexas.edu | 6 Sale!  “Save 100%”  I doubt it.

7. R.G. Bias | rbias@ischool.utexas.edu | 7 Probabilities  “It’s safer to drive in the fog than in the sunshine.” (Kinda like “Most accidents occur within 25 miles of home.” Doesn’t mean it gets safer once you get to San Marcos.)  Navy literature around WWI: –“The death rate in the Navy during the Spanish- American war was 9/1000. For civilians in NYC during the same period it was 16/1000. So... Join the Navy. It’s safer.”

8. R.G. Bias | rbias@ischool.utexas.edu | 8 Are all results reported?  “In an independent study [ooh, magic words], people who used Doakes toothpaste had 23% fewer cavities.”  How many studies showed MORE cavities for Doakes users?

9. R.G. Bias | rbias@ischool.utexas.edu | 9 Sampling problems  “Average salary of 1999 UT grads – “$41,000.”  How did they find this? I’ll bet it was average salary of THOSE WHO RESPONDED to a survey.  Who’s inclined to respond?

10. R.G. Bias | rbias@ischool.utexas.edu | 10 Correlation ≠ Causation  Around the turn of the 20 th century, there were relatively MANY deaths of tuberculosis in Arizona.  What’s up with that?

11. R.G. Bias | rbias@ischool.utexas.edu | 11 Remember...  I do NOT want you to become cynical.  Not all “media bias” is intentional.  Just be sensible, critical, skeptical.  As you “consume” statistics, ask some questions...

12. R.G. Bias | rbias@ischool.utexas.edu | 12 Ask yourself...  Who says so? (A Zest commercial is unlikely to tell you that Irish Spring is best.)  How does he/she know? (That Zest is “the best soap for you.”)  What’s missing? (One year, 33% of female grad students at Johns Hopkins married faculty.)  Did somebody change the subject? (“Camrys are bigger than Accords.” “Accords are bigger than Camrys.”)  Does it make sense? (“Study in NYC: Working woman with family needed $40.13/week for adequate support.”)

13. R.G. Bias | rbias@ischool.utexas.edu | We run an experiment...  to try to see if two (or more) levels of an IV differentially influence a DV.  We hope to find a difference.  Finding NO difference can mean one of two things: –Truly there’s no difference. –Our test – our experiment – just wasn’t good enough, or sensitive enough, to detect the difference. 13

14. R.G. Bias | rbias@ischool.utexas.edu | 14 Notice  Many things influence how easy or hard it is to discover a difference. –How big the real difference is. –How much variability there is in the population distribution(s). –How much error variance there is. –Let’s talk about variance.

15. R.G. Bias | rbias@ischool.utexas.edu | 15 Sources of variance  Systematic vs. Error –Real differences –Error variance  What would happen to the DV if our measurement apparatus was a little inconsistent?  There are OTHER sources of error variance, and the whole point of experimental design is to try to minimize ‘em. Get this: The more error variance, the harder for real differences to “shine through.”

16. R.G. Bias | rbias@ischool.utexas.edu | 16 Role of Data Analysis in Exps.  Primary goal of data analysis is to determine if our observations support a claim about behavior. Is that difference really different?  We want to draw conclusions about populations, not just the sample.  Two different ways – statistics and replication.

17. R.G. Bias | rbias@ischool.utexas.edu | And so... ... we make the transition to statistics!! (Yay!) 17

18. R.G. Bias | rbias@ischool.utexas.edu | 18 How to talk about a set of numbers  We can list ‘em. –Can get WAY unwieldy. –Plus hard to make any sense out of them.  First step – put ‘em in order.  Second step – –Graph ‘em, and/or –Calculate percentiles/deciles

19. R.G. Bias | rbias@ischool.utexas.edu | 19 Frequency Distributions - Histograms  # of pets ever owned –13 –2 –1 –4 –0 –1 –3 –0 –5 –1  Put ‘em in order – 0 – 1 – 2 – 3 – 4 – 5 – 13

20. R.G. Bias | rbias@ischool.utexas.edu | 20 Freq Dist  Raw Scores (in order) –0 –1 –2 –3 –4 –5 –13 Raw Score Freq Cumu Freq 022 135 216 317 418 519 13110

21. R.G. Bias | rbias@ischool.utexas.edu | 21 Histogram

22. R.G. Bias | rbias@ischool.utexas.edu | 22 Percentiles  LOCATION of 25 th percentile: –X.25 = (N+1).25  LOCATION of 50 th percentile: –X.50 = (N+1).50  LOCATION of 75 th percentile: –X.75 = (N+1).75  Example: If we had 10 scores, –the 25 th percentile would be the (11).25=2.75 th score or part way (half way!) between the 2 nd and 3 rd scores. –The 50 th percentile would be the (11).50=5.5 th score, or half way between the 5 th and 6 th scores.

23. R.G. Bias | rbias@ischool.utexas.edu | 23 Example  # of pets ever owned –13 –2 –1 –4 –0 –1 –3 –0 –5 –1  Put ‘em in order – 0 – 1 – 2 – 3 – 4 – 5 – 13

24. R.G. Bias | rbias@ischool.utexas.edu | 24 Note...  With an odd number of scores, the 50 th percentile will be an actual score:  Raw Scores (in order) –0 –1 –2 –3 –4 –5 –13 –100  50 th percentile = (N+1).50 = (12).5 = 6 th score = 2.

25. R.G. Bias | rbias@ischool.utexas.edu | 25 Let’s calculate some “averages”

26. R.G. Bias | rbias@ischool.utexas.edu | 26 A quiz about averages 1 – If one score in a distribution changes, will the mode change? __Yes __No __Maybe 2 – How about the median? __Yes __No __Maybe 3 – How about the mean? __Yes __No __Maybe 4 – True or false: In a normal distribution (bell curve), the mode, median, and mean are all the same? __True __False

27. R.G. Bias | rbias@ischool.utexas.edu | 27 Some rules... ... For building graphs/tables/charts: –Label axes. –Divide up the axes evenly. –Indicate when there’s a break in the rhythm! –Keep the “aspect ratio” reasonable. –Histogram, bar chart, line graph, pie chart, stacked bar chart, which when? –Keep the user in mind.

28. R.G. Bias | rbias@ischool.utexas.edu | References  Hinton, P. R. Statistics explained.  Shaughnessy, Zechmeister, and Zechmeister. Experimental methods in psychology. 28