1. Education Research: The Fourth “R”

2. Hollyyettick.co m Introduction
Holly Yettick, Director, Education Week Research Center
All resources mentioned during this presentation can be found at:
Hollyyettick.co m

3. Agenda
Overview: Five Mistakes Education Journalists Make When Covering Ed Research
Work Shop: News Release Activity
Conclusion: Group Discussion of News Release Activity

4. Five Mistakes Education Journalists Make When Covering Research

5. Mistake #1. Ignoring research, peer reviewed research
Mentioned Ed Research

6. Why Cover Research? Adds context Breaks news Breaks LOCAL news
Holds public officials accountable
Un-named conference; how many of you would be interested in learning about the longterm effects of a local test cheating scandal on the students whose scores were falsely inflated? What about finding out what really happened to kids who attended a school that was shut down for poor academic performance?

7. Why cover peer-reviewed research?

8. The Tour Guide Approach
HIGHLY RECOMMENDED!
Universities
EWA Resources
List-servs
Public editor
Organizations for researchers
American Educational Research Organization
Special Interest Groups: Officers can be a source of academic expertise
American Psychological Association
American Sociological Association

9. “Tourist Traps” The Eduttante The Nihilist
Sometimes I hear people talking about A LIST OR ONE LIST of experts to consult when covering ed research. The reality is no one is an expert on everything. If you have someone who claims to be or comes close, who never refers you to other experts because it’s not her area-that is a red flag.

10. DIY Approach A few places to start AERA Open
Educational Evaluation & Policy Analysis
American Educational Research Journal

11. I guarantee you will find something useful here
Annual Research Conference of the Association for Education Finance and Policy
Annual Conference of the American Educational Association
ex.php

12. Mistake #2: Ignoring research reviews
Review of Educational Research
com/toc/rer/current

13. Mistake #3: Focusing only on sample size, ignoring selection
1936 Literary Digest poll of 2.4 million Americans predicts Alfred Landon will beat Franklin D. Roosevelt

14. Mistake #4: Ignoring effect size
Questions to Answer in Your Story
Ask the researcher
How big was the effect, change, or difference?
Ask teachers/instructors, ed leaders:
Is [X] worth implementing given the size of the effect and the associated tradeoffs and costs?

15. Mistake #5: Over-generalizing Conclusions
Examples
AGE: Applying findings from higher ed to K-12, and vice versa.
NATIONS: Applying findings from one country to another
SOCIO-ECONOMIC STATUS: Applying findings from wealthy communities to low-income communities, and vice versa.

16. What about the real world?
PRACTICE WITH A PRESS RELEASE

17. Discussion Questions: Admissions Prep Study
HOLLYYETTICK.COM
The headline of the press release reads Admissions Prep Boosts ACT Scores and Gets Students into College. What do you make of that claim?
What can this study tell us about the success rate of late- night classes for high school students?
Are the results of the study really statistically significant?
What do you think of Jane Jones’ suggestion that every high school in America should adopt Admissions Prep?
What would you do if your editor insisted that you write about the Admissions Prep study? What questions would you ask? Who might you call?

18. Supplemental slides

19. Statistical significance

20. What is a p value?
P Value
What are the odds that I can REJECT the idea that the researcher got a result at least as this strong [i.e., did the equivalent of telling a man he’s pregnant] due to some kind of chance process that was part of the study-like randomly selecting the people who participated in the study?
Frequently-used thresholds of statistical significance
p < .01
p < .05
p < .10

21. Statistical significance
Not applicable if no chance process involved
Large numbers of study participants can lead to highly statistically significant findings that are meaninglessly small

22. The Best Academic Conference You’ve Never Heard of
Association for Education Finance and Policy annual conference

23. Director, Education Week Research Center
Contact info
Holly Yettick
Director, Education Week Research Center
@HollyYettick