Scientific Research in Education NRC Committee on Scientific Principles in Education Research Editors: Richard J. Shavelson Lisa Towne

Why Are We Here? Castle bill, fall 2000 Office of Educational Research and Improvement re-authorization OERI disbanded in 2002 Education Sciences Reform Act Now Institute of Education Sciences Encourage self-reflection

Is That All? Thought about a federal agency Did NOT evaluate current research or techniques

Scientific Method “Guiding Principles” Pose significant questions -- empirical investigation Link research to relevant theory Use methods w/ direct investigation Chain of reasoning Replicate and generalize Professional scrutiny

Properties of Education Values and Politics Differing views on human potential No single goal to current research Personal values too involved Decide on one goal, not “best” or “final” goal Governmental changes

Properties of Education Human Volition Individual desires confound experiments Parents are whiny and hate science Think of the children Think of them! Mobility of U.S. population 10%/year ?

Properties of Education Variability of Education Programs “Tinkering towards Utopia”-Tyack & Cuban 95 State standards Organization Group-Class-Grade-School-District-County-State-Federal Cultural Diversity

Properties of Education Research Multidisciplinary Integrate/correlate from different fields Advances depend on advances in other fields Development of training programs is hard

Properties of Education Research Ethical Considerations Cannot create education-free control Cannot hide purpose (for double-blind) Somebody think of the children! Reliance on Practitioners Recent shift to long-term partnered research

Designing Research Question dictates design Three important facets: Description: what is happening? Cause: is there a systematic effect? Mechanism: why/how is it happening?

Description Collect raw valid data Random sampling (surveys and tests) Reproducible valid results In-depth case studies 1970s: 23 college women Career in major, career outside major, housefrau Schoolwork predictor

Causation Find causal relationships between variables Isolate variables on random surveys/tests Non-random tests may generalize better Real-world education not dictated randomly Harder to validate - “selectivity bias” Model-fitting Salary vs. dropouts

Mechanism Bring all techniques to bear on “how” 10-year study on Catholic schools Three models: Sector (private spiritual nature) Compositional (composition of student body) School (feature of school operation) Conclusion: “coherence” (sense of community)

Mechanism: When Theory is Weak College women study Why did nontraditional majors not have careers? Design study of ratio reasoning “Real” problems (ramps, exchange rates) Videotape of classroom activities Compared to other classrooms over 3 years Produced refined theoretical model

Designing An Ideal Federal Agency An Exercise in Publicly Funded Common Sense Staff with Scientists Structures to Guide Agenda Insulate from Politics Balanced Research Portfolio Adequate, Consistent Funding Invest in Private Research Infrastructure