Howard University February 13,2013 The Integrity of Data Management Responsible Conduct of Research Workshop Peggy Carr, Ph.D. Associate Commissioner National Center for Education Statistics 1

Overview of Statistical and Research Standards 2

Best Practices for the Conduct of Research 3

1.Conceptualize Research Problem 2.Operationalize Study Design 3.Engage in Peer Review and Networking Activities 4.Publish and Publish Early/Often 5.Be Ethical 4

1. Conceptualize Research Problem 5 Literature Review Statement of Problem Hypotheses Research Question Definition of Terms, Concepts, and Constructs Conceptual Operational

2a. Operationalize Study Design The Concept of Study Design Hypothesis 1 Explained Variance Unexplained Variance Sampling Error Measurement Error Validity Reliability

TEL Video

Validation of Constructs Face – Content validation across judges Criterion – Predictive and concurrent Construct – Methodological approaches to validation 8 Jealousy Envy 2b. Operationalize Study Design

2c. Operationalize Study Design Reliability of Measures Test and Retest – stability over time Equivalency – consistency across forms Internal – internal consistency 9 UNRELIABLERELIABLE

2c. Operationalize Study Design Research Design Internal Validity – refuting rival plausible hypotheses External Validity – generalizability of findings Choice of Designs – experimental, quasi, survey, correlational 10

Quality Assurances of Data Data Checks (e.g., out of range variance, criterion reference checks, replication) Thoughtful and Appropriate Data Analysis The nature of your data Limitations of statistical technique Discuss Findings and Conclusions with Colleagues Relate back to literature review Implications for future research Flaws and limitations 11 NEXT 2c. Operationalize Study Design

Trend in fourth-grade NAEP mathematics average scores 12 2c. Operationalize Study Design SOURCE: The Nation’s Report Card, 4 th Grade NAEP: Mathematics,

13 2c. Operationalize Study Design Trend in fourth-grade NAEP mathematics percentile scores SOURCE: The Nation’s Report Card, 4 th Grade NAEP: Mathematics,

3. Engage in Peer Review and Networking Activities Find a specialty and stick with it Attend professional meetings and conferences – share your research Keep an open mind and learn from others 14

4. Publish and Publish Early/Often Start your publication record early in peer reviewed journals As first, second, or third author Publish and/or present often Regardless of professional focus Annually 15

5. Be Ethical Be committed and follow through Adhere to Subjects’ Rights Institutional Review Board Confidentiality and Privacy Rights 16

Questions? 17