A Comprehensive Unit Assessment Plan Program Improvement, Accountability, and Research Johns Hopkins University School of Education Faculty Meeting October 26, 2012 Toni Ungaretti Borrowed generously from Jim Wyckoff (October 10, 2010). Using Longitudinal Data Systems for Program Improvement, Accountability, and Research. University of Virginia

Why Assessment? Assessment is a culture of continuous improvement that parallels the School’s focus on scholarship and research. It ensures candidate performance, program effectiveness, and unit efficiency.

Overview Program Improvement: By following candidates and graduates both during their programs and over time after graduation, programs can learn a great deal about their programs Accountability: Value-added analysis of teacher/student data in longitudinal databases is one measure of program accountability Research: A systematic program of experimentally designed research can provide important insights in how to improve candidate preparation

Jim Wyckoff, 2010

Program Improvement Some Questions Who are our program completers—age, ethnicity, areas of certification? What characterizes the preparation they receive? How well do they perform on measures of qualifications, e.g., licensure exams? Where do our program completers teach/work? What is their attrition?—are they meeting program goals and mission? How effective are they in their teaching/work? Ultimate impact!

Accountability What Constitutes Effective Teacher Preparation? Programs work with school districts to meet the teaching needs of the schools where their teachers are typically placed Programs are judged by the empirically documented effectiveness of their graduates in improving the outcomes of the students they teach Retention plays a role in program effectiveness as teachers substantially improve in quality over the first few years of their careers.

Research How Can Programs Add Value ? Selection: Who enters, how does that matter, and how can we influence it? Preparation: What preparation content makes a difference? Timing: Does it matter when teachers receive specific aspects of preparation? Retention: Why is retention important to program value added and what can affect it?

Johns Hopkins University School of Education Comprehensive Unit Assessment Plan

What students learn Program Goals include Professional Standards Student Learning Outcomes Student Learning Outcome Assessment Assessment Tracking Analysis of Assessment Data Unit and Program Improvement How they learn it How we track the learning How we know that they learned What we learn from a review of their learning What we change SOE Vision & Mission Assessment Cycle – Close the Loop Johns Hopkins University School of Education

ADMISSIONS MID- PROGRAM/PRE- INTERNSHIP PROGRAM COMPLETION (CLINICAL EXPERIENCES) POST- GRADUATION 2 YEARS OUT POST- GRADUATION 5 YEARS OUT Entry GPA GRE/SAT scores Admission demographics Personal essay Teaching experience Interview ratings Disposition Survey Course assignments Course grades Content verification E-Portfolio evaluation Academic plan Survey on diversity /inclusion dispositions Reflection on personal growth and goals Advisor/instructor input Student experience survey Course Grades Test results (such as PRAXIS II, CPCE Exam) E-Portfolio evaluation Final comprehensive exam or graduate project Survey on diversity/ Inclusion dispositions University Supervisor and Cooperating Teacher evaluations Course and Field Experience Assessment results Exit interview or End of Program Evaluation Employer survey Alumni survey School partner feedback MSDE data linked to our graduates Employer survey School partner feedback MSDE data linked to our graduates Alumni survey Collect data from graduates through surveys and/or focus groups Major Assessment Points/Benchmarks

Conceptual Framework Themes/Student Outcomes Key Assessment PointsData Points Knowledgeable in their respective content area/discipline Admission Mid-program Program Completion Post-Graduation Comprehensive Exams Praxis Exams Graduate Projects Reflective practitioners Mid Program Program Completion Post Graduation Committed to diversity Admissions Mid-Program Program Completion Post Graduation Data-based decision-makers Mid-Program Program Completion Post Graduation Integrators of applied technology Mid Program Program Completion Post Graduation Alignment of Conceptual Framework to Assessment Plan’s Benchmarks

The Johns Hopkins University Mission Statement CONCEPTUAL FRAMEWORK THE SCHOOL OF EDUCATION VISION SOE Mission Inputs InitiativesDomains Key Assessment Points (For Assessments see Table 3) Outputs Resource Capability Admit Midpoint or Internship Program Completion Post Grad Student OutcomesImpact 2yr5yr Effective Teaching Innovative Tools Excellent Professional Preparation High Quality Research Innovative Outreach Knowledge Disposition Practice Content Experts Reflective practitioners Committed to Diversity Data Based Decision Makers Integrators of Applied Technology Education Improvement Community Well-Being SOE Conceptual Framework Logic Model

Program Assessment Plan Mission, Goals, Objectives/Outcomes aligned with SOE mission and outcomes National, State, and Professional Standards Assessments –Descriptions, Rubrics, Benchmarks Annual Process – Review of findings and recommendations for change – Review of assessments and adjustments Documentation of stakeholder input – – ALL faculty, students, university supervisors, cooperating teachers, partner schools, MSDE, professional organizations, community members, employers