ASSESSMENT SYSTEMS FOR TSPC ACCREDITATION Assessment and Work Sample Conference January 13, 2012 Hilda Rosselli, Western Oregon University

WHAT TSPC RULE SAYS…  (1) The unit has an assessment system that collects and analyzes data on applicant qualifications, candidate and graduate or program completer performance, and unit operations to evaluate and improve the performance of candidates, the unit and its programs ROSSELLI (2012) 2

WHAT TSPC RULE SAYS…  (2) Areas evaluated under this standard include:  (a) Assessment System;  (b) Data Collection, Analysis, and Evaluation; and  (c) Use of Data for Program Improvement ROSSELLI (2012) 3

TSPC EXPECTATIONS  Unit faculty collaborate with members of the consortium to implement and evaluate the system. Professional, state, and institutional standards are key reference points for candidate assessments.  The unit embeds assessments in programs, conducts them on a continuing basis for both formative and summative purposes, and provides candidates with ongoing feedback.  The unit has procedures to ensure credibility of assessments: fairness, consistency, accuracy, and avoidance of bias. TSPC Professional Standards Manual ROSSELLI (2012) 4

PROGRAM REVIEW TEMPLATE  TRANSITION POINT ASSESSMENT: BRIEF NARRTIVE OF THE KEY ASSESSMENTS AT THE GATES IDENTIFIED.  SUMMARY OF ASSESSMENTS AND GUIDES: FIRST LIST OUT ALL 6- 8 ASSESSMENTS AND THE AREA OF DATA COLLECTION  Example:  Assessment # 1: Content test Content test  Assessment #2: Work sample – Content Pedagogy  Assessemnt #3:  Assessment #4  Assessment #5  Assessment #6  Assessment #7  Assessment #8 Summarize the assessments, scoring guides and rubrics for each major assessment. ROSSELLI (2012) 5

UNIT REVIEW EXPECTATIONS  The unit uses multiple indicators (e.g., 3.0 GPA, mastery of basic skills, general education knowledge, content mastery, and life and work experiences) to identify candidates with potential to become successful teachers or assume other professional roles in schools at the point of entry into programs.  The unit has multiple decision points, (e.g., at entry, prior to clinical practice, and at program completion).  The unit administers multiple assessments in a variety of forms and aligns them with candidate proficiencies. These may come from end-of-course evaluations, written essays, or topical papers, as well as from tasks used for instructional purposes (such as work samples, projects, journals, observations by faculty, comments by cooperating teachers, or videotapes) and from activities associated with teaching (such as lesson planning, identifying student readiness for instruction, creating appropriate assessments, reflecting on results of instruction with students, or communicating with parents, families, and school communities). TSPC Professional Standards Manual ROSSELLI (2012) 6

TSPC EXPECTATIONS  The unit uses information available from external sources such as state licensing exams, evaluations during an induction or mentoring year, employer reports, follow-up studies or surveys, and state program reviews.  The unit establishes scoring guides, which may be rubrics, for determining levels of candidate accomplishment and completion of their programs.  The unit uses results from candidate assessments to evaluate and make improvements in the unit, and its programs, courses, teaching, and field and clinical experiences. ROSSELLI (2012) 7

WHAT DO YOU WANT A SYSTEM TO DO Disaggregate data on individual candidate progress and decisions Aggregate data on candidates’ documentation of specific proficiencies Aggregate data on program documentation of specific proficiencies Aggregate data on candidate demographics, GPA, PRAXIS, field placement, cohort, year etc. ROSSELLI (2012) 8

HOW TO MAKE IT HAPPEN  Establish structures to guide and support the work around identification, quality, collection, and use of data to make decisions.  Personnel, Purchased or Develop relational database  Institutionalize dedicated time for sharing results  Involve Consortium (including students)  Develop a secure assessment website for reports  Develop a rule to guide how data results are used ROSSELLI (2012) 9

HOW TO MAKE IT HAPPEN  Tie your curriculum proposal process to data to study impact  Share the importance of priority with Provost & President (will likely appreciate how this aligns with NWCCU expectations)  Focus on just a few key assessments that can be used consistently allowing academic freedom to still exist  Provide systematic orientations to faculty, staff, supervisors  Disaggregate data by program or delivery model to examine for program fidelity  Balance indirect with direct assessments that have external validity (PRAXIS, ACT/SAT, state survey of employer satisfaction) ROSSELLI (2012) 10

DIRECT  Teacher Work Samples  Professional Projects  Portfolios  Standardized tests or certification exams  Evaluations from field experiences INDIRECT  Exit surveys  Alumni surveys  Employer surveys  Job placement data EXAMPLES ROSSELLI (2012) 11

DESIGN PROCESS  Understand how faculty will want to use data  Sort out consistent from inconsistent sources of data  Map the assessments to the required standards  Establish design principles ROSSELLI (2012) 12

DESIGN PRINCIPLES The system we developed needed to be able to:  Make clear to everyone involved the knowledge, skills, and dispositions expected of candidates  Make clear to everyone what evidence will be looked to in making these assessments  Make clear “how good is good enough” with respect to areas of performance SAMPLE ROSSELLI (2012) 13

A METAPHOR…  You are designing a road map that traces the path backwards from the destination (proficiency attained) to various points along the journey where decisions are made about attainment. ROSSELLI (2012) 14

KEEP IN MIND…  Not all elements should be assessed at each gate.  Some of the least valuable data are the most easily gathered.  What is not measured may be a statement about what is not valued or important.  Some of the most important things may be the most difficult to measure.  It may be worth measuring something even though the assessment may not be as desirable as you would like. ROSSELLI (2012) 15

KEEP IN MIND  Make sure there is congruity between Standard 2 and what is shared about the assessment process in your Program Reviews.  Expect some ambiguity and continued evolution as your system develops.  Develop or purchase a relational database system. ROSSELLI (2012) 16

QUESTIONS FOR OUR PANELISTS  Briefly describe the data system you use and its best features.  What process was used to develop/adopt your data system?  What are the drawbacks of your data system?  What are the approximate costs affiliated with your system?  What would you do differently if you were starting over and designing a data system?  What tips would you offer others that can make their work easier? ROSSELLI (2012) 17