1. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Commission on Teacher Credentialing Common Standard 2: Unit and Program Assessment and Evaluation

2. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Topics to be Addressed ● Common Standard 2 ● Understanding the differences between the Unit and the Program ● Designing a Unit Evaluation and Program Assessment System 2 #1 Agenda, #2 PowerPoint

3. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Common Standards 2008 1: Educational Leadership 2: Unit and Program Evaluation System 3: Resources 4: Faculty 5: Admissions 6: Advice and Assistance 7: Field Experiences and Clinical Practice 8: District Employed Supervisors 9: Assessment of Candidate Competence 3

4. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Standard 2: Unit and Program Assessment and Evaluation The education unit implements an assessment and evaluation system for ongoing program and unit evaluation and improvement. The system collects, analyzes and utilizes data on candidate and program completer performance and unit operations. Assessment in all programs includes ongoing and comprehensive data collection related to candidate qualifications, proficiencies, and competence, as well as program effectiveness, and are used for improvement purposes. The education unit implements an assessment and evaluation system for ongoing program and unit evaluation and improvement. The system collects, analyzes and utilizes data on candidate and program completer performance and unit operations. Assessment in all programs includes ongoing and comprehensive data collection related to candidate qualifications, proficiencies, and competence, as well as program effectiveness, and are used for improvement purposes. 4

5. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Standard 2: Unit and Program Assessment and Evaluation The education unit implements an assessment and evaluation system for ongoing program and unit evaluation and improvement. The system on candidate and program completer performance and unit operations. Assessment in all programs includes ongoing and comprehensive data collection related to candidate qualifications, proficiencies, and competence, as well as program effectiveness, and are used for improvement purposes. The education unit implements an assessment and evaluation system for ongoing program and unit evaluation and improvement. The system collects, analyzes and utilizes data on candidate and program completer performance and unit operations. Assessment in all programs includes ongoing and comprehensive data collection related to candidate qualifications, proficiencies, and competence, as well as program effectiveness, and are used for improvement purposes. 5

6. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Designing the Unit and Program Evaluation System ● Collect Data ● Analyze the Data ● Utilize the Analysis for Unit/Program Improvements 6

7. Unit & Programs Common Standard 2 7

8. Medium Institution A School of Health & Human Services Dean School of Hard Knocks Dean Programs MS/SS Ed Spec No credential programs UNIT School of Education Programs MS/SS Rdg SpecAdmin Ed Spec HR, Credential Analyst, etc. Dean 8

9. UNIT Large Institution B School of Health & Human Services Dean School of Education Dean School of Hard Knocks Dean SLPPPS Programs Ed Spec Admin CTE MS/S S Rdg Spec Library No credential programs HR, Credential Analyst, etc. 9

10. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Small Institution C Program Induction UNIT District/COE Induction HR, Credential Analyst, etc. Institutional Administration 10

11. Small Institution D Program Induction UNIT Small IHE— 1-2 Programs HR, Credential Analyst, etc. Education Dept Chair 11

12. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Reflection ● Did the explanation and graphics clear up any of your misconceptions about the relationship between the unit and programs? ● Do you have any examples to share that illustrate this relationship— positive or negative? ● Do you have any questions about this relationship? 12

13. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Unit and Program Evaluation System Admin Services Program 13 CTEL Program Multiple Subject Program

14. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California LET’S START WITH THE PROGRAM EVALUATION SYSTEM Handout #3 14

15. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Common Standard 9: Assessment of Candidate Competence Candidates preparing to serve as teachers and other professional school personnel know and demonstrate the professional knowledge and skills necessary to educate and support effectively all students in meeting the state-adopted academic standards. Assessments indicate that candidates meet the Commission-adopted competency requirements, as specified in the program standards. Candidates preparing to serve as teachers and other professional school personnel know and demonstrate the professional knowledge and skills necessary to educate and support effectively all students in meeting the state-adopted academic standards. Assessments indicate that candidates meet the Commission-adopted competency requirements, as specified in the program standards. 15

16. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Designing the Program Evaluation System ● What do we want to know about the candidates and the program(s)? ● Who will be assessed? ● What will be evaluated? ● Who will provide data on the program? ● When will the data be collected? ● How will data be collected and stored? ● How will data be analyzed and reported? 16 Handout #4

17. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California What do we want to know about candidates and programs? ● About candidates? Common Standard 9 and the Candidate Competence Standards in the Program Standards ● About programs? Common Standard 1 (leadership), 3 (resources), 4 (faculty), etc. 17

18. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Who/what will be assessed? (CS 9) ● Candidates’ competence in areas appropriate for the credential –Performance on the TPA –Performance on class projects –Performance in student teaching/clinical experiences ● Completer’s competence ─ Assessment by supervisor ─ Evaluation of a portfolio of work ● Program effectiveness –Evaluation of Instructor/mentor –Satisfaction with program offered 18

19. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Who has the information we want? ● Candidates ● Program Completers ● Faculty/Instructors ● Field Supervisors ● program and district ● Employers ● Advisory Boards 19

20. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Who has the information we want? ● Multiple pathways ● Traditional vs. intern ● Traditional vs. Early Completion Option ● Induction—main program vs. privates ● Multiple locations ● Main campus vs. satellite locations ● Face to face meetings vs. online 20

21. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California When to Collect Data? Institution and/or Program must decide: ● Annually ● Each Term ● With each cohort ● Beginning, middle and/or end of the program 21

22. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California How to collect data? Overall question – What kind of data do we want? What data collection instrument would be most effective in collecting the data? SurveysSurveys AssignmentsAssignments EvaluationsEvaluations Focus groupsFocus groups 22

23. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Data Collection What kind of data will each instrument provide? Open-ended (qualitative)Open-ended (qualitative) Multiple choice, scaled scoresMultiple choice, scaled scores (quantitative) (quantitative) A combination of both?A combination of both? 23

24. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Data Storage ● Collection procedures will impact storage and analytical procedures ● If you use electronic data collection instruments, the results can be easily stored. ● Other collection procedures might require entry or data manipulation. ● Who will enter the data? ● Who will have access to the data? 24

25. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Analyze the Data ● Who has the skills to analyze the data, consider the questions? ● What tools will be used? ● How often will data be analyzed? ● How do you want the data presented? By cohort? By program? By geographic location? All of the above? Other? 25

26. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Presentation of Data  Section A is the work of each program and can be individualized or uniform  Benefits of a common template for the program reports –Comparing of apples to apples –Ease of examination by unit personnel  Example: CSULB Handout #5 26

27. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Triangulating the Data Data from multiple sources helps locate program strengths and areas for growth: candidate competence data, completer and employer surveys, and faculty surveys Data from multiple sources helps locate program strengths and areas for growth: candidate competence data, completer and employer surveys, and faculty surveys 27 Course Evaluations Employer surveys Faculty feedback TPAs Focus Groups Key Assessments Program self-study MS Program

28. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Utilize the Analysis for Improvement ● How will results from each of the analyses be shared? ● With whom will the results be shared? ● When? ● Who will ensure that changes identified by the analyses are undertaken? 28

29. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Utilize the Analysis for Improvement (cont’d) ● How will decisions be made if the data indicates that changes should be made?—Will each professor decide? Does the faculty as a whole? Is it an administrative decision? ● How will changes be noted for institutional history and accreditation activities? 29

30. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Reflection on Your Program ● What part(s) of the suggested system do you have in place? ● What part(s) are you currently working on? ● What part(s) have you not taken into consideration at all? 30

31. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California LET’S MOVE TO UNIT EVALUATION 31

32. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Unit Evaluation System ● The unit evaluation system is composed of each program’s evaluation process ● Evaluation of Common Standards ● When, what, who, how… 32

33. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Designing the Unit Evaluation System ● What do we want to know about the candidates, program(s), and the unit as a whole? ● Who will be assessed? ● What will be evaluated? ● Who will provide data on the program? ● When will the data be collected? ● How will data be collected and stored? ● How will data be analyzed and reported? 33

34. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California What will be evaluated? ● Program Design (CS 1) ● Assessments (CS 2) ● Faculty (CS 4) ● Admission (CS 5) ● Advising (CS 6) ● Field Work (CS 7, 8) ● Recommendation for credentials 34

35. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California “The education unit implements an assessment and evaluation system…” Evidence from Multiple Sources ● ● Common Standards document/attachments: - clear graphic of system showing flow of assessment data for decision-making ● ● Meeting minutes: - records of stakeholder involvement in system design/review ● ● Interviews with leadership: - clear descriptions of how data moves through the system and who is responsible for gathering, analyzing and organizing data for decision making 35

36. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California “The system collects, analyzes and utilizes data on candidate and program completer performance…” Evidence from Multiple Sources ● ● Interviews with candidates, completers: - clear descriptions of how their performance is/was measured throughout the program - descriptions of how feedback is collected from completers ● ● Interviews with program faculty: - examples of formative and summative data collected from candidates throughout program 36

37. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Evidence from Multiple Sources ● ● Interviews with unit leadership: - clear descriptions of how assessment system data informs unit operation decisions 37 “…and unit operations.”

38. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California “Assessments in all programs includes ongoing and comprehensive data collection….” Evidence from Multiple Sources ● ● Biennial Reports: - examples of candidate, completer competency data; analysis of data; plans for program improvements based on data analysis ● ● Supporting documentation: - multi-year data summaries; trend analyses; copies of annual data summaries submitted to institution leadership 38

39. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California “…[data] is used for improvement purposes.” Evidence from Multiple Sources ● ● Interviews with program faculty: - multiple examples of program improvements made on the basis of data analysis ● ● Interviews with unit leadership: - examples of unit operation improvements made on the basis of data analysis ● ● Interviews with stakeholders, advisory groups: - descriptions of stakeholder/advisory group involvement in program/unit improvement discussions, decisions 39

40. 40 THE UNIT Collect ‘Gather data’ Analyze ‘Organize data’ Utilize ‘Drive decision- making’ -Gather data across all of an institution’s approved programs related to the Common Standards: 1) Leader- ship, 3) Resources, 4) Faculty, 5) Admission and 6) Advice & Assistance, 7) Field Experience, 8)District- Employed Supervisors, and 9) Candidate Competence. -Collect data in ongoing and comprehensive manner. -Organize the data within the unit and across all of the approved program(s). -Discuss the data with faculty and others within the unit and all of the approved program(s). -Draw conclusions from the data to inform decision-making across the unit and all of the approved program(s). -Use the analysis of the data for unit and program(s) improvement purposes. -Document the cycle of improvement decision- making for the unit and its programs. -Document actions taken, the basis of those actions and how/when the results will be reviewed next at the unit level. Handout #6

41. 41 ALL PROGRAMS Collect ‘Gather data’ Analyze ‘Organize data’ Utilize ‘Drive decision- making’ -Gather data related to the candidate competencies identified in the Program Standards as well as program effectiveness. -Collect from candidates, completers, employers, field supervisors and faculty in ongoing and comprehensive manner. -Organize the data within the program. -Discuss the data with faculty and others working with the program. -Draw conclusions from the data to inform decision- making within the program. -Use the analysis of the data for program improvement purposes. -Document actions taken, the basis of those actions and how/when the results will be reviewed next at the program level.

42. Example of Unit Assessment System Standard Assess- ment Focus Evaluation Focus Evaluation Instrument Interval Data Analyzed by 1: Leader- ship VisionPrograms using vision Action planEvery 5 years SOE Faculty/ Advisory Groups 2: Assess- ment & Evalua- tion Unit & Program Policies and practices Qual/Quant analysis, results, action plan AnnuallySOE Faculty 3: Resources Inclusive of all programs Adequacy of budget: qualified personnel, facilities, etc. Division chair, roundtable survey AnnuallyDean, Chairs, Directors 42

43. Example of Unit Assessment System Standard Assess- ment Focus Evaluation Focus Evaluation Instrument Interval Data Analyzed by 4. Faculty/ Instr. Pers. Evalua- tion/ Retention Perform- ance of all personnel Survey by candidates, completers End of course, program Program Directors, Dean 5. Admission Well- defined well-run Diversity/ Qualifica- tions of Cand. Profile— admitted/ dropped candidates AnnuallyProgram Directors, Dean 6. Advice and Assistance Quality of advice, availabil -ity Candidatep erception of process, personnel Survey of candidates, advisors Annually, end of program Program Directors, Dean 43

44. Example of Unit Assessment System Standard Assess- ment Focus Evalua- tion Focus Evaluation Instrument Interval Data Analyzed by 7. Field Exper- iences Field- based/ clinical Exp. Quality of experi- ence Focus groups, advisory board AnnuallyProgram Directors, Dean 8. District- Employed Supervisors Training, evaluation, recognition Perform- ance of personnel Survey by candidates, completers End of course, program Program Directors, Dean 9. Candidate Assess- ment ProgramsCandidate compe- tency, program effective- ness Varies by program Annually, end of course, end of program Program Directors, Dean, Cred. Analyst 44

45. Guidance for Coming to a Standard Finding on Common Standard 2 Unit Assessment and Evaluation (unit operations— Common Standards) Program Assessment and Evaluation (candidates, completers, and program effectiveness) Common Standard 2 Finding CollectAnalyzeUtilizeCollectAnalyzeUtilize Yes Met 1 Yes NoYes Met with Concerns 2 YesNo Yes No Yes NoYes No Not Met 3 No Yes No Yes No YesNo YesNo 45

46. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Reflection ● What part(s) of the suggested unit system do you have in place? ● What part(s) are you currently working on? ● What part(s) have you not taken into consideration at all? 46

47. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Resources PSD News To subscribe, send an email to psd-news- subscribe@lists.ctc.ca.gov psd-news- subscribe@lists.ctc.ca.gov Commission Website www.ctc.ca.gov select Program Sponsors then Accreditation www.ctc.ca.gov select Program Sponsors then Accreditation www.ctc.ca.gov 47

48. Commission on Teacher Credentialing Inspire, Educate, and Protect the Students of California Resources Biennial Reports http://www.ctc.ca.gov/educator-prep/program- accred-biennial-reports.html http://www.ctc.ca.gov/educator-prep/program- accred-biennial-reports.html Program Assessment http://www.ctc.ca.gov/educator-prep/program- accred-assessment.html http://www.ctc.ca.gov/educator-prep/program- accred-assessment.html Site Visits http://www.ctc.ca.gov/educator-prep/program- accred-site-visits.html http://www.ctc.ca.gov/educator-prep/program- accred-site-visits.html 48