Program Review Update Webinar

Program Review Update Webinar
Program Review Update Webinar CEC Standards Preparation standards Revised Review Language Clarity and Quality Reviews Well-Written Conditions Reviews Stay focused Discussion Challenges, Suggestions, & Questions

CEC Preparation Standards
Revised Preparation Standards

CEC Preparation (Content) Standards
Existing CEC Initial Content Standards New CEC Initial Preparation Standards 2. Development & Characteristics 3. Individual Learning Differences 1. Learner Development & Individual Learning Differences 5. Learning Environments & Social Interactions 2. Learning Environments 1. Foundations 3. Curricular Content Knowledge 8. Assessment 4. Assessment 4. Instructional Strategies 6. Language - Embedded Within All Preparation (Content) Standards 7. Instructional Planning 5. Instructional Planning & Strategies 9. Professional & Ethical Practice 6. Professional Learning & Ethical Practice 10. Collaboration 7. Collaboration 11/7/2018 © 2013 CEC. All rights reserved.

CEC Advanced Preparation (Content) Standards
Learners and Learning Assessment Content Knowledge and Professional Foundations Curricular Content Knowledge Instructional Pedagogy Improving Supports & Services Research and Inquiry Professionalism and Collaboration Leadership and Policy Professional and Ethical Practice Collaboration 11/7/2018 © 2013 CEC. All rights reserved.

CEC Preparation (Content) Standards: Revised and Reorganized
November SASB approves CEC Preparation Standards Report and triggers the start of the two year transition period 2012 – 2015 During the transition period program faculty may align their program assessments to either the CEC Content Standards or the CEC Preparation Standards Spring, 2015 At the end of the transition period program faculty must align their program assessments to the CEC Preparation Standards 11/7/2018 © 2013 CEC. All rights reserved.

Consistency to Enhance Reliability and Quality

CEC Program Recognition FAQ
Program Report Developer Resources In addition to an array of technical support options, CEC provides the following resources: Initial Level CEC Content Standards & Assessments Alignment Example Language Analysis Project Language Analysis Scoring Rubric Section II: Program-based Performance Assessments CEC Program Recognition FAQ CEC Program Reviewer Application Evidence for Meeting Standards: Assessment 2 - Assistive Technology Project Section III Assessment to Standards Alignment Guidelines for Preparing Recognition with Conditions Reports Assessment II: Content Knowledge - Comprehensive Examination Evidence for Meeting Standards: Assessment 5 - Behavior Change Project Assessment 6 -Portfolio Artifact - IEP/IFSP Assessment VIII Special Education Assessment Work Sample Folio Developing Performance-based Special Education Preparation Program Reports (PowerPoint) Guidelines for Course Grades as an Assessment of Candidate Knowledge CEC Content Standards and Program Assessments Alignment Table Unit Standards 11/7/2018 © 2013 CEC. All rights reserved.

© 2013 CEC. All rights reserved.
Field Experiences Appropriate to the license and roles for which they are preparing, candidates progress through a series of developmentally sequenced field experiences for the full range of: ages, types and levels of abilities, and collaborative opportunities that are supervised by qualified professionals. 11/7/2018 © 2013 CEC. All rights reserved.

Field Experiences PART B - STATUS OF MEETING SPA STANDARDS Field Experiences and Clinical Practice Standard Met The evidence in the program report establishes that special education candidates progress through a series of developmentally sequenced field experiences for the full range of ages, types and levels of abilities, and collaborative opportunities that are appropriate to the license or roles for which they are preparing, and that these field and clinical experiences are supervised by qualified professionals. PART B - STATUS OF MEETING SPA STANDARDS Field Experiences and Clinical Practice Standard NOT Met The evidence in the program report establishes that special education candidates progress through a series of developmentally sequenced field experiences for the full range of ages, types and levels of abilities, and collaborative opportunities that are appropriate to the license or roles for which they are preparing. THERE IS INSUFFICIENT EVIDENCE that these field and clinical experiences are supervised by qualified professionals. 11/7/2018 © 2013 CEC. All rights reserved.

Program Assessment Components
Description Instructions Evaluation Scoring Guides & Rubrics Data Total Disaggregated 11/7/2018 © 2013 CEC. All rights reserved.

CEC Preparation Standards
Seven Preparation (Content) Standards Twenty-eight Elements (Bolded phrases) Informed by Specialty Sets CEC does NOT expect Specialty Set Knowledge and Skills ITEMS to be explicitly referenced to assessment items or rubrics!!!! 11/7/2018 © 2013 CEC. All rights reserved.

Major Elements of CEC Preparation (Content) Standards
The program assessments, scoring guides/rubrics, and data tables are to be aligned to the major elements (bolded phrases) of each CEC Preparation (Content) Standard as informed by the appropriate specialty set(s) 11/7/2018 © 2013 CEC. All rights reserved.

Specialty Sets Inform the Preparation (Content) Standards
Question What does “informed by the appropriate specialty knowledge and skills set mean? Should the program report explicitly refer to specific CEC Knowledge and Skills Specialty sets? Response The major elements of the CEC Preparation (Content) Standards as informed by the appropriate specialty set are to be reflected in the program’s assessments, rubrics, and data. This means that every program must demonstrate alignment to the major elements of the CEC Preparation (Content) Standard as informed by the appropriate specialty set(s) whether the program uses the Initial or Advanced Preparation (Content) Standards. Without being informed by the appropriate specialty knowledge and skills set, every special education preparation program would look the same. The content of the Specialty sets differentiate program assessments. Programs can assure that the assessments, rubrics, and data are informed by the appropriate specialty area in a variety of ways, but the most meaningful way is to assure that performance levels within rubrics use of the content from the appropriate specialty set(s). There is no requirement or expectation for explicit or complete correspondence between the items in a specialty set with assessment items, and reviewers do not look for this level of correspondence. Likewise, reviewers do not expect that programs use the exact wording of the knowledge and skills within the rubrics. However, it is expected that the content from the appropriate specialty set(s) is used in designing assessments and rubrics. 11/7/2018 © 2013 CEC. All rights reserved.

Question Do program reviewer references to “CEC Standards” pertain to the Initial or advanced CEC Preparation (Content) Standards or to the CEC Knowledge and Skill Specialty sets? Response CEC Program Reviewers use the CEC Preparation (Content) Standards with the key elements as the organizing focus for their review, and their references to “CEC Standards “are to the CEC Preparation (Content) Standards. 11/7/2018 © 2013 CEC. All rights reserved.

Rubrics & Scoring Guides
Question Where does the program report provide evidence that the specialty set informs the major elements of the CEC Preparation (Content) Standards? Response CEC does not require or promote that program reports cite specific specialty set items. However, CEC expects program reviewers to assure clear and convincing evidence that the content of the appropriate specialty set(s) is used in the assessments, rubrics and scoring guides, and Section 1 narratives. Assessment Program faculty should assure that the content, populations, vocabulary, concepts, settings, and issues from the specialty set are used throughout the assessment items and components. Rubrics & Scoring Guides Program faculty should assure that the content, populations, vocabulary, concepts, settings, and issues from the specialty set are used throughout. Section I Narrative Program faculty should describe how the assessment addresses the specialty set specific content, populations, vocabulary, concepts, settings, and issues. 11/7/2018 © 2013 CEC. All rights reserved.

Aligning Program Assessments and Major Elements of the CEC Preparation (Content) Standards Question Must program reports provide evidence that program candidates master the major elements of the CEC Preparation (Content) Standards? Response CEC requires that a preponderance of the evidence establish that the assessments align with the major elements of the CEC Preparation (Content) Standards as informed by the content of the appropriate specialty set(s) and that program candidates master the major elements in CEC Preparation (Content) Standards as informed by the appropriate specialty set(s). 11/7/2018 © 2013 CEC. All rights reserved.

Preponderance of Evidence
Question What does CEC mean by “a preponderance of the evidence” for the major elements of the CEC Preparation (Content) Standards? Response “Preponderance of evidence” is a standard of proof indicating that the evidence is clear and convincing, as opposed to the more rigorous “beyond a reasonable doubt” standard. CEC program reviewers use the “preponderance of the evidence” standard with each of the CEC Preparation (Content) Standards. Using a preponderance of the evidence standard, the reviewer judges whether the evidence in the report is clear and convincing. A preponderance of evidence cannot be reduced to a simple quantity, i.e. 75%. It is a reasoned judgment by a set of collegial reviewers and auditors based on the evidence presented. In order to determine that a program meets a CEC Preparation (Content) Standard, the reviewers judge whether the pieces of evidence presented in the program report are clear and convincing that the program assessment aligns with the major elements of the respective CEC Preparation (Content) Standard and that the program data demonstrate that the program candidates are mastering the major elements of the CEC Preparation (Content) Standard. 11/7/2018 © 2013 CEC. All rights reserved.

Rubrics Question Our program faculty use a 3-tier rank i.e. "Unacceptable", "Acceptable", or "Proficient". With a range of points assigned to each of these categories. Response Routinely, reviewers look for whether the performances at "Unacceptable", "Acceptable", or "Proficient“ are clearly described on each scale. Assigning each of the three tiers with a range of scores is only acceptable as long as the ranges are sufficiently described and differentiated to make a reasonable level of inter-rater reliability possible. 11/7/2018 © 2013 CEC. All rights reserved.

Data Question How many years of performance data are required in the Program Report? Response Initial submission program reports must include data for at least 2 administration cycles of the assessments. In the case of state or national examinations that are given multiple times throughout the year, data from two university terms must be included Recognition with Conditions reports must include data from at least 1 administration cycle of the assessments beyond the data in the initial report. Under no conditions does CAEP require program reports to include data from more than three administration cycles of the assessments. A program is eligible for “Recognition with Conditions” with no data 11/7/2018 © 2013 CEC. All rights reserved.

State - Assessments Faculty may use State Assessments as supplementary evidence for meeting CEC Preparation Standards, But faculty may not use State Assessments as a sole source of evidence for meeting any CEC Preparation Standard. 11/7/2018 © 2013 CEC. All rights reserved.

Well-Written Reviews

Writing Style Proof, proof, and proof again!
Write in professional language DO NOT Use “I” or “me” or any other first person language State opinions or tell program faculty how to solve problems Make side comments or direct questions to program faculty or CEC Audit Team AVOID Prescriptions Overstatements Personal observations Proof, proof, and proof again! 11/7/2018 © 2013 CEC. All rights reserved.

Thorough Report Write comments that support the rating for each CEC Preparation (Content) Standard Write narrative for each section and each part excluding only “Directions to the BOE” for which reviewers may or may not choose to respond The Evidence The program report identifies the following program assessments as having elements that align with CEC Preparation (Content) Standard 3 Assessment 3 Explicit Instruction Lesson Plan Assessment 5 Curriculum-Based measurement Project Assessment 6 Behavior Change Project Your Findings In regards to the cited Assessments, the description, scoring rubric, and program candidate data for each are present and aligned to each other. The Explicit Instruction Lesson Plan and the Curriculum-based Measurement Project provides evidence that the assessments and scoring guides are aligned with the major elements of the CEC Preparation (Content) Standard as informed by the appropriate specialty set. The report provides data for these assessments that the candidates are mastering the major elements of the cited CEC Preparation Standard. Conclusions The preponderance of the evidence establishes that the program assessments align with the CEC Preparation (Content) Standard, and that the program candidate data indicate candidates are mastering the respective knowledge and skills. 11/7/2018 © 2013 CEC. All rights reserved.

Non-Alignment Problem
The Evidence The program report identifies the following program assessments as having elements that align with CEC Preparation (Content) Standard 3 Assessment 3 Explicit Instruction Lesson Plan Assessment 5 Curriculum-Based measurement Project Assessment 6 Behavior Change Project Your Findings In regards to the cited Assessments, the description, scoring rubric, and program candidate data for each are present and aligned to each other. The Explicit Instruction Lesson Plan and the Curriculum-based Measurement Project NOT provide evidence that the assessments and scoring guides are aligned with the major elements of the CEC Preparation (Content) Standard as informed by the appropriate specialty set. Since the program assessment elements are NOT aligned to the major elements of the CEC Preparation (Content) Standard , the report cannot provide data that the candidates are mastering the major elements of the cited CEC Preparation Standard. Conclusions The preponderance of the evidence does not establish that the program assessments align with the CEC Preparation (Content) Standard, and that the program candidate data indicate candidates are mastering the respective knowledge and skills. 11/7/2018 © 2013 CEC. All rights reserved.

Decision Support The Program Review Report must support the decision. Areas for considerations should tell program faculty what needs to be addressed, not what is wrong or how to fix it C.1. Candidates’ Knowledge of Content The preponderance of the evidence presented for the program assessments and the extent of their alignment to the major elements of the CEC Preparation (Content) Standards (or does not) establishes that the program candidates have satisfactory mastery of special education professional content knowledge. C.2. Candidates’ Ability to Understand and Apply Pedagogical and Professional Content knowledge, skills, and dispositions The preponderance of the evidence presented for the program assessments and the extent of their alignment to the major elements of the CEC Preparation (Content) Standards (or does not) establishes candidates’ satisfactory ability to understand and apply pedagogical and professional content knowledge, skills, and dispositions. 11/7/2018 © 2013 CEC. All rights reserved.

Recognized with Conditions Decisions
Reviewing Writing

RECOGNIZED WITH CONDITIONS
Well-written “Recognized with Conditions” decisions are: BASED ON THE EVIDENCE CLEAR PRECISE OBJECTIVE (Unbiased) CONSISTENT with all narrative in the Review Report COMPLETE, i.e. state everything to be included in the next report 11/7/2018 © 2013 CEC. All rights reserved.

CEC program reviewers need sufficient information to review “Recognition with conditions” reports. So program report developers know specifically what parts of the report to address and resubmit, use the following italicized language in Part A. Recognition Decisions of program review reports. 11/7/2018 © 2013 CEC. All rights reserved.

Question Is there specific language to include in writing a “Recognition with conditions program report”, Response The following language should be included in Part A. Recognition Decisions of program review reports. CEC Preparation (Content) Standards x, y, z were found to be either “not met” or “met with conditions”. For each CEC Preparation (Content) Standard or CEC Field Experience Standard judged either “not met” or “met with conditions”. the program resubmission report must provide: The Section II and Section III tables that document the alignment of each program assessment to the major elements of the CEC Preparation (Content) Standard as informed by the appropriate specialty set(s); The assessment descriptions, scoring guide/rubric, and data for each of the program assessments that provide the evidence that they are aligned to the major elements of each of CEC Preparation (Content) Standard as informed by the specialty area knowledge and skills set(s); and Sufficient performance data for reviewers to determine that the preponderance of the performance data for each of the CEC Preparation (Content) Standard as informed by the appropriate specialty set(s) demonstrate that the program candidates master the major elements of the CEC Preparation (Content) Standards as informed by the appropriate CEC knowledge and skill set(s). 11/7/2018 © 2013 CEC. All rights reserved.

Whenever a “conditions” program report has additional conditions, add these specific conditions to the review report. The assessment description, scoring guide/rubric, and data form a vital chain and as the metaphor points out, the chain is only as strong as the weakest link. It is helpful to program faculty if the program review report specifically identifies weak link(s). For example, “While all the materials described above are required in the resubmission, the scoring rubrics were particularly problematic and will require extensive modifications.” 11/7/2018 © 2013 CEC. All rights reserved.

Reviewing Resubmitted Conditions Reports
All previously “not met” conditions need to be “met”. DO NOT RE-REVIEW CEC PREPARATION STANDARDS THAT WERE MET PREVIOUSLY. No new conditions may be cited. If third review and clear progress toward meeting conditions has not been made, bite the bullet and give a “not recognized.” 11/7/2018 © 2013 CEC. All rights reserved.

For the Programs You are Reviewing
Do: assessments, scoring guides/rubrics and data align in clear and convincing ways to the major elements of the CEC Preparation (Content) Standards informed by the appropriate knowledge and skill set? Do they meet the data requirements? 11/7/2018 © 2013 CEC. All rights reserved.

Program Review Update Webinar

Similar presentations

Presentation on theme: "Program Review Update Webinar"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Program Review Update Webinar

Similar presentations

Presentation on theme: "Program Review Update Webinar"— Presentation transcript:

Similar presentations

About project

Feedback