Pilot Testing of NCATE’s Continuous Improvement: Fall 2011 Visits Donna M. Gollnick Senior Vice President, NCATE August 19, 2009 Microphone Checks will begin at 1:45 p.m. Web Seminar will begin at 2:00 (eastern time)

NCATE Pilot Visits in Spring :45 - 2:00Audio Checks 2:00 – 2:05Introductions and Review of Software 2:05 – 2:15Continuous Improvement Focus 2:15 – 2:25Institutional Report Options 2:25 – 2:45Continuous Improvement Process 2:45 – 3:25Exhibits 3:25 – 3:30Timeline

Icon Review To participate in this web conference, use these icons: To participate in this web conference, use these icons: Raise hand Yes, No Mic Clapping Happy face Sad face

NCATE staff on the web seminar Donna M. Gollnick, Senior Vice President Donna M. Gollnick, Senior Vice President Mary Anne Kirkland, Assistant to Donna Mary Anne Kirkland, Assistant to Donna Julien Goichot, Webmaster Julien Goichot, Webmaster

Two Options for Continuing Visits: 1.Continuous Improvement 2.Transformation Initiative

Option 1: Continuous Improvement

Focus of the Review: Continuous Improvement The unit will engage in continuous improvement between on-site visits. The unit will engage in continuous improvement between on-site visits. –Submit annual reports –Describe changes in the IR –Submit IR at least one year before visit for feedback The unit will assess itself against the target level of one or more standards. The unit will assess itself against the target level of one or more standards.

The Institutional Report (IR) for Continuous Improvement

Content of IR by Standard: Prompt #1 (“Big” question related to standard) 1. 1.What do candidate assessment data tell the unit about candidates’ meeting standards? 2. 2.How does the unit use its assessment system to improve the performance of candidates and the unit and its programs? 3. 3.How does the unit work with school partners to deliver field experiences and clinical practice to enable candidates to develop the K,S,PDs to help all students learn?

“ Big” question related to standard 4. 4.How does the unit prepare candidates to work effectively with all students? 5. 5.How does the unit ensure that its professional education faculty contributes to the preparation of effective educators? 6. 6.How does the unit ensure that its governance system and resources are adequate to prepare candidates to meet professional, state, and institutional standards?

Content of IR by Standard: Prompt #2   Standard on which unit is moving toward to target Level (maximum of 5 pp)   Describe work undertaken to move to the target level   Discuss plans for continuing to improve OR, if this is not the standard on which the unit is moving to the target level   Continuous Improvement (max of 3 pp)   Summarize the most significant changes related to this standard that have led to improvement related to the standard

Content of IR by Standard: Prompt #3 Exhibit links uploaded as Word document Name of exhibit + Link to exhibit on the website

Tables in the IR Standard 4 Standard 4 –Faculty diversity –Candidate diversity –P-12 student diversity in clinical practice Standard 5 Standard 5 –Summary of faculty qualifications & involvement in scholarship & service

Options for Submission of IR Online template with prompts in AIMS Template with prompts written in Word & uploaded to AIMS Holistic response to each element OR standard written in Word & uploaded to AIMS

Exhibits To be available for the Offsite Review

Available to BOE Teams in AIMS 1.Program reports submitted for national review 2. National recognition reports 3. Reports from the previous NCATE visit 4. 3 rd party testimony 5. Annual Reports 6. Other relevant national or state reports uploaded in AIMS

General Background and Conceptual Framework 1.Links to unit catalogs and other printed documents describing general education, specialty/content studies, and professional studies. 2. Syllabi for professional education courses 3. Conceptual framework(s) 4. Findings of other national accreditation associations related to the preparation of education professionals (e.g., ASHA, NASM, APA, CACREP)

Standard 1: Candidate Knowledge, Skills, and Professional Dispositions 1.State program review documents and state findings. [1] (Some of these documents may be available in AIMS.) [1] 2. State licensure test scores aggregated by program area and reported over multiple years (Title II data reports submitted to the state for the last year must be available to the team.) 3. Data tables and summaries that show how teacher candidates (both initial and advanced) have performed on key assessments over the past three years for programs not included in the national program review process or a similar state process 4. Key assessments and scoring guides used by faculty to assess candidate learning against standards and the outcomes identified in the unit’s conceptual framework for programs not included in the national program review process or a similar state process 5. Samples of candidate work (e.g., portfolios at different proficiency levels) 6. Follow-up studies of graduates and data tables of results 7. Employer feedback on graduates and summaries of the results 8.List of candidate dispositions, including fairness and the belief that all students can learn, and related assessments, scoring guides, and data

Standard 2: Assessment System & Unit Evaluation 1.Description of the unit’s assessment system in detail including the requirements and key assessments used at transition points 2. Data from key assessments used at entry to programs 3. Procedures for ensuring that key assessments and unit operations are fair, accurate, consistent, and free of bias 4. Policies and procedures that ensure that data are regularly collected, compiled, aggregated, summarized, analyzed, and used to make improvements 5. Sample of candidate assessment data disaggregated by alternate route, off-campus, and distance learning programs 6. Policies for handling student complaints 7. File of student complaints and the unit’s response 8. Examples of changes made to courses, programs, and the unit in response to data gathered from the assessment system

Standard 3: Field Experiences & Clinical Practice 1.Memoranda of understanding, contracts, and/or other documents that demonstrate partnerships with schools 2. Criteria for the selection of school faculty (e.g., cooperating teachers, internship supervisors) 3. Documentation of the preparation of school faculty for their roles (e.g, orientation and other meetings) 4. Descriptions of field experiences and clinical practice in programs for initial and advanced teacher candidates and other school professionals 5. Guidelines for student teaching and internships 6. Assessments and scoring rubrics/criteria used in field experiences and clinical practice for initial and advanced teacher candidates and other school professionals (These assessments may be included in program review documents or the exhibits for Standard 1. Cross reference as appropriate.)

Standard 4: Diversity 1. Proficiencies related to diversity that candidates are expected to develop 2. Curriculum components that address diversity proficiencies (This might be a matrix that shows diversity components in required courses.) 3. Assessment instruments, scoring guides, and data related to diversity (These assessments may be included in program review documents or the exhibits for Standard 1. Cross reference as appropriate.) 4. Policies and practices for recruiting and retaining a diverse faculty 5. Policies and practices for recruiting and retaining diverse candidates 6. Policies, practices, and/or procedures that facilitate candidate experiences with students from diverse groups

Standard 5: Faculty Qualifications, Performance, and Development 1.Licensure information on school faculty (e.g., cooperating teachers, internship supervisors) [1] [1] 2. Samples of faculty scholarly activities 3. Summary of service and collaborative activities engaged in by faculty with the professional community (e.g., grants, evaluations, task force participation, provision of professional development, offering courses, etc.) 4. Promotion and tenure policies and procedures 5. Samples of forms used in faculty evaluation 6. Opportunities for professional development activities provided by the unit [1] [1] The faculty chart in AIMS will also be used as evidence for this standard.

Standard 6: Unit Governance and Resources 1. Policies on governance and operations of the unit 2. Organizational chart or description of the unit governance structure 3. Unit policies on student services such as counseling and advising 4. Recruiting and admission policies for candidates 5. Academic calendars, catalogs, unit publications, grading policies, and unit advertising 6. Unit budget, with provisions for assessment, technology, and professional development 7. Budgets of comparable units with clinical components on campus or similar units at other campuses 8. Faculty workload policies 9. Summary of faculty workloads 10. List of facilities, including computer labs and curriculum resource centers 11. Description of library resources 12. Description of resources for distance learning, if applicable.

Presentation of Exhibits 1.Online Exhibit Room 2.List of exhibits identified in Prompt #3 of the IR provides links to key exhibits supporting each standard ***Please use NCATE’s list of exhibits***

The Offsite Review 6-12 months before visit

Offsite BOE Team Team of 3-5 persons that meets electronically with state consultant & NCATE staff member If state protocol calls for joint visit, state team member may join team If possible, one of team members will be assigned to Onsite BOE Team

Offsite Review Offsite Visit BOE Team reviews the following electronically & prepares feedback report for unit: Institutional report Annual reports Title II data Program reviews by SPAs or states Exhibits related to standards

Feedback Report by Offsite BOE Team 1. 1.Statement about the evidence Does the evidence make the case that standards continue to be met? What, if any, key evidence was not available to the team? 2. 2.Comments on unit’s progress toward meeting target level 3. 3.List of areas of concern by standard 4. 4.List of evidence for the Onsite BOE Team to validate during the onsite visit

Timeline: Offsite Review for CI 12 months before visitUnit submits IR 1-3 months after the IR is submitted Offsite BOE Team meets electronically to review IR & write BOE Feedback Report Generally within 1-2 weeks after Offsite BOE Team meeting BOE Feedback Report is available in AIMS 1-2 months before the onsite visit and before the previsit with the team chair, state team co-chair, & state consultant Unit submits IR Addendum, which responds to Areas of Concern in the BOE Feedback Report

The Onsite Visit

Onsite Visit: Continuous Improvement Smaller team will conduct shorter visit by 1. 1.Focusing on areas of concern raised by Offsite Visit BOE Team 2. 2.Validating that standards continue to be met 3. 3.Providing feedback on progress toward the target level on one or more standards

Activities: Sunday Team members arrive onsite by 1 pm Team members arrive onsite by 1 pm Team meeting Team meeting Orientation by institution & state Orientation by institution & state (Optional) Poster sessions (Optional) Poster sessions

Activities: Monday Visits to 2-4 schools Visits to 2-4 schools Observations Observations Interviews Interviews Sampling of Evidence Sampling of Evidence Team Meetings Team Meetings

Activities: Tuesday Follow-up interviews as needed Follow-up interviews as needed Team meeting Team meeting –Complete writing of BOE report Exit report to unit Exit report to unit Team leaves by 3 pm Team leaves by 3 pm

Continuous Improvement Visit: Fall 2011 Pilot Visits Unit submits programs for national review Feb 2010/ Sept 2010 National recognition reports are available in AIMS 5 months after program submission Mar-Jun 2011 Unit calls for 3 rd party testimony Unit submits revised program reports for national review Unit submits institutional report (IR) Offsite BOE Team reviews IR electronically & provides feedback Unit hosts previsit with BOE chair, state co-chair, & state consultant Unit submits response to feedback report in AIMS 3-5 member BOE team conducts visit from Sunday to Tuesday afternoon Sept-Dec 2011 after previsit days prior to visit Nov July 2011 Sept May 2011

First Accreditation Process

First Accreditation Number of preconditions reduced Timeline flexible IR to set baseline for future Process same as continuous improvement

IR: First Accreditation Unit must show how stands are being met to establish a baseline IR format options include: –C–C–C–Current online template with prompts for each element & 4 tables –B–B–B–By standard in a Word document that is uploaded in AIMS Exhibit list is same as for CI & TI options

Timeline: First Visit DateAction 1-3 yrs. before visitSubmission of preconditions, which are reviewed by NCATE’s ARPA Committee 1 yr. before visitIR submitted in AIMS 9-11 mos. before visitOffsite BOE Team reviews the IR & other evidence & writes BOE Feedback Report 1-2 mos. before visitSubmission of IR addendum responding to Areas of Concern in BOE Feedback Report 1-2 mos. before visitPrevisit to plan onsite visit with BOE team chair, state co-chair, & state consultant Sunday to Tuesday Onsite Visit Onsite BOE Team conducts visit & writes Onsite BOE Report 1-2 mos. after visitSubmission of rejoinder to the Onsite BOE Report

Major Changes in National Program Reviews

National Program Review 1.Programs have option to use current system of 6- 8 assessments. 2.Programs have option to select assessments and manner in which to make case for their validity and sufficiency of evidence. 3.Programs have option of submitting only new or substantially changed assessments and new evidence for continuing recognition.

National Program Review 4.Program reports are submitted at mid- cycle to provide a more formative process. 5.Requirements for contextual information and years of data are reduced.

National Program Review 6.Simplified processes are being developed for MAT-like secondary and low-enrollment programs. 7.Task Force is developing common principles for SPA program standards to provide greater consistency across them.

Thanks for assisting NCATE in continuing to redesign the accreditation system to make it more efficient and valuable.

Stay in touch Call or NCATE staff at anytime before, during, and after the visit. Call or NCATE staff at anytime before, during, and after the visit. – team information – program review