SAHS Advanced Placement Program Evaluation Education-Personnel Committee July 12, 2007

District Goal From October 18, 2006 E-P Meeting 5. The Sunshine Area School District will develop and pilot a system of program evaluation. The Superintendent will assist district administrators, building administrators and teacher-leaders in the process of evaluating and reporting on a major academic program (Advanced Placement) that will include: interrogation of program data, development of specific performance goals, and the use of a standardized evaluation format.

Practically… Program evaluation provides a specific answer to the question, “How do we know if the program or operating system is being successful or meeting its stated purpose?”

When is Program Evaluation Most Useful When the significant time, material and human costs of planning the evaluation, setting the standards, collecting the data, engaging in data analysis, drawing conclusions, writing reports, and presenting findings are outweighed by the increased organizational efficiency or success gained by the evaluation.

The 4 T’s Needed for Successful Program Evaluation Training of program evaluators so that they can develop a scientific evaluation plan Time to develop goals, questions and criteria Trust among all stakeholders that the evaluation’s intended purpose is indeed the purpose Technology to ease the data collection and analysis

Why Advanced Placement as the Area for Program Evaluation? AP is a key part of the overall SAHS program and high school reform effort potential benefits to the organization will outweigh the evaluation costs Large amounts of data and data measures The 4 T’s exist in the SASD

The AP Evaluation Process…. The Advanced Placement Program Evaluation Team (APPET) was formed. The team met monthly and included: Department Coordinators in Science, Math, English, Social Studies and Guidance High School Principal Assistant Superintendent

Learning Curve The APPET learned about national trends in Advanced Placement and AP issues The APPET learned about Program Evaluation. The APPET studied in depth the history and achievement of the AP program at the SAHS

The Key AP Questions…. Question #1—Does the SAHS provide an adequate number of AP courses?

The Key AP Questions… Question #2—Does the SAHS Advanced Placement program involve all of the potentially-qualified students?

The Key AP Questions… Question #3—Are specific benchmarks and/or goals for the SAHS students’ AP participation and achievement known by all of the stakeholders and used for program improvement?

Does the SAHS Provide an Adequate Number of AP Courses? EVALUATION: Yes. The most nationally-prevalent AP courses are offered in all of the SAHS’s departments. FOR 07-08: Implement AP American Government and maintain AP Physics courses; Continue monitoring trends in AP offerings STANDARD: Deliver all AP courses, with College Board-approved syllabi, as described in the SAHS Program of Studies

Does the SAHS Advanced Placement program involve all of the potentially-qualified students? EVALUATION: No. Students largely self-select their participation. Identification and recruitment efforts are inconsistent. A definition of “potentially-qualified,” and a baseline level of “potentially-qualified students,” must be developed and disseminated to stakeholders.

All potentially-qualified students? (cont.) FOR 07-08: Develop a data-driven definition of potentially-qualified by incorporating ranges from successful AP students’ PSAT score profiles and PSSA cut scores. Set annual school AP participation goal—yearly increase in number and percentage. Execute needed training of AP teachers.

In 07-08 (Cont.) Departments set participation goals in their own subject areas Create and circulate Introduction to the SAHS Advanced Placement Program publication. Make AP Program structure and goals a part of the orientation process when students transition to the SAHS

In 07-08 (Cont.) Vertical teaming—designate “pre-AP” and pathway courses that naturally lead to AP courses Use Grade 8 PSSA data to recruit high performing freshmen into AP courses during their Sophomore year.

In 07-08 (Cont.) Administer PSAT to all 10th and 11th grade students to create expanded list of students with AP potential. Expand recruitment accordingly. Budget permitting, execute a reliable graduate survey to develop longitudinal data related to the effectiveness of SAHS AP programs.

Are Benchmarks/Goals for SAHS students’ AP participation known and used for improvement? EVALUATION: No. A standard measure, goal setting procedure, and reporting system are needed. FOR 07-08: Establish the College Board AP Equity and Excellence Index (EEI) and Graduate Survey Data (if available) as the numerical bases for program performance

Benchmarks /Goals (cont.) Establish EEI trend line of AP student performance in 2006 and 2007 Establish annual EEI improvement as a building and department goal Investigate improved use of scheduling and time to complement AP participation and achievement goals.

Immediate Steps…. Solidify 2007-2008 Advanced Placement Goals with SAHS Administrators, Department Coordinators and AP Teachers. Provide updates to the Superintendent/Board on progress made on goals.