1. SAHS Advanced Placement Program Evaluation
Education-Personnel Committee
July 12, 2007

2. District Goal From October 18, 2006 E-P Meeting
5. The Sunshine Area School District will develop
and pilot a system of program evaluation.
The Superintendent will assist district administrators, building administrators and teacher-leaders in the process of evaluating and reporting on a major academic program (Advanced Placement) that will include: interrogation of program data, development of specific performance goals, and the use of a standardized evaluation format.

3. Practically…
Program evaluation provides a specific answer to the question, “How do we know if the program or operating system is being successful or meeting its stated purpose?”

4. When is Program Evaluation Most Useful
When the significant time, material and human costs of planning the evaluation, setting the standards, collecting the data, engaging in data analysis, drawing conclusions, writing reports, and presenting findings are outweighed by the increased organizational efficiency or success gained by the evaluation.

5. The 4 T’s Needed for Successful Program Evaluation
Training of program evaluators so that they can
develop a scientific evaluation plan
Time to develop goals, questions and criteria
Trust among all stakeholders that the evaluation’s
intended purpose is indeed the purpose
Technology to ease the data collection and analysis

6. Why Advanced Placement as the Area for Program Evaluation?
AP is a key part of the overall SAHS program and high school reform effort
potential benefits to the organization will outweigh the evaluation costs
Large amounts of data and data measures
The 4 T’s exist in the SASD

7. The AP Evaluation Process….
The Advanced Placement Program Evaluation Team (APPET) was formed. The team met monthly and included:
Department Coordinators in Science, Math, English, Social Studies and Guidance
High School Principal
Assistant Superintendent

8. Learning Curve
The APPET learned about national trends in Advanced Placement and AP issues
The APPET learned about Program Evaluation.
The APPET studied in depth the history and achievement of the AP program at the SAHS

9. The Key AP Questions….
Question #1—Does the SAHS provide an adequate number of AP courses?

10. The Key AP Questions…
Question #2—Does the SAHS Advanced Placement program involve all of the potentially-qualified students?

11. The Key AP Questions…
Question #3—Are specific benchmarks and/or goals for the SAHS students’ AP participation and achievement known by all of the stakeholders and used for program improvement?

12. Does the SAHS Provide an Adequate Number of AP Courses?
EVALUATION: Yes. The most nationally-prevalent AP courses are offered in all of the SAHS’s departments.
FOR 07-08: Implement AP American Government and maintain AP Physics courses; Continue monitoring trends in AP offerings
STANDARD: Deliver all AP courses, with College Board-approved syllabi, as described in the SAHS Program of Studies

13. Does the SAHS Advanced Placement program involve all of the potentially-qualified students?
EVALUATION: No. Students largely self-select their participation. Identification and recruitment efforts are inconsistent. A definition of “potentially-qualified,” and a baseline level of “potentially-qualified students,” must be developed and disseminated to stakeholders.

14. All potentially-qualified students? (cont.)
FOR 07-08:
Develop a data-driven definition of potentially-qualified by incorporating ranges from successful AP students’ PSAT score profiles and PSSA cut scores.
Set annual school AP participation goal—yearly increase in number and percentage.
Execute needed training of AP teachers.

15. In (Cont.)
Departments set participation goals in their own subject areas
Create and circulate Introduction to the SAHS Advanced Placement Program publication.
Make AP Program structure and goals a part of the orientation process when students transition to the SAHS

16. In (Cont.)
Vertical teaming—designate “pre-AP” and pathway courses that naturally lead to AP courses
Use Grade 8 PSSA data to recruit high performing freshmen into AP courses during their Sophomore year.

17. In (Cont.)
Administer PSAT to all 10th and 11th grade students to create expanded list of students with AP potential. Expand recruitment accordingly.
Budget permitting, execute a reliable graduate survey to develop longitudinal data related to the effectiveness of SAHS AP programs.

18. Are Benchmarks/Goals for SAHS students’ AP participation known and used for improvement?
EVALUATION: No. A standard measure, goal setting procedure, and reporting system are needed.
FOR 07-08:
Establish the College Board AP Equity and Excellence Index (EEI) and Graduate Survey Data (if available) as the numerical bases for program performance

19. Benchmarks /Goals (cont.)
Establish EEI trend line of AP student performance in 2006 and 2007
Establish annual EEI improvement as a building and department goal
Investigate improved use of scheduling and time to complement AP participation and achievement goals.

20. Immediate Steps….
Solidify Advanced Placement Goals with SAHS Administrators, Department Coordinators and AP Teachers.
Provide updates to the Superintendent/Board on progress made on goals.