Department of Defense Federal EMS Metrics – Development and Scoring Major Aaron Altwies, USAF 5 June Federal Environmental Symposium

Department of Defense 2 History EO directed EMSs be put in place at appropriate facilities Interagency Environmental Leadership Workgroup established IELW called for a metrics subgroup to be stood up to develop metrics Members from around the Federal government Project started in 2004 to create metrics for 2006 Mid-2006 – metrics approved and delivered

Department of Defense 3 Metrics Subgroup Members Team Members: Major Aaron Altwies, DOD Mr. TJ Granito, DHS Mr. Mike Green, NASA Mr. Eric Haukdal, HHS Mr. Russelle McCollough, USEPA Ms. Karen Waldvogel, USDA Mr. Pete Wixted, DOC Mr. Steve Woodbury, DOE In support: Mr. Ed Pinero, OFEE Mr. Will Garvey, USEPA Dr. Jean Shorrett, DOI

Department of Defense 4 Goals of Metrics Post-Implementation Determine value/benefit of the EMS Tough but attainable Pass OMB scrutiny Not “cream puff” metrics Be clearly and fully defined Give facilities feedback on how they are doing Provide agencies overall feedback Continuous improvement

Department of Defense 5 Challenges How to make the metrics effectively “rate” the agency How to provide appropriate feedback How to ensure leadership attention 2 options discussed 1. Pool all responses for the agency No fidelity at the facility level Cannot provide appropriate feedback to facilities Easier to collect/consolidate 2. Consolidate facility responses Each facility contributes equally Agency better analyze data More difficult to collect/ consolidate/understand? B A A A A A A B C B B B BD B B B C C C C C C C C C D DD D D D D D DD D D D 14 D 10 C 9 B 5 A Agency Inputs Agency Score 1.D C D B A C C 2.C D B C A D C 3.D D C C C D D 4.C D D D A C A Facility Inputs YYGRYYGR Facility Score

Department of Defense 6 More Challenges Define the metric Establish a performance level Make clear what the data represents (not a scaled response) Not every “C” is equal, but establishes a performance level for the particular metric area In all cases, “D” is the preferred response Establish metrics applicable across the Federal government, considering not all departments are of equal size or have the same amount of facilities

Department of Defense 7 Metric Criteria 6.EMS AUDIT / EVALUATION PROCEDURES. A.EMS audit/evaluation procedures were not established during this reporting period or previously. B.EMS audit/evaluation procedures were established during this reporting period or previously but no audit was conducted during this reporting period. C.EMS audit/evaluation procedures were established during this reporting period or previously; an audit was conducted during this reporting period; nonconformities are not yet being addressed or corrected. D.EMS audit/evaluation procedures were established during this reporting period or previously and an audit was conducted during this reporting period; nonconformities were addressed or corrected. 2. Objectives / Targets 1. Aspects 3. Operational Controls 5. Contracts 4. Training 6. Auditing 8. Management Review 7. Audit Date Example:

Department of Defense 8 Final Outcome Developed metric areas Developed performance levels What combination of facility scores yield green/yellow/red Questions for each letter response meant to build progress from initial implementation (A responses) to full operation (D responses) Allows for one problem area and still receive yellow Two deficient areas result in red score Based agency scores on facility input Facility rating method remains constant through 2008 Agency ratings change annually

Department of Defense 9 Understanding the Color Rating Green: All elements of the EMS are in place and working Yellow: Most elements are in place, but one or two areas need focused attention Red: Two or more elements are not functioning or may not be present

Department of Defense 10 At Least 5 D’s No More than 1 B No A’s at all At Least 4 D’s No B’s at all No A’s at all  OR  At Least 4 C’s (or D’s) No More than 1 A Neither Green nor Yellow For Green, a Facility Needs: For Yellow, a Facility Needs: For Red, a Facility is: Facility-Level Rating Constant From

Department of Defense 11 Possible Scoring Combinations

Department of Defense 12 At Least 70% of Facilities are Green No More than 5% of Facilities are Red No More than 20% of Facilities are Red Neither Green nor Yellow G Y R At Least 75% of Facilities are Green No More than 5% of Facilities are Red No More than 15% of Facilities are Red Neither Green nor Yellow G Y R At Least 80% of Facilities are Green No More than 5% of Facilities are Red No More than 10% of Facilities are Red Neither Green nor Yellow G Y R Department Level Scoring Sliding Scale – More Stringent Each Year

Department of Defense 13 Conclusions Difficult task tackled by outstanding professionals Based on the intent of the Executive Order Moving forward Next Steps: Develop next set of metrics standards Streamline reporting

Department of Defense QUESTIONS

The Air Force mission is to fly and fight