1. Is Our Nation More Prepared Since 9/11?
A Way Forward on Measuring Grant Effectiveness
July 10, 2018
National Homeland Security Conference

2. Agenda Overview Progress in Preparedness Grant Effectiveness
FEMA Strategic Plan
The National Preparedness System
Progress in Preparedness
Using Data to Demonstrate Improvements
National-Level Trends
Grant Effectiveness
Strategic Vision
THIRA/SPR Methodology
Case Studies and Findings
Return on Investment
Next Steps on Grant Effectiveness
Conclusion

3. Major Incidents Impacting National Preparedness
Calls to unify and further professionalize emergency management across the Nation
Recognizes the importance of working together to help people before, during, and after disasters

4. FEMA Strategic Plan (2018 – 2022)
Calls to unify and further professionalize emergency management across the Nation
Recognizes the importance of working together to help people before, during, and after disasters
Strategic Goal 1
Strategic Goal 2
Strategic Goal 3

5. The National Preparedness System
Provides a consistent, scalable approach to support decision making, resource allocation, and measuring progress
Enables a collaborative, whole community approach to national preparedness
FEMA gains insight into how jurisdictions implement NPS activities through data collection and analysis

6. Progress in Preparedness

7. Increased Whole Community Participation
THIRA/SPR Data
Increased Whole Community Participation
The Threat and Hazard Identification and Risk Assessment (THIRA) and Stakeholder Preparedness Review (SPR) ask stakeholders to:
Identify and assess risk
Estimate capability requirements
From Fiscal Year (FY) :
Whole Community participants in the THIRA process nationwide increased by 23 percent
Whole Community participants in the SPR process nationwide increased by 50 percent 7

8. Increased Validation of SPR Capability Assessment Data
Figure 4. In 2017, NEP exercises across the country tested 31 core capabilities and addressed a variety of threats and hazards, including cybersecurity, hazardous material, and earthquakes.
From FY , the percentage of SPR capability assessments:
Validated by an exercise increased 14 percentage points
Validated by a real-world incident increased 7 percentage points

9. Building National Incident Management System (NIMS) Teams
FEMA analyzes Homeland Security Grant Program (HSGP) funds allocated to supporting National Incident Management System (NIMS) resources to better understand the extent to which grant recipients are implementing NIMS typing
In FY 2017, HSGP recipients supported 1,879 investments towards NIMS resources
Jurisdictions most frequently invested HSGP funds in the following five NIMS resources:
FY 2017 Top Five Supported NIMS Resources
NIMS Resource
Number of Jurisdictions
Bomb Squad/Explosives Team 20
SWAT/Tactical Teams 18
HazMat Entry Team 12
Incident Management Team
Communications Support Team (CAP) 10 9

10. Using Grant Resources to Build Capabilities
FEMA tracks the percentage of HSGP grant funds that jurisdictions use to build and sustain capabilities using:
State Homeland Security Program (SHSP)
Urban Area Security Initiative (UASI)
Operation Stonegarden (OPSG)
Grant recipients can use grant awards to build or to sustain their current level of capability to fill gaps identified through THIRA/SPR data.
From FY , jurisdictions more than quadrupled the percentage of grant funds they invest in building capabilities from 4 percent to 18 percent

11. Emergency Operations Planning
Improve Plans Through Corrective Actions
Percentage of Jurisdictions that Updated EOPs Based on Corrective Actions
Percentage of States and Territories with an EOP aligned to CPG 101v2
FEMA measures state and territory alignment with Comprehensive Preparedness Guide 101 version 2 (CPG 101v2), including Whole Community involvement in the planning process
From FY , the percentage of states with Emergency Operations Plans (EOPs) aligned to CPG101v2 increased from 47 percent to 93 percent
The percentage of jurisdictions with EOPs updated based on corrective actions rose from 36 percent to 38 percent in the same time 11

12. National-Level Trends
FEMA analyzes nationwide data, including the THIRA, SPR, training and exercises, after-action reports, and preparedness grant funding to determine national trends in preparedness
Planning, Operational Coordination, and Public Health, Healthcare, & Emergency Medical Services are consistently among the most proficient capabilities
Low proficiency ratings for Operational Coordination have been cut in half since 2012
Cybersecurity, Housing, and Infrastructure Systems have been national areas for improvement every year since 2012
National Areas for Improvement ( )

13. Grant Effectiveness

14. Grant Effectiveness Strategic Vision
Objective 1: Implement projects that address state and national priorities
Objective 2: Improve capabilities and achieve preparedness outcomes
Objective 3: Manage projects in accordance with Federal standards and guidance

15. Improve Data Collection
THIRA/SPR Methodology
Improve Data Collection
Helps Communities by:
Establishing a three-year THIRA cycle that allows for incremental progress towards achieving capability targets
Ability to provide additional context to capability estimates
Integrated with other preparedness efforts and producing data more useful for informing preparedness activities
Helps FEMA to:
Track improvements in preparedness
Analyze the role of FEMA grants in closing capability gaps and improving disaster outcomes
Provide community-specific support before, during, and after emergencies

16. THIRA/SPR Methodology
Highlights
More intuitive approach for communities to compare their capability targets and current capabilities
Helps communities to conduct more comprehensive and useful assessments
Simplifies the process by introducing a set of standardized impacts and targets
THIRA Process (three-year cycle)
SPR Process (annual)

17. HazMat Team Accessibility
Increased Capabilities
HazMat Team Accessibility
NEMA found that states and local jurisdictions have made progress in developing advanced assets, such as hazardous materials (HazMat) teams
A much larger proportion of the United States is accessible to an advanced HazMat response team within a four-hour drive, compared to 2001
Areas accessible to an advanced HazMat response team within a four-hour drive
Source: NEMA “Homeland Security Grant Return on Investment,” July 2018

18. Increased Capabilities
Incident Management and Search and Rescue Team Accessibility
Similarly, NEMA found that the proportion of the United States covered by advanced Incident Management Teams (IMTs) and structural collapse/Urban Search and Rescue (USAR) teams within a four-hour drive has increased from 2001 to 2018
Areas accessible to an advanced IMT within a four-hour drive
Areas accessible to an advanced structural collapse/USAR team within a four-hour drive
Source: NEMA “Homeland Security Grant Return on Investment,” July 2018

19. Grant Effectiveness Case Studies
Detailed Insight on Grant Results
Highlight how preparedness grant recipients allocate funding and measure grant project outcomes
Improve understanding of how states and urban areas use grant funds to build and sustain capabilities
Help FEMA communicate the importance of the preparedness grants to external audiences, such as Congress and the public

20. Methodology and Findings
2016 – 2017 Case Studies
Methodology and Findings
Focus Area
Guiding Research Question
Findings
Risk and Capability Assessments
How are grant recipients using the results from risk and capability assessments to inform their grant allocation decisions and homeland security strategies?
Used to engage stakeholders
Process was burdensome relative to usefulness of results
Establishing Grant Priorities
How are recipients determining which grant projects to fund?
Used THIRA and SPR results, subject-matter experts, and state homeland security strategies
Applied population-based formulas and competitive awards to pass funding to local jurisdictions
Building and Sustaining Capability
How have grant funds helped recipients build and sustain capability and close capability gaps?
Prioritized shareable and deployable resources
Challenged aligning grant projects to core capabilities
Validating Project Outcomes
How are grant recipients measuring and reporting grant project outcomes?
Challenged developing quantitative performance metrics that evaluate grant project outcomes
Assessed project effectiveness through after-action reports and analyses

21. Case Study: District of Columbia
Notable Projects
Fatality Management Plan
DC developed a District-wide Fatality Management Plan, which includes agency- specific playbooks and describes the roles of all relevant agency, regional, and Federal partners
Situational Awareness Dashboard
The Dashboard assists with emergency alerts and enables streamlined data analysis and report generation for any incident across the National Capital Region (NCR)
AlertDC
AlertDC provides notifications on conditions affecting public safety, such as police activity or severe weather
The Situational Awareness Dashboard integrates DC, NCR, and Federal data

22. Case Study: Montana Notable Projects
Community Animal Response Teams (CARTs)
After nearby states experienced a blizzard that led to livestock losses, Montana invested grant funds to create CARTS trained on sheltering and care for animals during disasters
Situation Analyst Montana (SAM)
SAM is a web-based, interactive platform that enables emergency response officials and other stakeholders to access a common operating picture of incidents
Incident Management Team Training
Grant funds enable Montana to build and maintain a specialized team of emergency response professionals to coordinate local and regional incident response
CART members supported the 2017 Rolling Cow exercise, simulating the challenge of a livestock transport accident

23. Return on Investment Study
Effects of HSGP investments are unclear
Grant descriptions are broad and general
Local baseline capability data are unavailable
Challenging to account for other funding sources for projects
Barriers to Conducting a Return on Investment and
Cost Benefit Analysis for HSGP Projects
Analysis was Limited by Data Gaps and Other Issues
FEMA Identified Possible Options for Reducing Barriers to Analysis
Improving data collection and analytics

24. Return on Investment Strategies
Strategy 1: Improve Quantitative Data
Develop a data strategy to increase and improve quantitative data collection
Strategy 2: Revealed Preference Model for State-Level Analysis
Measure how valuable HSGP-funded preparedness projects are to emergency managers
Strategy 3: Detailed Operations Model –
Capability-Specific Analysis
Analyze a well-defined segment of the HSGP portfolio (e.g., mass search and rescue) and use a blend of data analysis techniques to produce a project investment benefit conclusion
Strategy 4: Breakeven Analysis
Pilot study to demonstrate how to develop a near-term investment benefit for the prevention portfolio of HSGP

25. Next Steps on Grant Effectiveness
FEMA is continuing to work on activities that align to the FEMA Strategic Plan and NPS to further enable stakeholders to demonstrate the effective use of their preparedness grant funding. Activities include:
NPAD will conduct grant effectiveness case studies focused on real world incidents in 2018
FEMA will specifically examine how jurisdictions used grants to prepare for and respond to the 2017 hurricane season
NPAD will continue to improve data collection and analysis systems to help state, local, tribal, and territorial partners measure grant investment outcomes

26. Conclusion
“Achieving the vision of a prepared and resilient Nation is a shared responsibility. Meaningful improvements will occur only when we work in concert with leaders from Federal, state, local, tribal, and territorial governments and non- governmental organizations and the private sector to enhance our collective readiness.” -Brock Long, FEMA Administrator FEMA Strategic Plan