Disaster Recovery: Lessons from Katrina Gail Ann McCreary, CRM Mississippi Power Company
Hurricane Katrina – August 29, 2005 Worst natural disaster in U.S. history Worst natural disaster in U.S. history Affected Alabama – Mississippi – Louisiana Affected Alabama – Mississippi – Louisiana 250 miles wide 250 miles wide 140 mph winds 140 mph winds 30 ft tidal surge along coast line 30 ft tidal surge along coast line
Were Dealing With… Rain and wind damage Rain and wind damage Tornadoes Tornadoes Flooding Flooding “Coal water” “Coal water” Submerged records Submerged records Bio-hazard Bio-hazard
Determined Priorities Damage incurred Damage incurred Business unit and function Business unit and function Criticality to restoration effort Criticality to restoration effort Building access issues Building access issues
The Plan…. Identify records needing to be dried Identify records needing to be dried Pack into boxes Pack into boxes Stage on pallets Stage on pallets Shrink wrap Shrink wrap Load into freezer truck Load into freezer truck
First Priorities Substation Construction Substation Construction Powerco Powerco Plant Watson Plant Watson
Plan Changed Downtown buildings had to be vacated Downtown buildings had to be vacated All dry records All dry records All PCs, printers, phones All PCs, printers, phones Employee personal effects Employee personal effects Undamaged furniture and modular Undamaged furniture and modular Office supplies Office supplies 400 employees 400 employees
New Issues Needed more boxes!!!! Needed more boxes!!!! Needed temporary office space for 400 employees Needed temporary office space for 400 employees What to do with all the “stuff” What to do with all the “stuff” Employees still on storm duty Employees still on storm duty 2-3 weeks 2-3 weeks
Plan B DR would stay and pack dry records too I did not have the manpower I did not have the manpower DR was experienced DR was experienced Had supply and manpower resources in place Had supply and manpower resources in place
Determined Numbering Scheme Building ID Building ID Floor number Floor number Location number Location number = General Office 2 nd floor cubicle #202 SI = G&SI 2 nd floor location # = Hancock Bank 8 th floor location #842
Challenges Faced Locked doors Locked doors On-going building clean-up & restoration On-going building clean-up & restoration No elevators No elevators No electricity No electricity Storage for boxes Storage for boxes
Back to “Normal” Back to “Normal”
Phase II Multiple temporary offices Multiple temporary offices No storage in temporary offices No storage in temporary offices No file cabinets No file cabinets Delivery issues Delivery issues Palletized boxes not in order by location Palletized boxes not in order by location
Phase II Employees would have to review Employees would have to review Coordinate with occupancy schedule Coordinate with occupancy schedule Named coordinators for each site Named coordinators for each site Small volumes delivered where possible Small volumes delivered where possible
Location No. ______818 _________ Name _Lateral file __ SkidBox No.DestroyTookDeliveredInterimR/C
Options Take Take Destroy Destroy Interim Interim Records center storage Records center storage
Lessons Learned - Disaster Recovery Process Utilize document recovery contractor Utilize document recovery contractor
Lessons Learned - Disaster Recovery Process Utilize document recovery contractor Utilize document recovery contractor Understand the process / clarify costs Understand the process / clarify costs
Lessons Learned - Disaster Recovery Process Utilize document recovery contractor Utilize document recovery contractor Understand the process / clarify costs Understand the process / clarify costs Utilize multiple people Utilize multiple people
Lessons Learned - Disaster Recovery Process Utilize document recovery contractor Utilize document recovery contractor Understand the process / clarify costs Understand the process / clarify costs Utilize multiple people Utilize multiple people Work closely with contractor Work closely with contractor
Lessons Learned - Disaster Recovery Process Utilize document recovery contractor Utilize document recovery contractor Understand the process / clarify costs Understand the process / clarify costs Utilize multiple people Utilize multiple people Work closely with contractor Work closely with contractor Walk-thru after packing completed Walk-thru after packing completed
Lessons Learned - Disaster Recovery Process Utilize document recovery contractor Utilize document recovery contractor Understand the process / clarify costs Understand the process / clarify costs Utilize multiple people Utilize multiple people Work closely with contractor Work closely with contractor Walk-thru after packing completed Walk-thru after packing completed Have floor plans for all buildings Have floor plans for all buildings
Lessons Learned - Disaster Recovery Process Utilize document recovery contractor Utilize document recovery contractor Understand the process / clarify costs Understand the process / clarify costs Utilize multiple people Utilize multiple people Work closely with contractor Work closely with contractor Walk-thru after packing completed Walk-thru after packing completed Have floor plans for all buildings Have floor plans for all buildings Document! Document! Document! Document! Document! Document!
Lessons Learned - Disaster Recovery Process Need business continuity plan Need business continuity plan
Lessons Learned - Disaster Recovery Process Need business continuity plan Need business continuity plan Definition of vital records has changed Definition of vital records has changed
Lessons Learned - Disaster Recovery Process Need business continuity plan Need business continuity plan Definition of vital records has changed Definition of vital records has changed Have emergency contact numbers Have emergency contact numbers
Lessons Learned - Disaster Recovery Process Need business continuity plan Need business continuity plan Definition of vital records has changed Definition of vital records has changed Have emergency contact numbers Have emergency contact numbers Post-critique Post-critique
Lessons Learned - Facilities Get to know your Facilities department Get to know your Facilities department
Lessons Learned - Facilities Get to know your Facilities department Get to know your Facilities department Establish liaisons with field/branch offices Establish liaisons with field/branch offices
Lessons Learned - Facilities Get to know your Facilities department Get to know your Facilities department Establish liaisons with field/branch offices Establish liaisons with field/branch offices Limit employees’ personal items Limit employees’ personal items
Lessons Learned - Facilities Get to know your Facilities department Get to know your Facilities department Establish liaisons with field/branch offices Establish liaisons with field/branch offices Limit employees’ personal items Limit employees’ personal items No locked file cabinets or overheads No locked file cabinets or overheads
Lessons Learned - Facilities Get to know your Facilities department Get to know your Facilities department Establish liaisons with field/branch offices Establish liaisons with field/branch offices Limit employees’ personal items Limit employees’ personal items No locked file cabinets or overheads No locked file cabinets or overheads Take office keys with you Take office keys with you
Lessons Learned - Facilities Get to know your Facilities department Get to know your Facilities department Establish liaisons with field/branch offices Establish liaisons with field/branch offices Limit employees’ personal items Limit employees’ personal items No locked file cabinets or overheads No locked file cabinets or overheads Take office keys with you Take office keys with you No critical or only copy records in disaster susceptible areas No critical or only copy records in disaster susceptible areas
Lessons Learned - Personnel Storm procedures not heeded Storm procedures not heeded
Lessons Learned - Personnel Storm procedures not heeded Storm procedures not heeded Records preparation should be part of storm preparation and communications Records preparation should be part of storm preparation and communications
Lessons Learned - Personnel Storm procedures not heeded Storm procedures not heeded Records preparation should be part of storm preparation and communications Records preparation should be part of storm preparation and communications Files should be labeled / identified Files should be labeled / identified
Lessons Learned - Personnel Storm procedures not heeded Storm procedures not heeded Records preparation should be part of storm preparation and communications Records preparation should be part of storm preparation and communications Files should be labeled / identified Files should be labeled / identified Enforce corporate policies Enforce corporate policies
Lessons Learned - Personnel Storm procedures not heeded Storm procedures not heeded Records preparation should be part of storm preparation and communications Records preparation should be part of storm preparation and communications Files should be labeled / identified Files should be labeled / identified Enforce corporate policies Enforce corporate policies People are stressed People are stressed
Lessons Learned - Personnel Storm procedures not heeded Storm procedures not heeded Records preparation should be part of storm preparation and communications Records preparation should be part of storm preparation and communications Files should be labeled / identified Files should be labeled / identified Enforce corporate policies Enforce corporate policies People are stressed People are stressed Promote disaster preparedness and vital records protection Promote disaster preparedness and vital records protection
Silver Linings Files were cleaned out Files were cleaned out –Inactive records sent to storage –Destruction of expired and unnecessary records –Reduction in file volume
Silver Linings Files were cleaned out Files were cleaned out –Inactive records sent to storage –Destruction of expired and unnecessary records –Reduction in file volume New contacts with business units New contacts with business units
Silver Linings Files were cleaned out Files were cleaned out –Inactive records sent to storage –Destruction of expired and unnecessary records –Reduction in file volume New contacts with business units New contacts with business units Multiple opportunities to promote RIM Multiple opportunities to promote RIM
Silver Linings Files were cleaned out Files were cleaned out –Inactive records sent to storage –Destruction of expired and unnecessary records –Reduction in file volume New contacts with business units New contacts with business units Multiple opportunities to promote RIM Multiple opportunities to promote RIM Department credibility enhanced Department credibility enhanced
Silver Linings Disaster planning for records now priority Disaster planning for records now priority
Silver Linings Disaster planning for records now priority Disaster planning for records now priority Records protection procedures part of storm manual Records protection procedures part of storm manual
Silver Linings Disaster planning for records now priority Disaster planning for records now priority Records protection procedures part of storm manual Records protection procedures part of storm manual No people – or records – in Lower Level No people – or records – in Lower Level
Silver Linings Disaster planning for records now priority Disaster planning for records now priority Records protection procedures part of storm manual Records protection procedures part of storm manual No people – or records – in Lower Level No people – or records – in Lower Level EOC and critical functions moved from coastal zone EOC and critical functions moved from coastal zone
Silver Linings RIM storm assignment changed RIM storm assignment changed
Silver Linings RIM storm assignment changed RIM storm assignment changed Employees will not be complacent about next storm Employees will not be complacent about next storm
Conclusions Be proactive vs. reactive Be proactive vs. reactive
Conclusions Good sense, sound judgment and creativity are necessary Good sense, sound judgment and creativity are necessary
Conclusions Be proactive vs. reactive Be proactive vs. reactive Good sense, sound judgment and creativity are necessary Good sense, sound judgment and creativity are necessary Don’t be complacent Don’t be complacent
Conclusions Be proactive vs. reactive Be proactive vs. reactive Good sense, sound judgment and creativity are necessary Good sense, sound judgment and creativity are necessary Don’t be complacent Don’t be complacent Expect the unexpected Expect the unexpected
Conclusions Be proactive vs. reactive Be proactive vs. reactive Good sense, sound judgment and creativity are necessary Good sense, sound judgment and creativity are necessary Don’t be complacent Don’t be complacent Expect the unexpected Expect the unexpected Evaluate suppliers and services Evaluate suppliers and services
Conclusions Be proactive vs. reactive Be proactive vs. reactive Good sense, sound judgment and creativity are necessary Good sense, sound judgment and creativity are necessary Don’t be complacent Don’t be complacent Expect the unexpected Expect the unexpected Evaluate suppliers and services Evaluate suppliers and services Empower people Empower people
Questions? Gail Ann McCreary, CRM