1. Developing and Communicating Warnings
Example of the UK Met Office in delivery of warnings
Graeme Forrester, SWFDP, Pretoria, November 2013

2. In this session… Brief Overview of the UK Warning Service
Methods of communicating warnings
Communicating uncertainty
Assessment of Warnings
Questions?

3. The UK National Severe Weather Warning Service (NSWWS)
“Legislation supporting the Civil Contingencies Act 2004 states that (emergency) responders must have regard to the Met Office duty to warn the public and provide information and advice if an emergency is likely to occur or has taken place.”
To: The public and civil emergency authorities
Why: So that mitigating action can be undertaken to reduce
potential impacts.
When: Ahead of the possibility of significant disruption to
transport and communications, but the main driver is the
potential risk to life and property.

4. What the NSWWS warns for:
Severe Weather - Rain, Wind, Ice, Snow and Fog.
When the weather is forecast to pose widespread disruption or a danger to life.

5. National Severe Weather Warning Service (NSWWS)
Two levels of message
Alert – given more than 24 hours in advance
Warning – within 24 hours of a severe weather event
A need for a warning is assessed against two considerations:
The likelihood of an event
The potential impact of an event
To be honest, the difference between the two “levels” is only one of timing so there is a debate as to whether we should dispense with the distinction.
(Some similarity between flood alerts and flood warnings given by our hydrological agencies.
Key point about our warning system is that it is impacts based (since 2011), following consultation with stakeholders in 2009.
Now it’s likelihood and impact level which is considered, rather than meteorological thresholds.

6. Risk Matrix We then plot them on the risk matrix X
HIGH
MED
LOW X
VERY LOW
MEDIUM
IMPACT
LIKELIHOOD
Then assign a colour to the warning which is a combination of potential impact and likelihood

7. Severe Weather Warnings Colour Codes – Basic Messages
NO SEVERE WEATHER EXPECTED
Keep up to date with latest forecast
BE AWARE
Remain alert and keep up to date with latest forecast
BE PREPARED
Remain vigilant, keep up to date with latest forecast and take precautions where possible
TAKE ACTION
Remain extra vigilant, keep up to date with latest forecast. Follow orders and any advice given by authorities and be prepared for extraordinary measures
There is a basic message for each of the colours though
The traffic light colour coding, introduced into our warning system a few years ago will remain with the same general messages. Note though that “No Severe Weather expected” does not rule out some minor impacts from the weather, for example, heavy rain i.e. some surface water on roads etc. However, such impacts would not be expected to impinge greatly on emergency responders.

8. Risk Matrix
The colour of every alert/warning is based on a combination of anticipated impacts and likelihood of occurrence.
Important to note that it is a combination which determines the colour.
Let’s look at the two aspects of that risk

9. Likelihood Allows us to highlight uncertainty
In general uncertainty is greater at longer lead times. Mostly due to:
Uncertainty re exact location of severe weather
Uncertainty re severity of weather
Most alerts at 3 to 4 day lead time will be assigned a low/very low likelihood

10. Impacts Very important to ascertain which impact column is being used
Unwise to get into habit of dismissing yellow warnings/alerts
“it’s only a yellow”
Yellow could be a very low likelihood of HIGH impacts

11. Impacts
Generic impacts here. Specific tables are available for the 5 elements (wind, rain, snow, ice and fog)
Public tables available on our website.

12. Impacts http://www.metoffice.gov.uk/weather/uk/advice/
Very Low
Low
Medium
High
Impact and advice associated with RAIN
Isolated flooding of low-lying land and roads – risk of aquaplaning.
Minimal disruption to infrastructure and resources.
Isolated instances of spray/waves overtopping coastal locations.
Localised flooding of property and susceptible roads possible – be aware.
Stay vigilant. Water on roads – drive accordingly to the conditions.
Flooding affecting parts of communities.
Potential danger to life due to fast flowing/deep water.
Potential damage to buildings/structures.
Possible impacts on human health due to contaminated water.
More widespread disruption to infrastructure and resources.
Failure or overtopping of defences affecting small or parts of communities.
Localised evacuation may be necessary.
Widespread flooding affecting whole communities.
Danger to life due to fast flowing/deep water.
Prolonged loss of critical infrastructure and resources
Failure or overtopping of defences, affecting large communities.
Large scale evacuations may be required.
A list of typical impacts at the four levels (Very Low, Low, Medium, High) is given on the website.
This slide gives some generalisations for rain. On the website there are other impacts specific to snow, wind, ice and fog
Other impacts specific to snow, wind, ice and fog also listed.

13. Warning text
The text of the warning contains the key information about the weather we’re expecting, as well as giving an indication of some of the likely impacts.
The Chief Forecaster’s Assessment is a space for the Chief to elaborate a little on the expected conditions as well as explain where any uncertainties in the forecast lie.
Text needs to be clear
free of jargon
easy to understand
concise
The main text of the warning gives the expected weather and and idea of impacts. – the main part of the message
An additional section – the Chief Forecaster’s Assessment – gives the forecaster the chance to elaborate and explain any uncertainties.
All the text should be clear, concise and free of jargon but to be honest that is still something we need to work on with the forecasters!

14. Communicating the Warnings
So that’s the basic system, how to we communicate these warnings
Communicating the Warnings

15. Communicating the warnings
Our challenge is that we have to communicate the message to the public and to emergency responders.
Need to use a wide variety of methods to maximise “reach”
All warnings are displayed on the Met Office website
AMBER and RED warnings are disseminated by and/or fax to emergency responder organisations

16. UK warnings on the web Homepage ticker banner
All warnings are displayed on our website.
Banner on our home page if warnings have been issued.

17. UK warnings on the web National maps and summary
Click on the banner to take you to more details.
This was a particularly complex example last March where we had a lot of late snow in several parts of the UK and rain in the south!

18. UK warnings on the web Regional maps and text details
Can go down to regional level too.

19. UK Warnings on the web Challenges
Other weather can be complex with different weather happening in different parts of the country!
Overlapping warning areas or two warnings for different elements (e.g. wind and snow) can look confusing,
We need to try to keep the picture as simple as possible.
But keeping the picture simple, sometimes makes the messaging more difficult!

20. How weather information reaches the public
Public website
Forecasts for media (TV, Radio, newspapers)
Social Media
Smartphone Apps
Mobile Weather
Widgets
RSS feeds
In fulfilling its public task the Met Office has a broad remit to act in the public’s best interest, and the requirement to protect life and property is at the centre of what we do.
Public Website
Free for use at
Forecasts for media
Basic national and regional forecasts provided to the public without charge and distributed through various channels including TV, radio and the internet.
Widgets and Gadget
Widgets are things that you can download onto your Windows Vista desktop, Mozilla Firefox browser or iGoogle page
Gadgets are things that webmasters (people who build webpages) can use to embed Met Office weather within their pages (this was used extensively in the snow during the winter of 2009/10 – their first season)
RSS Feeds
Available from the website to be taken by your preferred RSS reader
Really Simple Syndication'. In essence, the feeds themselves are just web pages, designed to be read by computers rather than people
iPhone
Our app designed for the iPhone received a huge amount of interest when it was first launched propelling it to number three in the most downloaded app
Mobile Weather
Designed for the basic mobile with browsing capability; a cut down version of the main Met Office website; the weather you need on the move.
Designed for basic web enabled phones. During the volcanic ash incident extra pages were added, these received how many hits over how long.
As the UK's national meteorological service, the Met Office provides accurate and reliable weather forecasts on TV and radio, in print, and online. Now, you can get our forecasts wherever you go with the free Met Office iPhone application.
What information is available?
Five-day forecasts for locations.
Regional forecast out to five days.
UK forecast out to 15 days.
Mountain area forecasts.
Weather warnings.
Observations for the last three hours.

21. Alerting the public via Social media
Increasingly we are making use of social media to extend the reach of our forecasts and our warnings.
Twitter (140 character messages) – we have over 170,000 followers
We tweet when we issue warnings with links to our web pages.
We have a 24/7 desk dedicated to Twitter – we respond to our followers tweets
Many of our stakeholders (police, local authorities etc.) re-tweet our messages.
We use Facebook as well
You Tube – explainers – Chief Forecasters explain what is happening.
Web Logs (Blogs) – used a lot by our press office as explainers and also as a means of calming down some of the more extreme predictions in the newspapers.
All helps to educate the public as well as get our message out.

22. Communicating Uncertainty
Issuing warnings isn’t easy though and there is invariably some uncertainty about what is going to happen. How can we best communicate that uncertainty?
Communicating Uncertainty

23. Communicating Uncertainty
Interesting discussion started on this yesterday
Maybe we can continue it here!
Forecasts are inherently uncertain.
The challenge
How can we convey that uncertainty without compromising our credibility?
Is the problem different when we are communicating with the civil protection managers to that when we are communicating with the public?

24. Communicating Uncertainty
Risk Matrix – likelihood
Allows us to highlight uncertainites
The longer the lead time, the greater the uncertainty (normally!)
Uncertainty in severity or in location
The tick in the box highlights uncertainty but doesn’t explain it.
Disaster managers understand risk. They understand this matrix as they use it almost daily for all their risk assessments.
Not so easy for the public

25. Communicating Uncertainty
Language
“There is a chance of a shower”
“Temperatures may rise as high as 35C”
“It should stay dry until after dark”
“At this stage it looks most likely to affect the south of the country but…
Can appear to be vague
Can be understood differently by different people
“You said there was a chance of a shower but it was dry all day here”!

26. Communicating Uncertainty
Probability
“There is an 80% chance of a shower”
“There is an 20% chance of a shower”
“There is a 60% chance of rain in the morning but only 30% chance in the afternoon.
Seems more scientific, but is it?
Do people fully understand probabilities?

27. Communicating Uncertainty - using graphics
At longer range, we’re finding graphics to be very good for explaining what might happen as an alternative
Example from a storm a few weeks ago. Graphic produced a few days out and updated – on our web, tweeted and on facebook. Also used in TV broadcasts & Video explainers
Feedback on such graphics is very good.

28. Verifying/Assessing Warnings
But once the warning is out there we need to make sure it has given useful information and, where necessary, changed people’s behaviours. How can we verify or assess our warnings?
Verifying/Assessing Warnings

29. Why do we need to assess? What happened? Did we forecast it?
At the right time
In the right place
What were the impacts?
Was the warning communicated effectively
Improve understanding, better warning next time
When there is a severe weather event we need to know what happened – what was the severe weather, and what were the impacts of that weather. This will improve our understanding and help us to improve our knowledge so that next time we will hopefully be able to forecast an event better
Did we issue a warning? – we need to keep a database of severe weather and its impacts not only for events that we have forecast,but also for events that we have ‘missed’
Was the warning issued for the right time? Did we issue it with enough lead time for people to do something about it? Was it forecast for the right place? This is all information that we need to keep so that we can improve our understanding and improve our warning service. WE also need to guage whether our warning was communicated effectively – did people understand it and know what to do.

30. What do we do? Verification Objective verification
Did we issue a warning when a threshold was exceeded?
Subjective verification
Did the warning give good advice?
What were the impacts?
Was the timing correct?
Was the area correctly identified?
Can we identify any learning?
We keep all our warnings and after each severe weather event we verify those warnings, objectively – the easy bit….and subjectively, much more difficult

31. Subjective verification
Verifying the impacts
Media reports
Twitter feeds
Impacts reported by responders
Verifying the usefulness
Right place?
Right time?
Enough lead time?
Good advice?
Subjective verification is more difficult – trying to asses whether we got the warning right compared to the impact…of course, we hope that the impacts will be less because we have issues the warning so that also complicates it!
We use websites like the BBC and newspapers – but that has limitations as it can depend on what else is happening in the news, we can use social media, and – use the advisors to liaise with responders and ask them what happened.
We also try to guage the usefulness of the warning…
Initially we tried a method which gave a “hit” or “ miss” as an assessment but it soon became clear there were shades of gray in between. We may not have got a warning exactly right but what we did issue may have had some use to the customers and helped in some way.
This has led to a marking scheme

32. Subjective Verification “Hot” Assessment
Impact Levels 3
Impact column ticked in warning is consistent with impacts experienced 2
Impact column ticked in warning is within one level of impacts experienced e.g. if warning indicated “significant” impacts while those experienced were “low” 1
Impact column ticked in warning is within two levels of impacts experienced e.g. if warning indicated “significant” impacts while those experienced were “very low”
Impacts were reported and no warning was issued or no impacts were reported.
Comments as necessary (Add any further information which influenced your marking)
Assessment Score for Impacts (0,1,2 or 3)
Area Affected 3
All impacts noted were within the warning area. 2
The impacts occurred generally within the area indicated but the area is deemed to be too large or slightly too small 1
The area is generally displaced from the main impacts but a few impacts occurred within it
No warning was issued or there were no reported impacts in the area identified by the warning.
Comments as necessary (Add any further information which influenced your marking)
Assessment Score for Area (0,1,2 or 3)
Validity Time 3
All the impacts were noted within the warning validity time and the warning was issued at least 2 hours before the start validity time 2
Most of the impacts occurred within the validity time while others were no more than 2 hours outside the period 1
Some of the impacts occurred within the validity time but most occurred within 2 hours either side of the period
No warning was issued or none of the impacts identified occurred within the validity time period.
Comments as necessary (Add any further information which influenced your marking)
Assessment Score for Validity Time (0,1,2, or 3)

33. Subjective Verification “Hot Assessment
Overall Marking Assessment
0-2
Very Poor
Warning was missed or gave very poor guidance to customer, perhaps being classed as a “False Alarm”
3-5
Poor Guidance
Although a warning was issued it gave poor guidance to the customer
6-7
Good Guidance
A warning was issued which gave generally good guidance to the customer
8-9
Excellent Guidance
The warning issued gave excellent guidance to the customer
Target is to get at least 60% of all warnings assessed as “Good” or “Excellent” Guidance

34. Questionnaires/Surveys
The key assessment of a warning though can only be done by the user – the emergency responder or the public
Our advisors seek feedback from responders in the area affected by the weather
We also carry out ad hoc surveys of the public after a severe weather event.

35. Ad-hoc public surveys Carried out after warnings issued via telephone
Around per year
As much as possible try to survey different parts of the country and different warning types
Try to assess whether public have heard the warning and have acted on it

36. Survey Questions Were you aware of the weather warning?
How did you see or hear about the warning?
Did you change your plans as a result of the warning?
Overall usefulness of the warning?
Overall accuracy of the warning?
How do you prefer to receive weather warnings?

37. Example of National Severe weather warnings survey
Snow warning in Southern & Central Wales and the West Midlands
16th January 2013
Use an external company
Short telephone interviews in the areas from 18th to 21st January to measure awareness of the warning, whether it prompted a change in people’s plans and how useful it was.

38. Headlines
91% of respondants saw or heard the weather warning – mostly via TV bulletins
55% claimed to have taken action based on the warning.
95% thought the warning was useful
96% thought the warning was accurate
59% of those who did not see or hear the warning said that they would have changed their plans had they been aware of it
63% prefer to get their warnings from TV
8% prefer to get their warnings from radio

39. Results (500 respondents)

40. Results

41. Results

42. Questions?