Effectiveness of Early Warning Messages

Outcome

% of [specify target group] who took protective action after receiving an early warning message

This indicator measures to what extent people who received an early warning message took at least one protective action in response, such as evacuating, securing assets, or helping others. It is most relevant for hazards where there is enough lead time to act (e.g. floods, cyclones, or volcanic eruptions). Used together with the indicator Reach of Early Warning System, it helps assess whether warnings not only reached households but also motivated appropriate action, which is essential for reducing disaster impacts.

Determine the indicator’s value by using the following methodology:

 

1) Conduct quantitative survey with a representative sample of households who are covered by the early warning system. During the survey, identify households who received an early warning message during the specified hazard or event, using the methodology for indicator Reach of Early Warning Systems. Ask the following question:

RECOMMENDED SURVEY QUESTION (Q) AND POSSIBLE ANSWERS (A)

Q1: What action did your household take after receiving the warning message? Probe: What else did you or your household members do?

A1: (allow multiple answers; adjust the options to your context)

• Evacuated to a safe place

• Warned or assisted others

• Secured important items, livestock, or documents

• Switched off electricity or gas / moved to higher ground

• Followed the instructions in the message (specify)

• Decided to take no action

• Wanted to take action but could not (e.g. due to mobility issues)

• Other - specify: ..................................

• Don't remember

 

2) Count as “protective action” all actions that directly reduced risk or increased safety. When interpreting “no action” responses, distinguish between households that chose not to act and those that were unable to act due to constraints (e.g. mobility limitations, dependence on others).

 

3) To calculate the indicator’s value, divide the number of households who received the warning and took at least one protective action by the total number of households who received the warning. Multiply the result by 100 to convert it to a percentage.

Disaggregate by gender, age group, location, type of communication channels, type of protective action taken, dependency status (e.g. households with older persons, children, or persons with disabilities), and other relevant criteria.

1) Keep in mind that you can only collect the required data from respondents who received an early warning message. In a standard household survey sample, many households may not have received a warning at all (or may not recall receiving one). Those households cannot answer the follow-up questions on actions taken, so they will be excluded from this indicator. As a result, the number of eligible responses may be much smaller than your planned sample, especially when analysing smaller groups. If resources allow, increase the overall sample size so that you still have enough respondents who received a warning to produce reliable results.

2) In your analysis, report separately:

• % of people reached by early warning messages who took the action(s) recommended in the official message(s); and

• % of people reached by early warning messages who took other effective action(s)

This helps assess whether official recommendations were both understood and realistic.

3) Consider asking respondents who took no action, or who were unable to act, about the reasons why:

Q2: "Can you please tell me why did you not take any action?” Probe: "Was there any other reason?"

A2: (allow multiple answers; adjust the options to your context)

• I did not receive the warning early enough

• I did not understand the warning / what to do

• I did not understand how to do what I was asked to do

• I thought my household was not at risk

• I could not evacuate due to disability/health/age

• I could not leave because of caring responsibilities (children/elderly/sick)

• I could not leave because of livestock/assets/property

• I could not afford to take action (costs/transport)

• There was no safe place to go / shelter was too far

• I waited for more instructions from authorities / community leaders

• Other - specify: .............................................

• Don’t know / Don’t remember

4) For hazards like floods, cyclones, or volcanic eruptions, lead time is essential for evacuation and asset protection. Consider asking whether the message was received early enough to take action. Example of recommended question:

Q3: "Did your household receive the warning with enough time to take protective action?"

A3: Yes / No / Don’t remember

For sudden-onset hazards (e.g., earthquakes, flash floods), the window for action may be too short, so timing data might be less relevant.

5) Keep the definition of ‘protective action’ consistent across rounds and locations to ensure comparability.

6) Protective action may be taken with assistance. For some groups action may depend on support from family members, neighbours, or authorities; such groups include older persons, children, people with disabilities. Where relevant, record whether protective actions were taken independently or with assistance to better understand inclusion gaps.