Understanding of the Promoted Messages

Output

Process

% of [specify the target group] who correctly understand what the promoted messages are asking them to do

The indicator measures the proportion of target group members who can correctly explain what selected social and behaviour change (SBC) messages are asking them to do when the communication materials are presented to them. It helps assess whether the messages are sufficiently clear to the intended audience.

Determine the indicator’s value by using the following methodology:

 

1) Select the materials that your activities use to communicate those messages whose clarity you want to assess (for example, posters, billboards, radio recordings, text messages, voice messages, etc.). If you use several materials with exactly the same message (i.e. no difference in the message's wording, format, etc.), choose only one material per one message. 

 

2) Set the minimum number of messages whose meaning the respondent needs to understand in order to meet the indicator (for example, at least 3 out of 5 messages). For each message, define what a correct answer must include. Depending on the message, this might include the intended action, when to take it, and any essential conditions.

 

 

3) Conduct individual interviews with members of the target group. If the aim is to estimate the indicator's value for the wider target population, use a representative sample. If the aim is to pre-test messages before they are disseminated, a small, varied group of intended users can be sufficient to identify problems and improve the materials, but the results should not be presented as a representative population estimate.

RECOMMENDED QUESTIONS (Q) AND POSSIBLE ANSWERS (A)

Q1: Now I would like you to [choose: look at this material / listen to a recording]. Let the respondent read it / listen to it. Do you think that this [specify the material / audio recording] is asking you to do anything in particular?

A1: yes / no / does not know

 

(ask the following question only if the previous answer is „yes“)

 

Q2: What exactly is the [specify the material / audio recording] asking you to do?

Ask respondents to explain the action in their own words, without suggesting the correct answer. If the answer is too general, use a neutral follow-up question, such as "Can you explain a little more?" Assess the answer against the criteria defined in step 2. Do not count an answer as correct if it omits an essential part of the message.

A2: _

1) the respondent clearly understands what the message is asking her/him to do

2) the respondent does not (clearly) understand what the message is asking her/him to do

 

Repeat the questions for all the messages you want to assess. Present each material in the language and format normally used with the intended audience. As far as practical, reflect the conditions under which people would normally encounter it. For example, do not automatically read a poster aloud if people are usually expected to read it themselves, as this would test a different way of communicating. Consider recording any difficulties respondents experience in accessing the message, such as difficulties with reading, hearing, seeing or understanding its language.

 

 

4) Count the number of respondents who correctly understood the minimum number of messages. 'Understanding' means that the respondent knows what the messages are asking them to do.

 

5) To calculate the indicator’s value, divide the number of respondents who correctly understood the minimum required number of messages by the total number of respondents who completed the assessment. Then multiply by 100 to obtain a percentage.

Count incorrect answers and responses indicating a lack of understanding (e.g. “does not know”) as not understanding the relevant message. Record refusals and incomplete assessments separately. Also report the percentage of respondents who correctly understood each message to identify which messages need improvement.

This data can be disaggregated by age group and literacy level, and, if relevant, also by gender, language, location, and disability status.

1) Ensuring that people correctly understand what the messages are asking them to do should be done through pre-testing (see guidance below) before SBC communication activities start. The main purpose of collecting data after the activities have been implemented is to gain a better understanding of how clear the messages are to the target audience. If a representative sample is used, the results can also provide a more reliable estimate of understanding across the target population.

 

2) This indicator is suitable for messages asking people to take a specific action. If the messages mainly provide information without asking people to do anything, use questions that assess understanding of that information instead. Where different language or format versions could affect understanding, assess the relevant versions separately.

3) Consider where to place these questions in the questionnaire so that answers are not influenced by previously asked questions and do not influence answers to subsequent questions.

 

4) Make sure that the data collectors clearly understand the meaning of each assessed message and can assess whether the respondent truly understands what the message is asking her/him to do (including what to do if the respondent provides an answer that is too general).

Also consider using the following indicators:

• Information in the Preferred Language (helps assess whether communication is delivered in languages preferred by the intended audience)

• Information Through the Preferred Channel (helps assess whether communication reaches people through their preferred channels)

• Trustworthiness of Social and Behaviour Change Messages (assesses whether people believe the content of the messages)

• Intention to Adopt a Behaviour (assesses whether people intend to adopt the promoted behaviour)

• Prevalence of Barriers to Change (identifies obstacles that can prevent people from taking action even when they understand the messages)