IV. Implementation, monitoring and evaluation

a) Implementation and monitoring

Implementing, monitoring and steering the behaviour-change intervention is no different in principle to any other type of intervention: it requires having a well-thought out implementation plan, sufficient resources (people, materials and money) to carry it out, and a monitoring plan that helps the project manager identify and address challenges and opportunities for improvement that arise.

Here’s a list of essential ingredients for an effective behaviour-change intervention in practice:

• Recruitment of staff and volunteers, and ensuring their availability for the behaviour-change activities;
• Training, supervision and support of staff and volunteers, with a focus on providing useful feedback, which is a particular challenge when there are many of them engaged in activities such as household visits;
• Production of materials such as communications products, physical items and services required to carry out the behaviour-change activities – either directly, or through an implementing partner or service provider;
• Formation and support of any clubs, committees and groups required for the behaviour-change intervention;
• Evaluating and improving any pilot activities before scaling up;
• Engaging the community of the target population and important local stakeholders so as to have maximum support for the behaviour-change intervention;
• Coordination with other components within the AAH project or programme, and with other initiatives that may influence the behaviour-change intervention (e.g. other organisations working on the same issues or with the same population);
• Keeping up with the delivery schedule as planned, or changing it in a controlled way if needed;
• Ensuring activities are carried out well, with the use of a quality checklist for each one;
• Having regular feedback sessions with field staff and meetings with community members to discuss experiences and seek improvements.

All these aspects of implementation require active monitoring and management with the help of the monitoring plan developed during the design phase.

b) Evaluation

The behaviour-change intervention may be evaluated as part of the evaluation of the project within which it sits, or as a specific intervention. The list below presents evaluation criteria, questions and possible measurement techniques for evaluating the behaviour-change intervention.

Evaluation criteria: provision (activities)

Evaluation questions:

• Was the intervention delivered as planned?
• Were the planned activities carried out at the scale and with the quality intended?
• What adjustments were made and why were they made?

Measurement techniques:

• Refer to monitoring records.
• Examine promotional materials and protocols produced for the intervention.

Evaluation criteria: coverage

Evaluation questions:

• What percentage of the target population was reached by the behaviour-change intervention (exposure)?
• What was the intensity of exposure (for example, did people just hear a thirty-second radio spot, or did they also receive a household visit and participate in a community meeting)?

Measurement techniques:

• Refer to monitoring records.
• Include questions on coverage in a survey.
• Use interviews to assess intensity of exposure.

Evaluation criteria: behavioural impact (results)

Evaluation questions:

• Did provision and coverage result in practice of the intended behaviour and, if so, to what extent?
• Were there any groups within the target population who were reached by the intervention but who did not adopt the new behaviour?
• If so, why?

Measurement techniques:

• Do structured observation of behaviours (e.g. observe people washing their hands).
• Carry out spot checks to look for behaviour proxies (e.g. check for the presence of soap and water).
• Include self-reporting questions in a survey (e.g. ask people about their handwashing practice).

Provision and coverage should be monitored during implementation of the behaviour-change intervention as well as in the evaluation. Behavioural impact should ideally be evaluated by comparing behaviours before and after the intervention.

Evaluating the impact of behaviour change on health, livelihoods etc. is a complex process requiring careful study design and substantial resources. In most cases, it is sufficient to assume that adopting the intended behaviour leads to impact at a higher level, as established by previous studies.