Monitoring & evaluation

If you can’t measure it, you can’t manage it.

Peter Drucker

Monitoring is an ongoing process that provides project stakeholders with early indications of progress towards the achievement of results. Evaluation is the systematic and objective assessment of an ongoing or completed project, program, or policy. Evidence-based monitoring and evaluation (M&E) activities are essential tasks in the development and implementation of children’s health and mobility initiatives.

5 KEY LESSONS

1. Budget

Monitoring and evaluation are essential components of successful interventions for child mobility and health and require explicit budget allocations.

2. Use results to improve performance

Strategic information obtained from data collection efforts can inform decisions about how to improve the project on an ongoing basis.

3. Facilitate participation

Participatory monitoring and evaluation involving stakeholders at various levels can improve community buy-in.

4. Develop a set of meaningful indicators

The monitoring and evaluation process should focus on a limited set of key indicators to simplify the data collection process.

5. Monitor traffic violence

Pedestrian and cyclist safety is most effectively measured by recording the number of people killed or injured in the project area.

Best practices

An M&E plan is a document that outlines how an implementation project is monitored and evaluated, and that links strategic information obtained from various data collection systems to decisions about how to improve the project on an ongoing basis. The M&E plan serves several main purposes, including:

  • Stating how achievements of the project will be measured;
  • Documenting consensus, thereby encouraging transparency, accountability and responsibility;
  • Guiding implementation of M&E; and
  • Preserving institutional memory.

Before the M&E plan is developed, the stakeholder groups involved in the planning of the M&E process must be identified. M&E objectives should be shaped and informed by stakeholder input. Once stakeholders are engaged and objectives are developed, the M&E plan will clarify project components, defines the relationships between the elements key to implementation, and helps identify the internal and external issues that can affect the project’s success.

An M&E plan is built on the key parameters of a project, which include the:

  • Overall goal or desired change or effect
  • Main beneficiaries or audience of the project
  • Hypotheses or assumptions that link the project objectives to specific interventions or activities
  • Project scope and size
  • Extent of participation in and capacity for M&E
  • Project duration
  • Overall project budget

M&E are essential components of successful planning initiatives for which a budget should be established. In general, project managers should factor in an additional 3 to 10 per cent to the project’s budget for M&E activities.

Case study: Evaluating the success of school area interventions in Dar es Salaam

The School Area Road Safety Assessments and Improvements (SARSAI) program developed by non-governmental organisation Amend seeks to reduce child injury through measures that reduce vehicle speeds. SARSAI employed a rigorous evaluation process to evaluate whether a package of interventions, including speed bumps, footpaths, and signage, would have a statistically significant impact on road traffic injury for children at schools in Dar es Salaam, Tanzania. The evaluation tested motor vehicle speeds before and after the installation of the traffic calming features and determined that speeds fell by nearly 60 percent. Schools where the interventions were tested saw a reduction of one road traffic injury per year per 286 pupils.

Speed bump implemented outside a school in Dar es Salaam through the SARSAI programme.
 

Identifying project monitoring indicators can be the most difficult part of the M&E process. One size does not fit all, and M&E systems must be tailored to local conditions. For example, performance indicators must suit a country’s norms and values, and not be taken off the shelf from international agencies and used “as is.”

In some cases, a common set of indicators is developed, while in other instances different stakeholder groups develop their own sets of indicators. In practice, it is most useful to keep the number of monitored indicators between three and fifteen. Controlling the number of indicators results in greater management attention on the most meaningful ones (key levers) and it also helps manage the M&E budget.

When developing indicators, it is important to assess the link between their activities and the ultimate change it achieves (outcome) through periodic external evaluations. Project inputs (e.g. financial, human resources) and outputs (e.g. number of people directly affected by the activities) should be monitored regularly.

Tracking progress on improving the walking and cycling environment in Lagos

The Lagos Metropolitan Area Transport Authority (LAMATA) recently developed a Non-Motorised Transport (NMT) Policy to guide interventions to improve the walking and cycling environment. Key to the Policy’s strategic importance is a set of clear indicators outlining how conditions need to approve over the short, medium, and long term. Indicators to be monitored under the project include:

  • Mode share for walking, cycling, and public transport.
  • Vehicle kilometres traveled by private vehicles.
  • Number of fatalities and injuries from traffic crashes.
  • Ambient emissions of harmful local pollutants and greenhouse gas emissions
Participants deliberate on the Lagos Non-Motorised Transport Policy at a stakeholder workshop.

 

Sustainable Development Goal indicators

Developed by the Inter-Agency and Expert Group on SDG Indicators (IAEG-SDGs), the Sustainable Development Goal indicators were adopted on 6 July 2017 by the United Nations General Assembly. Two indicators in particular are relevant to child health and mobility: 3.6.1, the death rate from road traffic crashes, and 11.2.1, the proportion of the population with convenient access to public transport.

Data collection can include the use of both quantitative and qualitative methods and tools. Quantitative methods can include community surveys, interviews, and observations. Qualitative methods can include various learning methods incorporating visual, interviewing, and group tools. Representation of data in the form of clear maps and infographics can help ensure that the results of the M&E process are transparent and easily understood.

While data analysis is often thought of as a rather mechanical and expert-driven task, M&E represents an opportunity to actively engage with stakeholders in the critical analysis of the program and discussion of appropriate actions to be taken based on the findings.

A representation of the presence of footpaths in the city of Kisumu, Kenya.
 
A tracking diagram indicating the paths taken by pedestrians across an intersection can help in identifying preferred crossing locations.
 

Participatory monitoring and evaluation (PM&E) is a process through which various stakeholders engage in monitoring or evaluating a particular project. The stakeholder groups typically involved in a PM&E activity include: the end users; intermediary organizations, including NGOs; private sector businesses involved in the project; and government staff at all levels. PM&E differs from traditional approaches in that it engages key project stakeholders more actively in reflecting and assessing the progress of their project and in particular the achievement of results.

For M&E, it is important that local people are active participants—not just sources of information. In this process, the stakeholders evaluate, while outsiders facilitate. The focus is on building stakeholder capacity for analysis and problem-solving, and the PM&E tasks build commitment to implementing any recommended corrective actions.