RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Item
- Title
-
RBM Framework for Monitoring
Progress and Evaluating
Outcomes and Impact - extracted text
-
World Health Organization
20 Avenue Appia, CH 1211 Geneva 27
Switzerland
RBM Framework for Monitoring
Progress and Evaluating
Outcomes and Impact
FINAL VERSION
*
Founding Partners:
'^5?
e
-
■
'r
1
Table of contents
1.
Introduction
1.1
1.2
What is Roll Back Malaria
Monitoring and Evaluation of Health Programmes
1
1
2.
Framework for Roll Back Monitoring and Evaluation
3
2.1
2.2
General principles
The Monitoring and Evaluation Framework
3
3.
Roll Back Malaria indicators
7
3.1
7
7
3.3
3.4
3.5
Case definitions
Proposed core indicators for monitoring and evaluating
Roll Back Malaria
General comments on indicators
Applicability of indicators to different regions
Global indicators
4.
Possible approaches to data collection
14
4.1
4.2
4.3
4.4
4.5
Routine surveillance (Health Information System)
Demographic Surveillance System
Community and Household Surveys
Health facility assessment
Review of documents
15
15
5.
Planning the implementation of Roll Back
Monitoring and Evaluation at regional and country levels
17
6.
Developing tools for data collection
17
7.
7.1
7.2
Annexes
Comparative table of global indicators per WHO Region
Case definitions (Extract from WHO Expert Committee on Malaria,
WHO Technical Report, Series 892, WHO, Geneva - 2000)
Table of Proposed RBM Global Core Indicators
References
3.2
M
7.3
7.4
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - August, 2000
5
9
13
13
15
16
16
List of Abbreviations
African Regional Office
African Initiative for Malaria
AnnualParasite Incidence
AFRO
AIM
ACDR
!L
CHW
DALY
DSS
EDM
GDP
HIS
HS
IDS
,____
IMCI
Snp
MOH
OR
RBM
RJ1
WHO
Crude D™
Bi
ya
Demographic Surveillance Systems
Essential Drugs and Medicine
Gross Domestic Product
Health Information System
Heal.b Sysiem
■
‘■•♦vxf*5>. ' ■«.’
?-,b
s sffifc
■A ’ i aa
••
?
“S^MX^Sdbuud,
Insecticide Treated Net
Operational Research
I-• I'C- rl
^17™^'°'’"“*
World Health Organizatio!
,B
aaaaii
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 1
1. Introduction
1.1 What is Roll Back Malaria ?
Roll Back Malaria is a new global initiative against malaria. RBM has been built on
the foundations of the accelerated implementation of malaria control in the African region,
which was based on the Regional and Global strategy for Malaria Control.
Its objective is to halve the malaria burden in participating countries through interventions
that are adapted to local needs and by reinforcement of the health sector. The principal
mechanism for achieving this is through intensified national action by country-level
partnerships working together towards common goals within the context of health sector
development and using agreed strategies and procedures. Country partnerships will be
supported by a sub regional, regional and global partnerships, and technical support networks
from these three levels will provide the necessary technical assistance. Roll Back Malaria will
also encourage strategic investments in the development of better tools and intervention
strategies through focused support for research, including operational research.
Following its launch in October 1998, Roll Back Malaria started with a preparatory
phase, which lasted until December 1999. During this period, intercountry Roll Back Malaria
inception meetings have been held and country level Roll Back Malaria inception processes
were started. RBM operations will soon begin in many countries and their effectiveness will
need to be monitored and evaluated.
An effective system for monitoring progress and evaluating outcomes and impact will
be critical for the success of Roll Back Malaria. Roll Back Malaria will need to report on
progress and lessons learned, on reduction of mortality and morbidity as well as on economic
impact. This information will be crucial for identifying areas where modifications may be
needed in relation to the intervention strategies and allocation of resources in subsequent
phases of Roll Back Malaria at national, sub-regional, regional and global levels.
1.2 Monitoring and evaluation of Health Programmes
Monitoring is needed to verify step by step the progress of Health Programmes at
district, provincial, national, regional and global levels e.g. to verify whether activities have
been implemented as planned, ensure accountability, and to detect any problems and/or
constraints in order to provide local feedback to the relevant authorities and to support them
for promoting better planning through careful selection of alternatives for future action. For
this purpose process indicators must be carefully selected.
Evaluation of outcomes and impact is needed to document periodically whether defined
strategies and implemented activities lead to expected results in terms of:
• outcomes: to document e.g. treatment seeking, improved quality of treatment,
changes in knowledge, attitudes, and behaviour at community level or on the
performance of key components of the local health care system such as the improved
quality of services, rate of coverage, establishment of inter-sectoral linkages so that
improvements can be made where and when needed
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 2
•
impact: the assessment of impact, e.g. the measure of the desired change in terms of
reduction of mortality, morbidity or economic losses. The selection of impact
indicators and the collection of data needed for their calculation, is by far the most
difficult step in the evaluation process.
While monitoring is a continuous process, evaluation will need to be conducted
intermittently. The periodicity of evaluation varies considerably according to the changes
expected in the different areas evaluated.
OBJECTIVES OF RBM
PLANS OF ACTION
PROCESS OR
FUNCTIONING
INTERMEDIATE RESULTS
RELATED TO PRIORITY
INTERVENTIONS OF THE
PROGRAMME
IMPACT
WHAT HAS TO BE
MONITORED AND
EVALUATED
EXAMPLES OF INDICATORS
TO BE SELECTED
Process indicators
should
check that what was
planned
(i) has been carried out
(ii) on time
% of health personnel involved in
patient care trained in malaria case
management and IMCI
Outcome indicators should
reflect the changes in
knowledge, attitudes,
behaviour or facility
resources specified in the
outcome objectives
Mortality and morbidity
reduction
% of patients with uncomplicated
malaria getting correct treatment,
at health facility and community
levels, according to national guidelines
within 24 hours of onset of symptoms
Malaria death rate
(probable and confirmed)
among target groups
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 3
2.
Framework for RBM Monitoring and Evaluation
The World Health Organization has developed this proposal for a framework along
withjndicators for monitoring the progress and evaluating the outcomes and impact of Roll
Back Malaria. This framework relies on the large amount of past work accomplished by the
WHO Regional Offices and on the more recent efforts of the WHO Regional Offices for
Africa and for Eastern Mediterranean during the Accelerated Implementation of Malaria
Control in Africa in 1997-1998. To avoid any duplication of efforts, it is therefore proposed
that all national partnerships use this general framework, and that they select within the
framework those indicators that are most appropriate for their specific epidemiological
situation and intervention strategy.
Since malaria epidemiology is determined by natural factors which does not fit with the
political shape of WHO Regions, it is quite difficult to have regional indicators which will be
applicable to the three main typologies of malaria situation :
• high endemic countries, i.e. Sub-Saharan Africa;
• countries where malaria has been controlled although is still a problem, i.e. Asia and Latin
America;
• countries with disappearing malaria and malaria free countries with receptive areas, i.e.
North Africa and Southern Republics of the former USSR.
Despite the extensive variation of malaria epidemiology between and within regions and
countries, this framework proposes 5 global indicators, of which at least 3 should be used by
all the regions.
2.1 General principles
In developing the framework and indicators, the following general principles were
taken into account:
Relevance to RBM objectives
The framework and indicators should be directly relevant to the RBM objectives.
They should enable the monitoring of the principal malaria control interventions and related
efforts to reinforce the health sector and allow an evaluation of the impact of RBM action on
the malaria burden.
Availability and timeliness of the information
Data must be available without undue delay during both implementation and
monitoring/evaluation phases.
Reliability
Indicators and criteria must be consistent and dependent across applications or time.
Standardised but adaptable approaches
The epidemiology of malaria, intervention strategies and health sector development
vary considerably between regions and countries, and this variation will need to be taken into
account in the monitoring and evaluation process. It has been attempted, therefore, to
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 5
2.2 The monitoring and evaluation framework
The Monitoring Group has developed a proposed monitoring and evaluation
framework that identifies five “critical areas” for monitoring the progress and evaluating the
outcomes and the impact of Roll Back Malaria. These five critical areas relate directly to the
objective of RBM and include:
the impact on malaria burden i.e. mortality, morbidity and economic losses
(i)
the improvements in malaria prevention and disease management including prevention
(ii)
and control of epidemics
the related health sector development
(iii)
the intersectoral linkages which need to be created or reinforced
(iv)
the support and partnerships.
(v)
It is proposed that all regions and RBM countries use this framework to develop their
own monitoring and evaluation system. The framework would then be the same for all
regions and countries, but the indicators for each critical area could vary between sub-regions
and/or regions and countries according to the local malaria epidemiology and the actual
strategy for rolling back malaria.
The diagram below shows the framework for Roll Back Malaria monitoring and
evaluation as proposed by the Monitoring and Evaluation Group, indicating the five critical
areas for monitoring. The shaded boxes in the diagram refers to country-level monitoring and
evaluation.
Critical areas for Monitoring and Evaluating RBM
Malaria Burden
Impact
Malaria Prevention
and Disease Management
|
Prevention and Control of
Epidemics
Prevention
Health Policy
Health Sector Development
|
Community
Action
Early Diagnosis and
Prompt Treatment
Service Delivery. | .
HS
Management
Intersectoral
Intersectoral Linkage
Support/Partnership
Linkages
National
Partnership
R&D
Technical support
______
Global Partnership
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 6
The framework describes the main components of RBM, especially at the country level.
The ultimate objective of RBM is to halve the burden of malaria, and one of the critical areas
to evaluate is obviously the impact of RBM on disease burden, i.e. mortality, morbidity and
economic impact. The reduction in burden will be achieved through interventions that are
initiated by the national Roll Back Malaria partnership and this partnership is another critical
area to monitor and evaluate. The actual interventions will vary according to malaria
epidemiology and status of the health sector. However, malaria specific interventions will
include the critical areas of: (i) prevention, e.g. prevention of malaria during pregnancy, use
of insecticide-treated materials and other vector control measures; (ii) early diagnosis and
treatment of the disease; (iii) prevention and control of epidemics in epidemic-prone areas and
situations; (iv) community action; (v) operational research. To deliver these interventions,
there is a need to strengthen, and thus monitor and evaluate, the relevant components of the
health sector. These range from health policy, health systems management, and service
delivery especially at first line health facilities. A problem as important and complex as
malaria control is not just an issue for the health sector, and the involvement of other
important sectors such as agriculture, education or meteorology needs to be monitored and
evaluated also.
These interventions will require international support, and other critical areas to monitor
and evaluate are the resources made available at all the levels, the technical support provided
to countries, and the effectiveness of Research and Development to develop new tools and
control strategies.
REM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 7
3
RBM indicators
It is recommended that the principal monitoring and evaluation system of RBM at
regional, sub-regional and country levels be based on a small number of core indicators that
will represent each of the critical areas in the monitoring and evaluation framework. These
indicators should be intervention-oriented and provide information for action at the relevant
operational level, especially at the district level. Through an extensive review of documents
and consultative process, the RBM monitoring group has proposed a set of RBM core
indicators by critical area as defined in the monitoring and evaluation framework.
3.1 Case definitions
The wide variety of malaria data collected and notified by different malaria control
programs makes comparison between countries difficult. However, as stated in the Twentieth
Report of the WHO Expert Committee on Malaria, “...agreement on standard definitions of
malaria morbidity and mortality, and on a limited number of “indicators” that could be used
in all situations to monitor malaria control activities, would represent a major step forward.
The use of core indicators will not preclude countries from collecting other information they
consider necessary to monitor progress of their individual plans of action for malaria
control.” Thus, it would be useful if groups of countries or sub regions with similar
epidemiology and intervention strategies would select the same minimum group of indicators.
This would greatly facilitate, at the global level, the monitoring of regional initiatives to Roll
Back Malaria and of inter-country review of progress at regional level. It is also
recommended that the information for mortality and morbidity, where appropriate and
possible, be dis-aggregated by age, sex, and socio-economic status.
It is also strongly recommended that the official case definitions of severe malaria,
uncomplicated malaria, therapeutic failure and of some other indicators as formulated in the
above mentioned WHO Report be used (Annex 2). These definitions differentiate between
countries where parasitological diagnosis is possible and countries where this is not possible.
3.2 Proposed core indicators for monitoring and evaluating RBM
A provisional list of core indicators for country and/or regional level is given per critical
area in the box on the next page. A detailed description of the proposed indicators (classified
into impact, outcome and process) is given in Annex 3. For each indicator, this table gives its
operational definition, the method of data collection, the source of information, the level of
data collection, and the periodicity of data collection. The table lists also some comments.
As mentioned before, it is proposed that each sub-region, region or country select from this
list only those indicators that it considers important for the local epidemiology and
intervention strategy. It is recommended that at least two indicators, one process and one
outcome, be selected from each critical area in the framework.
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 8
Proposed RBM Core Indicators
I. Impact
1.
2.
3.
4.
5.
6.
Crude death rate among target groups
Malaria death rate (probable and confirmed cases) among target groups
. ■' •
% of probable, and confirmed malaria deaths among patients with severe malaria admitted to a health' facility ; ■
dumber of cases of severe malaria (probable and confirmed) among target-groups'
Number of cases of uncomplicated malaria (probable and confirmed) among target groups.
Annual Parasite Incidence (API) among target groups (by region/according to the, epidemiological situation)
H. Malaria prevention and disease management
Prevention
1. % of countries having introduced pyrethroids for public health use and insecticide-treated materials in the list of essential 1
drugs and materials
.(
, 1
'2. % ofservice providers (Kealth^personnel, CHW.„) trainedin techniquesogtreatment of nets and/orindoor spraying
according to the national policy
'
3. % of households having at least one treated bednet :
4. % of pregnant women who; have taken chemoprophylaxis or intermittent drug treatment • according to the national drug
policy
5. % of antenatal clinic staff trained in preventive intermittent antimalarial treatment for pregnant women
Prevention and'control of epidemics
1. % of countries with epidemic prone areas/situation having a national preparedness plan of action for early detection and
control of epidemics
; , •.
2. % of malaria epidemics detected within two weeks of onset and properly controlled
Early diagnosis and Prompt Treatment
1. % of health personnel involved in patient care trained in malaria case management and IMCI
2. :% of health facilities able to confirm malaria diagnosis according to the national policy (microscopy, rapid test etc.)
3. % of patients hospitalised with a diagnosis of severe malaria and receiving correct antimalarial and supportive treatment
according to the national guidelines
• .
,
4. % of patients with uncomplicated malaria getting correct treatment at health facility and community levels according
to national guidelines within 24 hrs of onset of symptoms
1
HI. ;Hea 1th Sector Development
Health Policy
1. % of districts with plans of action reflecting national’health policy
2. % of districts using health information for planning
3. % of countries having a policy of universal coverage for all with a basic package including relevant malaria control ■
activities
....
Service Delivery
T. % of health facilities reporting no disruption of stock of antimalarial drugs, as specified in the national drug policy, for
more than one week during the previous .3 months
Community Action
...
1. % of countries having national guidelines for malaria prevention and treatment including training of all the informal
health providers and recommendations for home treatment of febrile illness/suspected malaria, recognition of the most
frequent signs of danger for children, prevention of malaria during pregnancy, and use of insecticide treated materials . .
2. % of villages/communities with at least one Community Health Worker trained in management of fever and recognition
of severe febrile illness 1 ,
’ .
,,
.
3. % of mqthers/caretakers.able to recognise, signs and symptoms of danger of a febrile disease in a child <5 years'
IV. Intersectoral linkages
1. % of countries with multisectoral and .inter-agencies partnership established
2. % of countries having established official linkages, including the elaboration of research agenda of public health interest,
between research institutions and Ministry of Health
V. Support/Partnership
1. % of countries with agreed national RBM budget met by donor funding
2. % of countries with functional sentinel sites for surveillance efficacy of 1st and 2nd line antimalarial drugs
3. Number of antimalarial drugs which have progressed, to the level of phase III trials
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 9
3.3 Some general comments on the indicators
Impact indicators
Six core indicators are proposed in this framework : three for mortality, and three for
morbidity.
Mortality
The objective of RBM is “to half the burden of malaria through interventions adapted
to local needs and strengthening of the health sector”. Given that malaria mortality is by far
the most important contributing factor to the burden of malaria as measured in DALYs, it is
proposed that malaria related mortality be the principal impact indicator for RBM.
The relevant mortality indicators in endemic areas are the Crude Death Rate (CDR)
and the Malaria Death Rate (MDR) of children aged 0-59 months, by age and sex.
Both CDR and MDR are recommended because:
• there are circumstances where changes in CDR can be reliably measured, but changes in
MDR cannot: deaths counted in surveys sometimes cannot accurately be attributed to
malaria.
• where CDR is low, interventions against malaria can have indirect as well as direct
benefits, reducing deaths partly attributable to other conditions; it is highly desirable to
quantify these additional, indirect benefits.
• where CDR is high, malaria is sometimes a “competing risk”, in which case fewer malaria
deaths are offset by more deaths from other causes; when there is no measurable change
in CDR, we need to distinguish between two possible explanations - the failure of malaria
control and compensating mortality.
• a number of RBM interventions will not be specific to malaria, e.g. management of
anaemia in pregnancy.
N.B.: In sub-Saharan Africa where Plasmodium falciparum is predominant, case fatality rate
for severe malaria among under-five children and pregnant women or other target groups will
be assessed in a few sentinel sites (district or national hospitals).
Morbidity
Although RBM’s global target for malaria control has been expressed in terms of
mortality (50% reduction in deaths by 2010), many countries will need to set morbidity
indicators as impact targets.
In sub-Saharan Africa and in other regions where Pfalciparum is common or reemerging as a problem, the number of cases of severe malaria/cerebral malaria reported per
year is a good indicator for indirect measurement of the effective treatment of uncomplicated,
malaria both at health facility and at the community level. In many countries the only data
presently reported routinely are the number of malaria cases (severe and uncomplicated), and
the majority of these cases are based on presumptive diagnosis rather than parasitologic
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 10
confirmation. While these data are limited and frequently represent only a small proportion
of malaria cases, if there are no major changes in the reporting system, an understanding of
these limitations will allow for use of the data to generate estimates of the overall burden of
disease affecting communities and for tracking of trends over time.
Outside Sub-Saharan Africa and Papua New Guinea, the single best, “core indicator”
is the Annual Parasite Incidence (API) i.e. the number of microscopically confirmed malaria
cases detected during 1 year per unit of population, by age, sex and parasite species, as
measured through routine surveillance. Many countries in Europe, Asia, Oceania, North
Africa and Latin America have shown that they can measure API by routine surveillance
(patients with symptoms contacting health services), and identify Plasmodium spp by
microscopy. A morbidity indicator will suit countries where reducing incidence is the
principal goal, and reduced incidence is likely to mean fewer deaths. Furthermore, many
countries have high case loads, but few malaria deaths, e.g. where P. vivox predominates, and
where most P. falciparum cases receive adequate treatment.
Economic losses
Malaria imposes a harsh economic burden on families who are least able to pay for
treatment, prevention costs and loss of income. In addition, malaria-endemic countries must
use scarce hard currency on drugs, bednets and insecticides in an effort to control malaria.
According to estimates from a recent Harvard study commissioned by RBM, Africa's GDP
would be 32% greater today if malaria had been effectively controlled 35 years ago. Since
the issues related to assessment of the economic burden of malaria are complex, RBM has
convened a special working group to develop indicators and guidelines for tracking changes
in the economic burden at the household, national, and macroeconomic levels. It is expected
that by the end of 2000 recommendations from this work group will be available for
dissemination as an addendum to the present document.
Malaria prevention and disease management
Prevention
The proposed core indicators for malaria prevention reflect the most important
interventions for reducing the global burden of malaria.
• Hence, though Insecticides Treated Nets (ITNs) or other materials will not be used
everywhere, this intervention is considered critical for reducing malaria mortality and
morbidity, especially among children and pregnant women in areas with stable
transmission in Africa. In areas of unstable malaria transmission, other age groups must
be considered.
•
Preventive intermittent treatment during pregnancy is recommended as an appropriate and
effective method for reducing the consequences of malaria in pregnancy in highly
endemic areas, especially for first and second pregnancies. At present only a few countries
in Africa have adopted such preventive intermittent treatment as national policy. There is
a need to have good data on the effectiveness of preventive intermittent treatment during
pregnancy and chemoprophylaxis where it is delivered. It is expected that other countries
will follow in the near future.
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 11
Disease management
Early diagnosis and appropriate treatment is the most important action against malaria.
The eight core indicators proposed (five outcome and three process) will provide information
on the changes in the quality of the management of the disease and allow national authorities
to take appropriate action, if required. One of the outcome indicators is defined as the
“Percentage of patients with an uncomplicated malaria getting correct treatment at health
facility and community levels, according to the national guidelines, within 24 hours of the
onset of symptoms”. Where and when good diagnostic and clinical services are available and
actively used, this indicator may be restricted to parasitologically confirmed cases of malaria.
However, in most endemic areas, especially in Africa, health services are not widely available
or used and more than 80% of the cases are managed at home. In such areas, a more
appropriate indicator is the “Percentage of persons (or mothers/caretakers for children < 5 )
who report at community level that within 24 hours after fever began the patient received
the recommended first line antimalarial or was brought to the health facility”.
Preliminary results of pre-testing an instrument to measure this at the community level
indicate that less than 7% of children with fever in endemic areas in Africa get appropriate
treatment for malaria. Evaluation of the improvement in this indicator will be extremely
important.
Prevention and control of malaria epidemics in epidemic- prone areas and situations
A good indicator for epidemic prone areas and situations is the timely detection of the
epidemic, i.e. within two weeks after the notification of the first cases, and the correct control
of the epidemic. This includes, according to tire national policy, correct treatment of cases
and appropriate vector control measures such as the use of insecticide treated materials and/or
indoor spraying.
Health Sector Development
The two core indicators proposed for health sector development (1 outcome and 1
process) are not restricted to malaria but touch on broader health sector issues that need to be
taclded to ensure effective malaria control. They measure the presence of the relevant health
policies, and the implementation of these policies at the operational level (usually the district).
It is also proposed to monitor whether these operational units have adequate funds and skilled
staff to implement the policies and to provide basic services (pre-testing showed that some
health districts spent over 95% of their budget on personnel and that they had virtually no
funds to provide services). Below are two examples of indicators dealing with malaria related
services as well as with management of the disease:
1) Percentage of health services reporting no disruption of stock of antimalarial drugs (as
specified in the national drug policy) continuously for one week during the previous 3
months.
2) For areas where this is policy: whether there is a functional parasitological laboratory, i.e.
a laboratory with one functional microscope, at least one trained personnel and available
reagents and material according to the national policy
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
pis' 3)^.
poo
Page 12
Inter-sectoral collaboration
Since implementation of RBM activities will require the agreement and involvement
of more than just the health sector (e.g., education, finance, environment, agriculture
ministries etc.) it will be important to monitor and evaluate the collaboration across these
sectors. It is proposed to monitor the collaboration in operational research between national
research institutions and the Ministry of Health because collaboration between research and
control is often reported to be poor. It will also be important to monitor whether prevention
and treatment seeking for malaria is taught in primary schools in endemic areas and whether
environmental risk factors for malaria are taken into account in the planning of development
projects.
Community action
The involvement of the community will be critical in most areas. Since knowledge,
behaviour and attitude changes depend on many factors which differ from one place to an
other one, local definitions of indicators will be required. Some examples of such indicators
are proposed in view of helping the regions and the countries to define the most appropriate
indicators according to their local sociological context.
Partnership and support
An effective national partnership is the key to success for Roll Back Malaria. It is
proposed to monitor whether the national partnerships really bring all potential partners
together, and whether these partners generate the necessary resources to roll back malaria in
the countries through a tracking system.
With respect to technical support, it will be assessed whether the countries are
satisfied with the technical support provided to them.
The main composite indicator for monitoring the global, regional and national
partnerships is the percentage of malaria endemic countries/districts with well defined
strategies for health for all accompanied by explicit resources allocation to RBM, whose
needs for external resources are receiving sustained and adequate support from partners.
Research and development
For Research and Development, two core indicators have been selected which are
considered most relevant for the short and medium term.
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 13
3.4
Applicability of indicators to different regions
The monitoring of RBM at national and global level will be mainly a matter of
aggregating the results for the country specific indicators.
3.5
Global indicators
Although most indicators will vary between countries, there are five indicators that are
considered so important that they have been selected as global indicators. It is recommended
that all RBM countries report on these global indicators wherever they apply. The five global
indicators are:
• Malaria death rate (probable and confirmed cases) among target groups (under-five and
other targets groups)
• Number of malaria cases, severe and uncomplicated (probable and confirmed) among
target groups (under-five and other targets groups)
• Proportion of households having at least one treated bednet
• Percentage of patients with uncomplicated malaria getting correct treatment at health
facility and community levels, according to the national guidelines, within 24 hours of
onset of symptoms
• Percentage of health facilities reporting no disruption of stock of antimalarial drugs (as
specificied in the national drug policy) for more than one week during the previous three
months.
As mentioned before, the actual selection of RBM core indicators for a given area will
depend on the epidemiological pattern, the health infrastructure and the local intervention
strategy. Annex 4 indicates to what extent the proposed RBM global indicators may be
applicable to countries in different regions of the world, using the regional breakdown of the
WHO. It is hoped that this table will help countries in the same region or sub-region to agree
on similar sets of RBM indicators. This would greatly facilitate the implementation of
monitoring of RBM within a given region.
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 14
4.
Possible approaches to data collection
There are five main approaches to collecting data for the proposed RBM indicators.
These are (i) the regular health information system, (ii) Demographic Surveillance Systems,
(iii) community surveys, (iv) health facility surveys and (v) review of documents. The box
below shows some of the proposed RBM indicators listed by the relevant data collection
method.
Proposed RBM indicators by data collection (some examples)
Routine surveillance (HIS)
• Crude death rate among target groups
• Malaria death rate (probable and confirmed) among target groups
• Number of cases of uncomplicated malaria (probable and confirmed) among target groups
• Annual parasite incidence (by region / epidemiology)
• % of malaria epidemics detected within two weeks of onset and properly controlled
Demographic Surveillance Systems (Africa)
• Crude death rate among target groups
• Malaria death rate (probable and confirmed) among target groups
Community Surveys
•
•
•
•
% of pregnant women who have taken chemoprophylaxis or preventive intermittent antimalarial treatment,
according to the national policy
% of patients with uncomplicated malaria getting conect treatment at community level, according to
national guidelines, within 24 hrs of onset of symptoms
% of mothers/caretakers able to recognise signs and symptoms of a febrile disease in a child <5 years
% of households having at least one treated bednet
Health facility Surveys
•
•
% of health services reporting no disruption of stock of antimalarial drugs (as specified in the national drug
policy) for more than one week during the previous 3 months.
% of health facilities able to confirm malaria diagnosis according to the national policy (microscopy, rapid test
etc.)
Review of documents
■ % of districts using health information for planning
■ % of countries having introduced pyrethroids for public health use and insecticide-treated materials in the list of
essential drugs and medicines
■ % of countries having established effective linkages, including the elaboration of a public health interest
research agenda, between research institutions and MOH
■ % of countries having a policy of universal coverage for all with a basic package including relevant malaria
control activities
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 15
4.1 Routine surveillance (Health Information System)
Both the Health Care System and the Health Information System (HIS) are well
developed in several endemic countries in Asia, America and Europe but only in a few
countries in Africa. Effective reporting systems are often in place which are adequate for the
purpose of RBM . In Africa, where the HIS is often limited, these data can be used in some
districts hospitals, and in some referral hospitals. This approach can also provide information
relevant to monitoring trends in national crude and malaria-related mortality
4.2 Demographic Surveillance Systems
One option for monitoring trends in malaria mortality in Africa where the quality and
reliability of information generated by the HIS is poor, and the HIS rarely provides
infonnation on the burden of malaria at the community level is through Demographic
Surveillance Systems (DSS) in sentinel sites. Such DSS are now operational in 28 sites in 14
African countries, and these sites have joined together in the INDEPTH network that works
towards standardisation of the methodology. Because of the need for reliable information on
trends in malaria mortality, RBM intends to support the INDEPTH network to include
surveillance of malaria mortality in its African sites.
4.3 Community and Household surveys
Community-based infonnation on prevention and treatment practices will be critical
for monitoring the effectiveness of related RBM interventions. Such information is especially
important for monitoring the outcomes and the effects of RBM action in areas where a large
proportion of cases are managed at the home and where the burden of malaria is usually most
severe. Some community surveys have already been conducted but there is a need to replicate
these studies in other countries for this purpose. Community surveys tend to be time
consuming and relatively costly, and they can therefore only be undertaken in selected
sentinel sites in each country and at intervals of 2-3 years. There are several ongoing activities
in which information on the proposed community-based indicators is already being collected.
The Demographic and Health Surveys, funded by USAID and executed by MACRO, has
been extended to include a malaria module which allows the collection of community-based
information on several of the RBM indicators. The same is true for the Multiple Indicator
Cluster Survey of UNICEF which will be undertaken in a large number of countries in the
year 2000. Finally, RBM has developed a methodology for situation analysis which includes
instruments for community level assessment of indicators such as % of underfives sleeping
under ITNs, provision of intermittent treatment in pregnancy, provision of timely and
appropriate treatment of children with fever and community action against malaria. Eight
countries have already undertook a situation analysis during the year 2000 as part of their
RBM strategy development, and the information to be generated will provide important
baseline data on key RBM indicators.
Special surveys are not a sustainable solution to the need for community-based
information and RBM will support the development of alternative approaches that would
enable the Health Information System to routinely collect community level data, and for the
community itself to monitor key RBM indicators.
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
Page 16
4.4 Health facility assessment
The WHO Regional Offices designed Monitoring and Evaluation systems to monitor
the progress, evaluate the outcomes and impact of their respective Regional Strategies for the
Prevention and Control malaria. The Region specific systems, indicators and tools were field
tested, revised and adapted to country specific situations in the respective Regions. These
systems provide the basis for country reports on the burden of malaria. Additionally, a guide
for evaluating the implementation of programmes for the promotion of the use of insecticidetreated nets and other materials in the WHO Region for Africa is being field tested in 14
countries.
IMCI has also developed and tested a methodology and instruments for a multi-country
evaluation of the integrated management of the sick child. The instruments include an
assessment of the clinical skills of health care staff, as well as an assessment of the available
supplies and equipment at the health facility. The evaluation provides all the information
needed for the proposed facility based RBM indicators. The advantage of the IMCI approach
is that the assessment is not limited to skills for the management of malaria only, but that it
addresses the management of the sick child, including malaria. It is therefore proposed to use
for the health facility assessment the relevant sections of the IMCI evaluation methodology or
to rely on the results of the IMCI evaluation where this is undertaken. The application of the
method requires special skills, and it cannot be done in every facility. It is recommended,
therefore, to combine the health facility assessment with the community-based surveys, and to
undertake them in the same districts and at the same interval.
In case the indicator on technical skills of health care staff is not selected, the health
facility assessment becomes much simpler and consists only of an assessment of the presence
of the required antimalarials and, where this is policy, of the presence of parasite detection
services. This simplified facility assessment can be done as part of routine supervisory visits
or be easily combined with the community surveys. But it is recommended that the technical
skills of health care staff be evaluated at least every two years.
4.5
Review of documents
This is the easiest and cheapest of the data collection methods. The main requirement
is that the necessary documents exist and are available, and some special efforts and travel
may be needed to ensure that this is indeed the case. It will also be important to retain copies
of the relevant documents so that these can be made available as supporting evidence for the
monitoring findings on the selected indicators .
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact - Final Version - August, 2000
ANNEX 1
Applicability of the Proposed Roll Back Malaria Global Indicators for
Monitoring and Evaluation in WHO Regions
on
AFRO
Indicators
■
,
--
-
■
-
■
.
.
•
.
_ ,
■_________________________________________________________________:____________________________________________________________________________________________________________________________ ---------------------------------------------------------------------------- :—
-
EMRO
■ _ iB
•
% of patients with uncomplicated malaria getting
correct treatment at health facility and community
levels, according to the national guidelines, within
24 hours of onset of symptoms
% of health facilities reporting no disruption of
stock of antimalarial drugs (as specified in the
national drug policy) for more than one week,
during the previous 3 months
SEARO
7
T . .
________________
................
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Malaria death rale (probable and confirmed cases)
among target groups (under-five and other target
groups)
__________
Number of malaria cases, severe and uncomplicated
(probable and confirmed) among target groups
(under-five and other target groups)
Proportion of households having at least one treated
bednet
EURO
WPRO
SY:
AMRO
-
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Final Version - August, 2000
(Total Death)
Yes
No
(Total Cases)
ANNEX 2
(WHO Expert Committee on Malaria, Technical Report Series 8927p.46-50)
9.1.1
Standardized case definitions
Morbidity and mortality
Definitions of malaria morbidity and mortality will vary, depending on the degree
of diagnostic capabilities at different levels of the health-care system. Whenever
possible, malaria case data should be reported by the patients age group and
parasite species.
In areas without access to laboratory-based diagnosis
• Case of probable uncomplicated malaria — a patient with signs and/or
symptoms of uncomplicated malaria, who receives antimalarial treatment.
• Case of probable severe malaria — a patient requiring hospitalization for signs
and/or symptoms of severe malaria, who receives antimalarial treatment.
• Probable malaria death — death of a patient who has been diagnosed with
probable severe malaria.
In areas with access to laboratory-based diagnosis
• Asymptomatic malaria — laboratory confirmation (by microscopy or
immunodiagnostic test) of parasitaemia in a person with no recent history of
signs and/or symptoms of malaria.
• Case of confirmed uncomplicated malaria — a patient with signs and/or
symptoms of uncomplicated malaria, who receives antimalarial treatment, with
laboratory confirmation of diagnosis.
• Case of confirmed severe malaria — a person requiring hospitalization for signs
and/or symptoms of severe malaria, who receives antimalarial treatment, with
laboratory confirmation of diagnosis.
• Confirmed malaria death — death of a patient who has been
diagnosed with severe malaria, with laboratory confirmation of diagnosis.
The signs and symptoms of malaria to be included in these definitions may vary in
different epidemiological settings (3). The uncomplicated and severe malaria
categories are intended to be mutually exclusive. For example, a patient who
initially presents with uncomplicated malaria but then develops symptoms or signs
of severe disease, should only be classified as having severe malaria, and not
counted twice. This also applies to the probable and confirmed malaria categories.
Thus, countries would be expected to report probable and confirmed cases
separately.
Because of the increasing importance of antimalarial drug resistance to malaria
control efforts, standardized case definition is needed for treatment failures:
• Malaria treatment failure — a patient with confirmed uncomplicated malaria with
a history of having taken the correct dosage and followed the regimen of the
nationally recommended antimalarial treatment, but presents with asexual
parasitaemia on a blood smear within 14 days of the start of treatment.
9.1.2 Indicators
Once standardized case definitions have been agreed upon, indicators can be
developed to measure the progress of the control programme (38). For the
purpose of monitoring and evaluation, the indicators should be closely linked to the
programme's objectives. In deciding how many indicators should be used, accurate
measurement of a small number of core indicators is preferable to imprecise
measurement of too many. As additional resources become available, and the
programme gains experience and makes progress, these indicators can be refined,
improved and added to. Most of the information needed to measure the indicators
can be obtained from three general sources, although these sources may vary
considerably in the quality of data they provide. The information sources are:
1. Routine data collected by the national health-information system (assuming that
standardized case definitions have been agreed upon and used, and these data
are of acceptable quality);
2. Interviews and/or observations in health facilities. These could be carried out
during routine supervisory visits or during special surveys;
3. Specific household or community surveys1.
The first two information sources should be available to all programmes; the third
will require additional programme resources.
Although the cost of conducting such surveys may be high, savings can be made by
measuring several indicators in the same survey.
Core indicators
Although the choice of indicators must be left up to individual national
programmes, the following core (impact and outcome) indicators should be used in
all malaria control programmes irrespective of the local epidemiological situation or
their goals:
Impact indicators
• Morbidity attributed to malaria-.
— Number of cases of uncomplicated malaria (probable and confirmed) among
target groups per unit population per unit time;
— Number of cases of severe malaria (probable and confirmed) among target
groups per unit population per unit time.
• Mortality attributed to malaria:
— number of malaria deaths (probable and confirmed) among target groups per
unit population per unit time;
— case-fatality rate — proportion of probable and confirmed malaria deaths
among patients admitted with severe malaria to a health facility per unit time.
• Malaria treatment failures — Number of microscopically confirmed malaria
treatment failures per number of patients treated. These data should be reported
for each drug used.
Outcome indicators
• Availability of antimalarial drugs — percentage of health facilities reporting no
disruption of the stock of antimalarial drugs (as specified in the national drug
policy) during the previous 3 months;
• Reporting of morbidity and mortality indicators — percentage of districts
'3r>
poo
reporting morbidity and mortality indicators to the national programme on a
monthly basis during the previous 12 months.
Additional indicators
The following additional indicators may be used depending on the epidemiological
situation and the goals of the programme:
• Annual parasite incidence (API) — number of microscopically confirmed
malaria cases detected during 1 year per unit population.
• Use of insecticide-treated mosquito nets — the proportion of target groups
covered by insecticide-treated nets and the proportion that report that they slept
under an insecticide-treated mosquito net the previous night. These indicators
will both require household or community surveys and is relevant to situations
where the programme objectives are to limit and prevent transmission of
falciparum malaria.
• Performance of mothers or carers — proportion of mothers or carers who
ensure correct home management of children with fever, in accordance with
national policies. This indicator will require household or community surveys.
• Protection of pregnant women — proportion of women in their first and second
pregnancies who report per unit time that they have taken chemoprophylaxis or
intermittent drug treatment, according to national drug policies.
• Preparation for malaria epidemics — proportion of epidemic-prone areas that
have an epidemic containment plan and adequate stocks of antimalarial drugs,
supplies, and functioning equipment in place or easily accessible at least 1 month
before the epidemic season begins. This indicator is relevant to situations where
there the programme objectives are to reduce mortality and morbidity,
and
to limit transmission and
prevent epidemics of falciparum
malaria.
• Intradomiciliary spraying of insecticides — proportion of houses sprayed per
total number targeted for spraying. This indicate is suitable for situations where
the programme objectives are to limit transmission and prevent epidemics of
both falciparum and vivax malaria.
• Laboratory diagnosis:
— proportion of health districts where quality control procedures for malaria
control are in place;
— proportion of health facilities with laboratory diagnostic capabilities, of which
an adequate sample of positive and negative slides have been confirmed by a
reference laboratory.
Outcome indicators specific to areas where there is residual or no transmission
Presence of foci of transmission:
• number of villages in which autochthonous cases have been reported since the
beginning of the previous transmission season;
• number of cases investigated (classified by species) and found to be
autochthonous;
• number of malaria cases investigated.
In these areas, mixed infections should be counted as P. falciparum cases.
Annex 3: Operational definitions of proposed (Impact, Outcomes and Process) core indicators for monitoring and evaluation of RBM
Operational Definition
Indicators
■
■
'
_______ - • '_____ _
_________________________________________________________________________________________________________________________________
■■
_____________________________________________________________________________________________________
j
|
Method of data
collection
IMPACT
Source of
information
_________ _
_________
Level of data
collection
Periodicity of
data collection
Comments
_______
.________ •-
I. Relatively inexpensive but
inacurrate
II. More accurate but difficult and
expensive - Surveys require welltrained interviewers
Health facility
reports
DSS, DHS and
health facility
and/or community
surveys reports
I Di., Pr., Co
11.1 Com.
11.2 Com.
11.3 Di.
11.4 Com.
I. 1 Year
II. 1.Ongoing
II.2. 5 years
11.3 2-3 years
11.4 2-3 years
I. Routine
HIS
II. Special surveys
11.1 DSS (INDEPTH)
11.2 DHS
II.3. Health facility
surveys
II.4 Community surveys
Health facility
reports
DSS, DHS and
health facility
survey reports
I. Di., Pr., Co.
II. 1 Com.
11.2 Com.
11.3 Di.
11.4 Com.
See indicator 1
I. 1 Year
II. 1.Ongoing
II.2. 5 years
11.3 2-3 years
11.4 2-3 years
Number of probable and confirmed malaria deaths I. Routine
3 % of probable and confirmed
HIS
malaria deaths among patients with occurring in a target group admitted in a given
severe malaria admitted to a health health facility for unit time, divided by total number II. Special surveys
of probable and confirmed severe malaria cases Health facility surveys
facility
(Inpatient surveillance)
admitted in the same target group for the same
unit time in the same health facility
I. HIS reports
II. Health facility
surveys reports
I. Di., Pr., Co.
II. Di.
I and II : Year I. As part of routine monitoring of
many NMCP - Reliant on
consistent reporting III. Relatively inexpensive and
more accurate
I. Routine
Number of cases of probable and confirmed
4 Number of cases of severe malaria
severe malaria reported per year among < 5 years HIS
(probable and confirmed) among
II. Special surveys
(other target groups)
target groups
Community surveys
I. HIS reports
II. Health facility
survey reports
I. Di.,Pr.,Co.
II. Com.
Year
For areas with P. falciparum.
Case definition of severe malaria
to be clearly defined at country
level according to WHO
definitions. Training is needed.
HIS reports
Di.Pr.-Co.
Year
As part of routine monitoring of
many NMCP - Reliant on
1 Crude death rate among target
groups
2 Malaria death rate (probable and
confirmed) among target groups
I. Routine
Total number of deaths per year among target
group divided by mid-year population of the same HIS
II. Special surveys
target group
11.1 DSS (INDEPTH)
11.2 DHS
II.3. Health facility
surveys
II.4 Community surveys
Total number of malaria deaths (probable and
confirmed) per year among target group divided
by mid-year population of the same target group
Number of cases of uncomplicated malaria
5 Number of cases of uncomplicated
(probable
and confirmed) reported per year
malaria (probable and confirmed)
among < 5 years (other target groups)
among target groups
6 Annual Parasite Incidence (API)
Number of microscopically confirmed malaria
cases detected during 1 year per unit population
(Usually 1000)
HIS
consistent reporting
NMCP/HIS reports Di.,Pr.ICo.
As part of routine
monitoring of many
NMCPs, mainly outside
Africa
Year
Not recommended for hightransmission areas where
specificity is low, nor for areas
where health information systems
are weak or where many malaria
cases are not seen by the health
system.
RBM Framework tor Monitoring Progress and Evaluating Outcomes and Impact
Final Version - August, 2000
1
Annex 3: Operational definitions of proposed (Impact, Outcomes and Process) core indicators for monitoring and evaluation of RBM
Indicators
Operational Definition
Method of data
collection
Source of
information
Level of data
collection
Periodicity of
data collection
Comments
OUTCOMES
_______________ _
Di.,Pr.,Co.
1. supervision
reports
2. Survey reports
3. RBM evaluation
reports
Requires standardised forms,
1:Year
2 & 3: 2-3 years skilled supervisors and training
Di.,Pr.,Co.
1. supervision
1. Part of routine
1. Number of patients (< 5 years other target
reports
supervision of
groups)
presenting
at
a
given
health
facility
for
malaria getting correct treatment at
2. Survey reports
NMCP/district and
one unit time with uncomplicated malaria and
health facility and community
3. RBM evaluation
province health team
receiving
correct
treatment
according
to
national
levels according to national
2.
Health
facility
surveys
reports
guidelines within 24 hrs of onset of guidelines within 24 hours of onset of symptoms
divided by total number of patients (< 5 years and 3. RBM evaluation
symptoms
other target groups) presenting at the same health
facility for the same unit time with uncomplicated
malaria x100
Requires standardised forms,
1:Year
2 & 3: 2-3 years skilled supervisors and training
’ Number of patients hospitalised for one unit lime
with a diagnosis of severe malaria and receiving
diagnosis of severe malaria and
correct antimalarial and supportive treatment
receiving correct antimalarial and
divided by total number of patients hospitalised
supportive treatment according to
with
a diagnosis of severe malaria for the same
the national guidelines
unit time x 100
1 % of patients hospitalised with a
1. Part of routine
supervision of
NMCP/district and
province team
2. Health facility survey
3. RBM evaluation
2 % of patients with uncomplicated
1. District supervision
2. Number of patients (< 5 years other target
groups) who are reported to have had fever in the 2. Community surveys
3. RBM evaluation
previous 2 weeks and reported to have received
the locally recommended antimalarial treatment
within 24 hours of onset of the fever divided by
total number of patients (< 5 years other target
groups) who are reported to have had fever in the
previous 2 weeks x 100
3 % of health facilities reporting no
disruption of stock of antimalarial
drugs (as specified in the national
drug policy) for more than one
week, during the previous 3
months
Number of health facilities reporting no disruption
of stock of antimalarial drugs (as specified in the
national drug policy) for more than one week,
during the previous 3 months divided by total
number of health facilities visited x 100
Reports
Di., Com.
Di.,Pr.,Co.
1. Supervision
1. District/province
health team supervision Reports
2. Health facility surveys 2. Survey reports
3. RBM evaluation
3. RBM evaluation
reports
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Final Version - August, 2000
1. Surveys require skilled
1. Year
supervisors
2. 2 years (I
year if possible) 2. Periodicity depends on
regularity of supervision
3. 2 years
3. At community level supervision
should follow IEC activities
implemented
1. Year
2. 1 year
3. 2 years
In countries with a short
transmission season
monitoring/evaluation to be
conducted during transmission
season
2
Annex 3: Operational definitions of proposed (Impact, Outcomes and Process) core indicators for monitoring and evaluation of RBM
Indicators
Method of data
collection
Operational Definition
Number of health facilities able to confirm malaria
4 % of health facilities able to confirm
diagnosis
according to the national policy
malaria diagnosis according to the
(microscopy, rapid test etc) divided by total
national policy (microscopy, rapid
number of health facilities supposed to confirm
test etc)
diagnosis of malaria according to the national
policy x 100
5 % of households having at least
one insecticide treated net
6 % of pregnant women who have
taken chemoprophlaxis or
intermittent drug treatment,
according to national drug policy
7 % of malaria epidemics detected
within two weeks of onset and
properly controlled
1. District/province
health team supervision
2. Health facility surveys
3. RBM evaluation
Number of households having at least one treated Community surveys
bednet divided by total number of households
visited x 100
Reports
1. Health facility surveys 1 .Survey reports
2. ANCS
2. Antenatal care
surveys
3. Community surveys
Number of pregnant women who have taken
chemoprophlaxis or intermittent drug treatment,
according to national drug policy divided by total
number of pregnant women interviewed/whose
ANCS has been reveiwed x 100
Review of HIS
Number of epidemics detected in a specific
documents
geographical area(district, country, region) or
situation within 2 weeks during the last 12 months
and for which appropriate control measures have
been initiated within the following week divided by
total number of malaria epidemics notified during
the same period in the same area/situation x 100
1. Number of mothers/caretakers able to
recognise
signs and symptoms of danger of a
recognise signs and symptoms of
febrile disease in a child<5 years (other target
danger of a febrile disease in a
child < 5 years (other target groups) groups) divided by total number of
mothers/caretakers interviewed x 100
2. Number of mothers/caretakers able to
recognise signs and symptoms of danger of a
febrile disease in a child <5 years (other target
groups) divided by total number of
mothers/caretakers who have had training in the
same sample
8 % of mothers/caretakers able to
Source of
information
Community surveys
Level of data
collection
Periodicity of
data collection
Comments
Requires skilled supervisors for
reading of sample slides
Di.,Pr.ICo.
Year
Com., Di.,Pr.,
Co.
2 years (1 year At random methodology needed
if possible)
Com., Di.,Pr.t
Co.
Year
1. TBA to be supervised in
countries where they deliver
intermittent treatment
2. To be integrated with RH
supervision
HIS reports
Di.,Pr.,Co.
1. Periodicity depends on the
occurrence of epidemics
2. Appropriate control measures
means actions based on national
preparedness POA where such
control measures are defined
according to WHO global
guidelines
3. Feasible but dependent on
quality of HIS
Community
surveys reports
Community
In some countries is already part
2 years (One
year if possible) of NMCP/District team supervision
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Final Version - August, 2000
3
i
Annex 3: Operational definitions of proposed (Impact, Outcomes
Method of data
collection
Operational Definition
Indicators
Source of
information
Level of data
collection
Periodicity of
data collection
Number of districts with a plan reflecting national
health policy in a given country divided by total
number of districts x 100
Review of documents
District POA
Di., Co.
Year
9 % of districts with plans of action
Number of districts using health information for
planning in 1 country divided by total number of
Review of documents
District POA
District
Epidemiological
data
National
Regional
Year
10 % of districts using health
information for planning
Review of documents
Regional
NMCP activities
Global
report
Partners meetings
reports
Year
Reports
Regional
Global
Year
Review of documents
Scientific papers
Publications
Global
5 years
Review of documents
Financial reports
Regional
Global
Year
reflecting national health policy
districts
x100
11 % of countries with multisectoral
and inter-agencies partnership
established
__________________ ___
Number of countries with multisectoral and inter
agencies partnership established divided by
number of countries implementing RBM POA x
100
12 % of countries with functional
sentinel sites for surveillance
efficacy of 1st and 2nd line
antimalarial drugs
Number of countries with functional sentinel sites Review of documents
Country mission
for surveillance efficacy of 1st and 2nd line
antimalarial drugs divided by total number of
countries implementing RBM POAx 100
13 Number of antimalarial drugs which
have progressed to the level of
phase III trials
14 % of countries with agreed national Number of countries with agreed national RBM
RBM budget met by donor funding budget met by donor funding divided by total
number of countries with an established RBM
partnership x 100
«
and Process) core indicators for monitoring and evaluation of RBM
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Final Version - August, 2000
Comments
To be monitored by skilled staff
according to WHO protocol
1. Periodicity will depend on the
duration of national POA.
2. Yearly monitoring is
recommended
4
Ann„ 3: OperaHon.l'defini.in, of proposed dmpact, Outcomes and Process) core indicators to. monHorlng and eealuadon ot RBM
Indicators
Method of data
collection
Operational Definition
Source of
information
Level of data
collection
PROCESS
Periodicity of
data collection
Comments
.____ ________ __ __________________ -—
1. NMCP and IMCllDi, Pr., Co"
1. NMCP and IMCI
Number of health personnel involved in patient
%
of
health
personnel
involved
in
1
supervision
supervision reports
care trained in malaria case management and
•Programme
ivy .a..... —
i
a
patient care trained in malaria case
IMCI divided by total number of health personnel x 2. Health facility surveys 2. Survey Report
management and IMCI
100
Year
Only for countries already
implementing IMCI activities+034
Di., Pr„ Co.
Year
1. In some countries TBA will also
be included into antenatal clinic
staff
2. To be integrated with RH
programme
■
1. NMCP/RH
1. NMCP and RH
supervision report
supervision
2. Health facility surveys 2. Survey report
Number of antenatal clinic staff trained in
2 % of antenatal clinic staff trained in
preventive intermittent antimalarial preventive intermittent antimalarial treatment for
pregnant women divided by total number of
treatment for pregnant women
antenatal clinic staff x 100
Number of countries having national guidelines for Review of documents
3 % of countries having national
malaria treatment including the predefined items
guidelines for malaria prevention
and treatment, including training of divided by total number of countries implementing
all the informal health providers and RBM POA x 100
recommendations for home
treatment of febrile
illness/suspected malaria,
recognition of the most frequent
signs of danger for children,
prevention of malaria during
pregnancy and use of insecticide
treated materials
4 % of villages/communities with at
least one Community Health
Worker trained in management of
fever and recognition of severe
febrile illness
5 % of countries having introduced
pyrethroids for public health use
and insecticide-treated materials in
the list of essential drugs and
materials
Number of villages/communities with at least one Community surveys
Community Health Worker trained in management
of fever and recognition of severe febrile illness
divided by total number of villages/communities
Guidelines
Regional
Global
Reports
Com., Di., Pr.
List of essential
drugs and
medicines
Regional
Global
This criterion should be monitored
every year until it is established
that such guidelines exist at
country level
2 years
investigated x 100
Number of countries having introduced
pyrethroids for public health use and insecticidetreated materials in the list of essential drugs and
materials divided by total number of countries
implementing RBM POA x 100
Review of documents
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Final Version - August, 2000
This criterion should be monitored
every year until it is established
that the list of essential list of
drugs and medicines has been
modified according to malaria
control national policy
5
Annex 3: Operational definitions of proposed (Impact, Outcomes and Process) core indicators for monitoring and evaluation of RBM
Indicators
Operational Definition
Method of data
collection
Source of
information
Level of data
collection
1. NMCP activities Di., Pr., Co.
1. NMCP Supervision
Number of service providers (health personnel,
6 % of service providers (health
CHW...) trained in techniques of treatment of nets 2. Health facility surveys report
personnel, CHW...) trained in
2. Health survey
and/or indoor spraying divided by total number of
techniques of treatment of nets
report
and/or indoor spraying according to service providers x 100
the national policy
Periodicity of
data collection
Comments
Year
7 % of countries with epidemic prone
areas/situation having a national
preparedness plan of action for
early detection and control of
epidemics
Number of countries with epidemic prone
areas/situation having a national preparedness
plan of action for early detection and control of
epidemics divided by total number of countries
with epidemic prone areas/situation x 100
Review of documents
National POA
Regional
Global
This criterion should be monitored
every year until it is established
that such a preparedness POA
exists according to the
epidemology of malaria
Number of countries having a policy of universal
coverage for all with a basic package including
relevant malaria control activities divided by total
number of malaria endemic countries x 100
Review of documents
National Policy
document
Regional
8 % of countries having a policy of
universal coverage for all with a
basic package including relevant
malaria control activities
Global
This criterion should be monitored
every year until it is established
that such a policy exists
Review of documents
Research agenda Regional
Research protocols Global
9 % of countries having established
official linkages, including the
elaboration of a public health
Number of countries having established official
interest research agenda, between linkages, including the elaboration of research
research institutions and Ministry of agenda of public health interest, between
Health
rresearch institutions and Ministry
....................
of Health divided
by total number of countries implementing RBM
POA x 100
ACRONYMS:_______________________________________ _____________________________
ANCS =Antenatal cards________________ ___________________________________________
CHW: Community Health Worker
_____________________ __________________________
DSS: Demographic Surveillance Systems _______________ ____ _______ _______________
DHS: Demographic and Health Surveys (USAID-Funded, MACRO Int'l is implementing group).
HIS: Health Information System
Di. = District
Pr^Province /Region
Co. = Pays
1. This criterion should be
monitored every year until it will be
established that such linkages
exist.
2. A follow-up is recommended
IMCI: Integrated Management of Childhood Illness
NMCP: National Malaria Control Programme
POA: Plan of Action
RH: Reproductive Health_________ ___________
TBA: Traditional Birth Attendant
__________
Com.= Community__________________________
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Final Version - August, 2000
6
ANNEX 4
REFERENCES
1. WHO (1981). Health Programme Evaluation. Guiding Principles for its Application in the
Managerial Process for National Health Development.
2. WHO (1981). Development of Indicators for Monitoring Progress towards Health for All by the
Year 2000.
3. WHO (1992). Report of the Ministerial Conference on Malaria, Amsterdam, 26-27 October 1992.
Geneva, World Health Organization (unpublished document CTD/MCM/92.6)
4. WHO (1993a). A Global Strategy for Malaria Control. Geneva, World Health Organization.
5. WHO (1993b). Implementation of the Global Malaria Control Strategy. Report of a WHO study
group on the implementation the global plan of action for malaria control 1993-2000. Geneva,
World Health Organization (WHO technical Report Series N° 839).
6. WHO (1993d). WHO Malaria Unit, Global malaria control. Bulletin of World
Health Organization (1993) 71 (3/4) 281-284
7. WHO (1994). Bryce J. Roungou, J.B. Nguyen-Dinh, P. Naimoli, J.F. Breman, J.G. Evaluation
of national malaria control programmes in Africa. Bulletin of the World Health Organization
(1994) 72 (3) 371-381[En. Fr. 16ref. 6 tab] Division of Diarrhoeal and Acute Respiratory Disease
Control. World Health Organization, 1211 Geneva 27, Switzerland.
8. WHO (1997). WHO Recommended Surveillance Standards, World Health Organization,
WHO/EMC/DIS/97.1, Distribution: Limited, Original: English.
9. WHO (1999). Indicators for IMCI at First-Level Facilities and Households, Department of Child
and Adolescent Health and Development (CAH), Health Systems and Community Health (CHS),
World Health Organization, WHO/CHS/CAH/98.1K, Rev.l 1999, Original: English, Distribution:
General.
10. WHO (2000). WHO Technical Report Series 892, WHO Expert Committee of Malaria, Twentieth
Report.
11. WHO (2000). Indicators To Measure The Impact Of Malaria Control. Discussion document
prepared for Roll Back Malaria, January 2000. Catherine Watt and Chris Dye. Communicable
Disease Prevention, Control and Eradication WHO Geneva.
12. WHO/AFRO (1992a). Guidelines for the diagnosis and treatment ofmalaria in Africa. Report of
an informal consultation of experts on malaria in the African Region. Brazzaville, World Health
Organization Regional Office for Africa Region. (AFRO Technical Papers No. 22, Rev. 1)
13. WHO/AFRO (1992c). Strategies for malaria control in the African Region and steps for their
implementation. Brazzaville, World Health Organization Regional Office for Africa.
14. WHO/AFRO (1994). Strategies of malaria control in the African Region: Evaluation and
information support. Report of a meeting of the regional working group on malaria control in the
African Region. Brazzaville, 15-18 March 1993. Brazzaville, WHO Regional Office for Africa,
1994 (unpublished document AFRO/CTD/MAL/94.2).
15. WHO/AFRO (1994). Information Systems For The evaluation ofMalaria Control Programmes.
(A practical guide).
16. WHO/AFRO (1999). Joint Evaluation of DFID and USAID support to the World Health
Organization Regional Office for Africa Malaria Unit (AFRO/MAL), 20 September to 8 October
1999, WHO/AFRO Evaluation Background documents (Volume 1)
17. ACSI-CCCD (1993). Africa Child Survival Initiative Combating Childhood Communicable
Diseases. Malaria Control in Africa: Guidelines For The Evaluation Of National Programs,
Bujumbura. Burundi, September 1993. United States Agency For International Development, U.S.
Department of Health and Human Services.
18. The Abuja Declaration on Roll Back Malaria in Africa. By the African Heads of State and
Government Abuja, Nigeria - African Summit on Roll Back Malaria - 25 April 2000.
19. Executive Summary for Economics ofMalaria Centerfor Development, Harvard University and
the London School ofHygiene and Tropical Medicine (Presented by Pr Sachs during the African
Summit on Roll Back Malaria - 25 April 2000)
RBM Framework for Monitoring Progress and Evaluating Outcomes and Impact
Final Draft - August, 2000
i
Position: 2824 (3 views)