10634.pdf
Media
- extracted text
-
IVMdHHH W W lH W W lW H
U1
1 (4II
Primary Health Care Management Advancement Programme
MONITORING
AND EVALUATING
PROGRAMMES
MODULE 5
FACILITATOR S GUIDE
AGA KHAN FOUNDATION
THE AGA KHAN UNIVERSITY
Primary Health Care Management Advancement Programme
MONITORING
AND EVALUATING
PROGRAMMES
Martine Hilton
University Research Corporation
MODULE 5
FACILITATOR S GUIDE
Aga Khan Health Services
University Research Corporation
Center for Human Services
33
X3
33
33
ra
58
58
3
33
ca
“3
&
S8
$
Sei
lea
Dedicated to
Dr. Duane L. Smith (1939-1992),
Dr. William B. Steeler (1948-1992)
and all other health leaders, managers and workers
who follow their example in the effort to bring quality health
care to all in need.
The Primary Health Care Management Advancement
Programme has been funded by the Aga Khan Foundation
Canada, the Commission of the European Communities,
the Aga Khan Foundation U.S.A., the Aga Khan
Foundation’s head office in Geneva, the Rockefeller Foun
dation, the Canadian International Development Agency,
Alberta Aid, and the United States Agency for Interna
tional Development under two matching grants to AKF
USA. The first of these grants was, "Strengthening the
Management, Monitoring and Evaluation of PHC Pro
grams in Selected Countries of Asia and Africa" (cooper
ative agreement no. OTR-0158-A-00-8161-00, 19881991); and the second was "Strengthening the Effective
ness, Management and Sustainability of PHC/Mother and
Child Survival Programs in Asia and Africa" (cooperative
agreement no. PCD-0158-A-00-1102-00, 1991-1994).
The development of Modules 6 and 7 was partially funded
through in-kind contributions from the Primary Health
Care Operations Research project (PRICOR) of the Center
for Human Services under its cooperative agreement with
USAID (DSPE-6920-A-00-1048-00).
This support is gratefully acknowledged. The views
and opinions expressed in the PHC MAP materials are
those of the authors and do not necessarily reflect those
of the donors.
All PHC MAP material (written and computer files) is
in the public domain and may be freely copied and
distributed to others.
g
&
L<
g
&
g
g
g
g
g
@r
g
n
a
g
&
l£
•5
Contents
3
3
V
3
IS
FOREWORD .............................................................................................................
1
SESSION 1: INTRODUCTION AND OVERVIEW ..............................
7
Transparencies
5-1: Workshop objectives........................................................................... 14
5-2: Purposes of module 5........................................................................ 15
5-3: Monitoring ........................................................................................... 16
5-4: A systems diagram of management and
service relationships........................................................................... 17
5-5; An indicator is
......................................................................... 18
5-6; Suggested indicators........................................................................... 19
5-7: Generic indicator categories for PHC services........................... 20
5-8: Generic indicator categories for PHC management................. 21
5-9; Types of monitoring ........................................................................... 22
5-10: Keep It Straightforward and Simple.............................................. 23
5-11: The KISS approach ........................................................................... 24
5-12: Steps in activity monitoring............................................................ 25
3
-Si
SESSION 2: SPECIFY THE MONITORING OBJECTIVES
(STEPl)
...............................................................................27
-'Si
”
Transparency
5-13: Session 2: Objective ........................................................................... 30
SESSION 3: DECIDE ON THE SCOPE AND SELECT
INDICATORS (STEPS 2 AND 3).................................................................... 33
I
Transparencies
5-14: Session 3: Objectives........................................................................... 37
Module 5: FG; contents
5-15: Planning the scope of monitoring.................................................. 38
5-16: Common indicator modifications .................................................. 40
5-17: Formulation of indicators ................................................................. 41
5-18; Frequency of data collection.............................................................43
SESSION 4: CHOOSE DATA SOURCES AND COLLECTION
PROCEDURES (STEP 4) ................................................................................... 45
Transparencies
5-19: Session 4: Objectives ......................................................................... 48
5-20: Data sources.......................................................................................... 49
5-21: Data collection techniques ............................................................... 51
SESSION 5: COLLECT, TABULATE AND ANYALSE THE DATA
(STEPS 5 AND 6) ................................................................................................... 53
Transparency
5-22: Session 5: Objectives........................................................................... 56
SESSION 6: PRESENT FINDINGS AND TAKE ACTION
(STEPS 7, 8 AND 9) .............................................................................................. 59
Transparencies
5-23: Session 6: Objectives........................................................................... 63
5-24: Report findings ....................................................................................64
5-25; Take action ............................................................................................ 65
5-26: Monitoring review guidelines ...........................................................67
Module 5: FG; contents
Foreword
What is the purpose of the Facilitator’s guide?
The Facilitator’s guide contains a suggested workshop design for use by
individuals who will help PHC managers and staff learn how to use the
Primary Health Care Management Advancement Programme (PHC MAP)
series. There is a Facilitator’s guide for each module in the PHC MAP series.
What information does it contain?
Each Facilitator’s guide contains instructions for conducting a workshop
on one of the modules in the PHC MAP series using the information
contained in the User’s guide.
The instructions are arranged in one to two hour sessions. The number
of sessions depends on the amount of information and/or the number of
steps contained in the User’s guide. The first page of each session lists
session objectives, major topic headings, time required, and materials and
equipment. The following pages describe the instructional activities in
outline form.
The outline is divided into two columns. The right-hand column
indicates what the facilitator says or does to conduct the session. The
left-hand column lists the handouts, overhead transparencies, or other
materials needed to support the activity. Copies of these materials, suitable
for duplication, appear at the end of the session in which they are first used.
Some transparencies are used in more than one session.
How are the workshops organised?
The first session of each workshop contains activities designed to
introduce the PHC MAP series and explain its importance to PFIC
programmes, present an overview of the workshop, explain the purpose of
the module, review specific terms or concepts used in the module, and
d
u
&
iM
j
& &
'
'
ft it iu ifej to to toi to ik) to ik)to to iM ikl
i
Module 5: FG; foreword
2
acquaint the participants with the organisation and content of the User’s
guide.
The final session of each workshop contains activities designed to review
key points of the User’s guide, if appropriate, help participants prepare an
action plan, and brings the workshop to a formal conclusion. The sessions
in between present the steps required to accomplish the purpose of the
module and provide an opportunity for participants to apply those steps
to their own programmes.
Each workshop is designed to be given on consecutive days, uninter
rupted except for breaks, meals, and rest, if the workshop is longer than
one day. It is recognised, however, that situations vary and the PHC MAP
modules will be used singly or in combination by individuals and organisations in the field and on university campuses. Therefore, it is
expected that the workshops described in the Facilitator’s guides will be
adapted to fit a range of circumstances. Indeed, those who facilitate PHC
MAP workshops are encouraged to make adaptations to meet the needs
of participants and/or specific PHC programmes.
gT
gg~~
{S_
@_
(S
£•
Who can be a facilitator?
Ideally, the person who serves as facilitator has training and experience
in the fields of public health, management, planning, and evaluation, as well
as sampling and survey design. It is recommended that this person also be
skilful in working with small groups.
The facilitator should be someone from outside the PHC programme,
perhaps from another agency or a university. As a person with no direct
interest in the programme, the facilitator can be neutral in disagreements
that may arise and help the group members resolve an issue objectively.
Having an outside person as facilitator also allows the programme director,
who usually has a leadership role, to contribute his or her knowledge and
experience as a full participant in the work of the group.
It is also recommended that a facilitator have a full set of PHC MAP
User’s guides and Facilitator’s guides. It is sometimes necessary, in the
course of completing one module, to refer to information in another module.
J6
[S.
{fS
St'
<S*
Who should participate in the workshop?
The primary audience for the PHC MAP series is the team that manages
PHC programmes or other population-based health programmes in either
the public or private sector. An example of a PHC management team in
the public sector is the core staff of a district ranging from 100,000 to
300,000 in size of population served.
®
SviS
g
Module 5: FG; foreword
13'
S
3
An example of a PHC management team in the private sector is the
core staff of an NGO that provides PHC services to a specific population
in a given geographic area. The average size of the service population may
range from 40,000 to 60,000, but may be as small as 10,000 or more than
100,000. In either case, the management team should have:
• the knowledge and skills needed to complete the steps in the module,
• the time and resources required to collect and analyse the required data,
the authority to plan and implement improvements in management
systems and procedures based on this analysis.
If feasible, teams from several districts or from several PHC programmes
within a given NGO, may work simultaneously with one facilitator to
complete a module. It is recommended that one facilitator work with no
more than four teams or 20 people at one time.
What does the facilitator do?
Understands the needs of participants. If the facilitator is working
with one PHC programme, the most efficient way to gather information
about participants and the programme in which they work is to meet with
the PHC director to determine:
• The nature of the PHC programme (services provided, size and descrip
tion of area served, number and type of facilities, number and qualifica
tions of staff, community involvement, computer capability, strengths,
major problems, etc.);
• if information needs have been clearly identified;
• previous staff training in management information systems or related
topics;
• resources available for training (time, space, equipment);
• possibility for subsequent technical assistance to workshop participants.
Understands the content of Module 1. Read the User’s guide and
Facilitator’s guide for Module 1 and consider options for workshop delivery.
» The purpose of Module 1 is to help participants select priority information
needs and to review some basic concepts underlying management
information systems. Depending on participant background, the amount
of time available, and the need to establish priorities among management
information needs, Module 1 can be presented in its entirety, limited to
Level 1: Quick start, limited to the PHC MAP systems framework, or in
Module 5: FG; foreword
4
summary form as contained in the first session of the Facilitator’s guides
for Modules 2 through 9.
• If information needs have not been clearly identified and participants
have no background in systems, then Module 1 should be presented in
full. The selection of the next module will depend upon priorities set by
the participants as they complete the steps in Module 1.
• If information needs have been clearly identified and participants have
no background in systems, then that portion of Module 1 can be added
to the first session of the selected module.
• If information needs have been clearly identified and participants have a
background in systems, then the overview of PHC MAP in the selected
module will be sufficient.
Understands the content of the selected module. Read the User’s
guide and Facilitator’s guide for the selected module and consider options
for workshop delivery.
• Depending upon the length of the module and the availability of staff for
training, the workshop sessions may be presented as described in the
Facilitator’s guide, or workshop sessions may be delivered at intervals (for
example, every morning for a certain number of days, one session per
week, or one day per week for three or four weeks).
• If follow-on technical assistance is available, the content of the module
could be presented in the workshop and the application of the procedure
to the PHC programme could be conducted on the job.
• Participants could plan steps in the workshop, complete them on the job,
and bring the results to the next workshop session. This approach is
particularly applicable to Module 2 which requires development of a
questionnaire, selection of a survey sample, training of data collectors,
and data collection. These steps cannot actually be carried out within the
time limits of the workshop as contained in the Module 2 Facilitator’s
guide.
• Other options are possible depending upon the circumstances of each
situation.
Determines what programme information, if any, is needed for
completion of the module. For example, census data and lists of households
are required to complete some of the steps in Module 3.
Module 5: FG; foreword
5
Determines the most appropriate people to attend the workshop. For
example, Module 7 is best completed by those responsible for planning,
training, supervision, logistics, and other management services.
Discusses the delivery options and participants with the PHC manager
and make decisions regarding the most appropriate option and participant
selection. Also determines if the needed information is available and if it
is not, the alternatives for obtaining the information.
Notifies participants of the date, time, and purpose of the
workshop and confirms attendance. Initial notice of the meeting may come
from the PHC manager, with follow-up by the facilitator.
Prepares the agenda, using as a model the sample in Session 1 of the
Facilitator’s guide for the selected module.
Inspects the room where the workshop will be held and answers these
questions:
• Is the room large enough to seat all the participants?
• Can chairs and tables be arranged in a variety of ways; all participants
around one table; participants in groups of three or four at smaller tables?
• Is the lighting adequate?
• Can temperature and humidity be controlled?
• Is there an electrical outlet for an overhead projector?
• Is a microphone necessary for everyone to hear and to be heard?
• Are toilets conveniently located?
• Is the site convenient to parking, restaurants and public transportation?
• Are supporting facilities available and convenient: duplication, word
processing, computers?
Prepares notes to conduct the workshop using the session outlines in
the Facilitator’s guide for the selected module, and makes the necessary
adaptations. Refer to the appropriate User’s guide for content.
Duplicates the worksheets and other handouts as indicated in
the Facilitator’s guide and obtains the necessary equipment.
Conducts the workshop as planned.
Provides follow-on assistance, if appropriate.
Module 5: FG; foreword
7
.-3
:3
3
3
Session 1: Introduction and overview
h
Objectives:
3
3
Participants will be able to:
• Explain the purposes of Module 5.
• Distinguish between routine and short-term
monitoring.
3
• Name at least two guidelines in the KISS ap
proach to monitoring design.
^3
• Explain the basic steps to planning monitoring
procedures.
3
3
Session outline:
I. Introduction (10 minutes)
II. Overview of workshop (5 minutes)
III. Introduction to Module 5 (40 minutes)
IV. Module 5 User’s guide (20 minutes)
Materials:
Module 5 User’s guide
Handout 5-1:
Agenda
Transparency 5-1: Workshop objective
Transparency 5-2: Purpose of Module 5
Transparency 5-3: Monitoring
Transparency 5-4: A systems diagram of ma
nagement and service re
lationships
Transparency 5-5: An indicator is...
Transparency 5-6: Suggested indicators
Transparency 5-7: Generic indicator catego
ries for PHC services
3
3
3
3
3
.3
:3
3
-jjj|
Module 5: FG; session 1
8
Transparency 5-8: Generic indicator catego
ries for PHC management
functions
Transparency 5-9: Types of monitoring
Transparency 5-10: KISS
Transparency 5-11: The KISS approach
Transparency 5-12: Steps in activity monitoring
Equipment:
Flip chart, stand, markers, masking tape, over
head projector and screen
Module 5: FG; session 1
9
Session 1: Introduction and overview
I. Introduction (10 minutes)
Note: Omit Section I if you have presented Mod
ule 1, or some other module, immediately prior to
the presentation of Module 5.
A. OPENING REMARKS
• Introduce self and other staff, as necessary.
a
-J
• Explain your role in the workshop.
• If you do not know the participants, ask each
one to state his or her name, position title, and
job location.
'-a
Name tags or place cards
wish to have them wear name tags or place cards
with their names at places where they are sitting.
Flip chart
• Ask each participant to complete this sen
tence: "As a result of attending this workshop,
1 expect..." and write their responses on the
flip chart.
23
• When all responses have been recorded, say
that you will return to this list after giving an
overview of the workshop.
&
B. OVERVIEW OF PHC MAP
“3
3>
s
Note: If the group is larger than 6 or 8, you may
See Module 1 User’s
guide for this information
• Explain why information is important for
PHC programmes.
• Summarise PHC MAP programme,
including:
Purpose of PHC MAP
Title and purpose of each module and other
materials.
-J9
.*9
• Make the following points about PHC MAP:
Has been field tested and revised to ensure
usefulness to PHC programme staff.
"Si
-“T3
Module 5: FG; session 1
10
Session 1: Introduction and overview
Modules can be used in any sequence.
Checklists and other materials can, and
should, be adapted to fit a particular situation.
The procedures outlined in the modules can
be adopted as routine monitoring activities in
a PHC programme.
• Explain why MAP is being introduced in this
PHC programme.
Note: The remarks here should be tailored to the
specific programme and should provide the an
swers to these questions:
Why are we doing this workshop?
What do we expect to achieve?
It may be appropriate for the PHC manager to
make these remarks.
II. Overview of workshop (5 minutes)
Transparency 5-1:
Workshop objective
Handout 5-1: Agenda
• Review the workshop objective.
• Distribute and review the workshop agenda,
describing the activities that will take place to
accomplish the workshop objective.
• Review the expectations contributed by par
ticipants at the beginning of the session.
Indicate which ones will be met and which
ones will not and why.
Suggest, if possible, alternatives for meet
ing the expectations that will not be met in
this workshop.
• Ask for questions or comments.
III. Introduction to Module 5 (40 minutes)
Transparency 5-2:
Purpose of Module 5
• Explain that the purposes of Module 5 are to
help PHC programme managers to;
Module 5: FG; session 1
11
■Ji
Session 1: Introduction and overview
Identify important PHC services and
management functions they wish to monitor.
Select key indicators for these services and
functions.
Design and implement a simple monitoring
system using these indicators.
■3
Transparency 5-3:
Monitoring
• Review the definition and purpose of
Transparency 5-4:
A systems diagram of
management and service
relationship
• Review the systems framework for PHC
Transparency 5-5:
An indicator is.."
• Review the definition of indicator.
.'3
■5
Transparency 5-6:
Suggested indicators
■5
'■a
Transparency 5-7:
Generic indicator categories
for PHC services
Transparency 5-8:
Generic indicator
categories for PHC
management functions
Transparency 5-9:
Types of monitoring
monitoring.
programmes presented in Module 1, if
necessary.
Explain the three categories (inputs,
processes and outcomes) and give an example
of each.
Distinguish among outputs, effects, and
inputs.
Discuss the relationship between the
management system components and the
service delivery system components.
• Explain that Module 5 suggests indicators for
the inputs, outputs, and effects for PHC serv
ices and management functions.
• Present and explain the suggested indicator
categories for PHC services.
• Present and explain the suggested indicator
categories for management functions.
• Ask for questions and comments.
• Explain the two types of monitoring and
give an example of each type.
Routine
Short-term
&
*
• Explain the relationship between routine and
short-term monitoring.
ta
Module 5: FG; session 1
12
St
&
Session 1: Introduction and overview
Transparency 5-10:
KISS
Transparency 5-11
The KISS approach
• Review guidelines for designing a monitoring
system:
Select only key indicators.
Keep data collection to a minimum.
Provide timely feedback.
Use a clear and convenient reporting format.
(gi.
• Discuss the limitations of activity monitoring
and give examples:
(ft
Important information may be difficult to
collect.
Important information may not be feasible
to collect.
• Discuss other options for assessing effects
(ft
and impacts.
Transparency 5-12:
Steps in activity
• Review the steps in activity monitoring.
• Ask for questions and comments.
IV. Module 5 User’s guide (20 minutes)
Module 5 User’s guide
iSJ-
• Distribute a User’s guide to each participant.
S'-
• Highlight key sections of the User’s guide:
is
The overview of PHC MAP with the descrip
tion of modules and other materials, which
they may wish to refer to for review of infor
mation needs met by other modules.
Quick start provides a procedure for moni
toring important performance indicators.
An explanation of generic categories of in
dicators for PHC services and management
functions.
Step-by-step directions for designing and
implementing a monitoring system.
Appendices containing suggested indicators
and blank worksheets.
• Ask for questions or comments.
Module 5: FG; session 1
B-
S
£
S
SAMPLE AGENDA
MODULE 5*
Session 1
(1 hr, 15 mins)
Introduction and overview
Session 2
(1 hr)
Specify the monitoring
objectives (Step 1)
Session 3
(1 hr, 20 mins)
Decide on the scope and
select indicators
(Steps 2 and 3)
Session 4
(1 hr)
Develop data collection
procedures (Step 4)
Session 5
(1 hr, 15 mins)
Collect, tabulate and
analyse the data
(Steps 5 and 6)
Session 6
(1 hr, 20 mins)
Present findings and take
action
(Step 7, 8 and 9)
*For an actual presentation, substitute the appropriate clock hours in the first column.
Module 5: FG; session 1; handout 5-1
q
$
® in im im
Plan monitoring procedures for PHC
and management services using the
steps and worksheets contained in
Module 5
® ran ®
WORKSHOP OBJECTIVE
i® i»
.
b $
14
§
Module 5: FG; session 1; transparency 5-1
s-
j
lid
id lid'lai lid
lid
1*1
'id lid
15
PURPOSES OF MODULE 5
To help PHC programme managers to:
1. Identify important PHC services and
management functions they wish to
monitor.
2. Select key indicators for these services
and functions.
3. Design and implement a simple
monitoring system using these
indicators.
Module 5: FG; session 1; transparency 5-2
MONITORING
Definition: The periodic collection and
analysis of selected indicators
Purpose:
Enables managers to determine
whether key activities are
carried out as planned and have
expected effects on the target
population
Value:
Identify problems; take
corrective action; measure
achievement of targets; assess
trends in health status
Module 5: FG; session 1; transparency 5-3
U) Ul ii 14 U U) 14 14 i4 4 U li 4 Ul 4 U Mi Ml
U U
Module 5: FG; session 1; transparency 5-4
A SYSTEMS DIAGRAM OF MANAGEMENT
AND HEALTH SERVICE RELATIONSHIPS
Management services
PHC services
Ml U
18
’5
AN INDICATOR IS...
C?
cr
An indirect measure of an event or
condition.
Example: indicator:
weightfor-age
condition:
nutritional
status
8S
S
8L
Module 5: FG; session 1; transparency 5-5
K
ii Li bi Li Li bi Li bi Li iii bi bi ua Ul ti ti Li ui Li
Module 5: FG; session 1; transparency 5-6
SUGGESTED INDICATORS
'*<
Jj
Li :i
20
GENERIC INDICATOR
CATEGORIES FOR PHC
SERVICES
• EFFECTS
• Coverage (including behaviour)
o Knowledge and skills (of the target
population)
® OUTPUTS
®
®
©
©
Utilisation of services
Quality of care
Contacts, visits
Access to services
e INPUTS
o Availability of personnel, supplies,
equipment, funds
Module 5: FG; session 1; transparency 5-7
GENERIC INDICATOR
CATEGORIES FOR PHC
MANAGEMENT
• EFFECTS
•
•
•
•
•
PHC services available, accessible
Staff performance
Worker knowledge, skills, behaviour
Community satisfaction with services
Quality of PHC services .
• OUTPUTS
• Services or activities completed
® Frequency of management activities
• Quality of management activities
® INPUTS
o
Availability of trained personnel, supplies,
information, guidelines/protocols
Module 5: FG; session 1; transparency 5-8
Eft
22
• core set of indicators
• tracks programme implementation
®\
SHORT-TERM:
ffii
• regular, on-going
m
ROUTINE:
® ® m
TYPES OF MONITORING
• limited period of time
if
Module 5: FG; session 1; transparency 5-9
m
n
m
• collects information to help solve a
problem
m fli
• tracks new activities or processes
23
3 • K EEP
-3
=3
jl
'
lU til id id Jl
3
•
I
T
• S TRAIGHTFORWARD AND
• S IMPLE
3
Module 5: FG; session 1; transparency 5-10
24
THE KISS APPROACH
• Key indicators
• Minimum data collection
• Timely feedback
• Usable report format
tn
Module 5: FG; session 1; transparency 5-11
STEPS IN ACTIVITY
MONITORING
PLANNING:
1. Specify objectives
2. Decide on scope
3. Select the indicators and standards
4. Choose information sources; develop
data collection procedures
IMPLEMENTATION:
5. Collect the data
6. Tabulate and analyse the data
7. Present/report the findings
8. Take appropriate action
ASSESSMENT
9. Decide whether to continue monitoring
Module 5: FG; session 1; transparency 5-12
3
27
3
3
3
3
3
3
Session 2: Specify the monitoring
objectives (Step 1)
Objectives:
Participants will be able to follow the directions
in the Module 5 User’s guide to specify the moni
toring objectives for their PHC programme.
Session outline:
I. Introduction (10 minutes)
II. Specify the objectives (50 minutes)
Materials:
Module 5 User’s guide
Transparency 5-13: Session 2: Objective
Worksheet for specifying
Handout 5-2:
the monitoring objectives
Equipment:
Flip chart, stand, markers, masking tape, over
head projector and screen
3
3
3
3
Module 5: FG; session 2
28
Session 2: Specify the monitoring objectives (Step 1)
I. Introduction (10 minutes)
Transparency 5-13:
Session 2 Objectives
• Review the session objective.
• Explain that the activity monitoring proce
dures can be applied to:
short-term monitoring of selected indicators
routine monitoring of all indicators
improve upon an existing MIS
II. Specify the objectives (50 minutes)
• Explain that specifying monitoring objectives
involves determining:
What is going to be monitored
For what purpose
For whom
• If Module 1 has been completed, instruct
participants to review the PHC and manage
ment services they prioritised.
• If Module 1 has not been completed, instruct
participants to review the list of the most
common functions and services.
Choose which services to monitor from the
list in Appendix A.
Choose which management services to
monitor from the list in Appendix B.
Rank services to be monitored on a scale of
1 to 10 (highest).
Determine whether to monitor input, out
put or effect indicators for these services.
Handout 5-2: Worksheet
for specifying the
monitoring objectives
• Distribute the worksheet for specifying the
monitoring objectives and explain what in
formation goes in each column and how that
information is determined.
Module 5: FG; session 2
29
Session 2: Specify the monitoring objectives (Step 1)
Note: This step assumes understanding of the
User’s guide
terms numerator and denominator. You may
need to review these terms, depending on the
background of the participants.
• Direct participants to complete the work
sheet for their own PHC programme using the
model on page 22 in the User’s guide as
necessary.
• When participants have completed the work
sheet ask them to:
Explain the rationale for their choice of
activities to monitor.
Specify why information is needed for each
purpose indicated.
Explain the extent to which the users are
involved in the selection of indicators and the
use each user will make of the information.
• Ask for questions and comments.
Module 5: FG; session 2
30
SESSION 2: OBJECTIVE
Specify monitoring objectives for your
PHC programme
Module 5: FG; session 2; transparency 5-13
WORKSHEET FOR SPECIFYING THE
MONITORING OBJECTIVES
What to monitor
PHC services
Management
services
— ’ ’’'/NX.
A
o v <<? \ T-A
7;. V'o„,
) th
Purpose & routine(s)
or short-term
Internal users
Providers
Managers
External users
Donors
Others
Session 3: Decide on the scope and select
indicators (Steps 2 and 3)
Objectives:
Participants will be able to follow the directions
in the Module 5 User’s guide to:
• Decide on the scope of the monitoring.
• Select, the indicators and performance stand
ards.
Session outline:
I. Introduction (5 minutes)
II. Decide on the scope of the monitoring (30
minutes)
III. Select the indicators and performance
standards (45 minutes)
Materials:
Module 5 User’s guide
Transparency 5-14: Session 3: Objectives
Transparency 5-15: Planning the scope of
monitoring
Transparency 5-16: Common indicator modi
fications
Transparency 5-17; Formulation of indicators
Transparency 5-18: Frequency of data collec
tion
Handout 5- 3:
Worksheet for specifying
the scope of monitoring
Handout 5-4:
Worksheet for selecting
indicators and standards
Module 5: FG; session 3
Equipment:
Flip chart, stand, markers, masking tape, over
head projector and screen
f| If! ffl If] If] If] If] If] If] f) If] f] If] If] If)
34
«8>.
S'
Module 5: FG; session 3
gj-.
7g
3
35
Session 3: Decide on the scope and select indicators
(Steps 2 and 3)
I. Introduction (5 minutes)
Transparency 5-14:
Session 3 objectives
• Review objectives for Session 3.
II. Decide on the scope of the monitoring (30 minutes)
Transparency 5-15:
Planning the scope of
monitoring
addressed in planning the scope of monitorjng, anj
factors jnvolved in each:
Geographic area
Facilities or sub-projects
Personnel involved
Duration
short-term vs. routine
deadlines
w
'-3
• Discuss the four areas that need to be
Handout 5-3: Worksheet
for specifying the scope
of monitoring
• Distribute the worksheet and direct partic-
ipants to complete Handout 5-3 using the
model on page 23 in the User’s guide as neces
sary.
• Ask for questions and comments.
9
III. Select the indicators and performance standards (45 minutes)
User’s guide
• Refer participants to Appendices A and B,
and explain that input, output, and effect
indicators are suggested for each PHC and
management service.
S3
• Direct participants to review the indicators
listed and the information contained in the
footnotes.
^8
• Explain that:
Participants will probably need to modify
the indicators to fit their own situations.
X3
Module 5: FG; session 3
36
Indicators can be expressed in different
ways.
Handout 5-4: Worksheet
for selecting indicators
and standards
• Distribute Handout 5-4.
Direct participants to select/modify indica
i*
Transparency 5-17:
Formulation of indicators
lw
The most common modifications are:
the definition of the target group
terminology
® Ui Li
Transparency 5-16:
Common indicator
modifications
tai
Session 3: Decide on the scope and select indicators
(Steps 2 and 3)
tors for their PHC programme and formulate
each one.
Direct participants to present the indica
tors they have formulated and to explain any
modifications or additions.
with monitoring objectives established in Session
2 and help participants make adjustments as
needed.
It igj
Note: Check selected indicators for consistency
• Explain that a performance standard, some
• Discuss some of the factors considered in
si
Transparency 5-18:
Frequency of data
collection
|| kb h
times referred to as a target, is set for each
indicator to establish a point at which it can
be agreed that the target has been achieved.
Demonstrate how those standards are
used to calculate the effectiveness of perform
ance.
Direct participants to set standards for the
indicators they have selected or modified and
record them on the worksheet.
establishing frequency of data collection.
Direct participants to establish frequencies
for their selected indicators and record them
on the worksheet.
• Ask for question or comments on Steps 2 and 3.
K'
Module 5: FG; session 3
SI
SESSION 3: OBJECTIVES
• Decide on the scope of monitoring
• Select the indicators and performance
standards
k
!
ktk' k' k kJ kikJ ki kl kl kJ k) kJ ki U kj U kJ k) kl k] k)
37
Module 5: FG; session 3; transparency 5-14
[p
]
PLANNING THE SCOPE
OF MONITORING
m
38
• Geographic area
• Facilities or sub-projects
rs if
• Personnel involved
• Duration
• Short-term vs. routine
m «
i
rw
• Deadlines
£
8L
£
£
Module 5: FG; session 3; transparency 5-15
39
WORKSHEET FOR SPECIFYING THE
SCOPE OF MONITORING
A. What geographic area will be covered?
B. Which facilities or sub-projects will be monitored?
C. Which personnel (managers, providers, and
volunteers) will be selected?
D. How long will the monitoring continue?
Module 5: FG; session 3; handout 5-3
40
COMMON INDICATOR
MODIFICATIONS
• Definition of target group(s)
• Terminology
• Phrasing
• Addition or deletion of indicators
Module 5: FG; session 3; transparency 5-16
41
FORMULATION OF
INDICATORS
Count:
Simplest type of measure
Rate:
Measures frequency of
events
Ratio:
A fraction or a decimal
Proportion:
Measures relationship
between a part and the
whole
Percentage:
A proportion multiplied
by 100
Module 5: FG; session 3; transparency 5-17
WORKSHEET FOR SELECTING INDICATORS
AND STANDARDS
Indicators
Standards
Formulations
Frequency
PHC services
Module 5: FG; session 3; handout 5-4
Management
f" n IT fp IT
-
f-l
-
S-l
IT IT IT IT 11 R H IP
43
FREQUENCY OF DATA
COLLECTION
• Consider how often users want to
examine results
• Establish the minimum frequency
necessary
o Continuous data collection - exact
number of times an event occurs
• Periodic data collection - determine if a
procedure is being carried out as
expected
Module 5: FG; session 3; transparency 5-18
45
Session 4: Choose data sources and
collection procedures (Step 4)
Objectives:
Participants will follow the directions in the
Module 5 User’s guide to:
• Choose information sources
• Select data collection techniques
Session outline:
I. Introduction (10 minutes)
II. Identify existing data sources (10 minutes)
III. Develop new data collection instruments (40
minutes)
Materials:
Module 5 User’s guide
Transparency 5-19: Session 4 objectives
Transparency 5-20: Data sources
Transparency 5-21: Data collection techni
ques
Handout 5-5:
Worksheet to select data
sources and techniques
Equipment:
Flip chart, stand, markers, masking tape,, over
head projector and screen
Module 5: FG; session 4
46
Session 4: Choose data sources and collection procedures
(Step 4)
I.
Introduction (10 minutes)
• Summarise the outcomes of Steps 1, 2, and 3.
Transparency 5-19:
Session 4: Objectives
II.
• Discuss the objectives for Session 4.
Choose information sources (10 minutes)
Transparency 5-20:
Data sources
• Provide examples of possible information
Handout 5-5: Worksheet
to select data sources
and techniques.
• Distribute Handout 5-5 and direct partici
III.
sources.
pants to list the indicators they have selected
in Step 3 and enter appropriate source of data
for each.
Develop data collection techniques (40 minutes)
Transparency 5-21
Data collection
techniques
• Define and give examples of techniques for
gathering monitoring data.
• Explain that instruments to collect data by
methods other than record review can be
found in other MAP modules.
Module 2 contains rapid surveys to identify
health needs.
Module 5 contains checklists for assessing
quality of PHC services delivered to meet
those needs.
Module 6 contains checklists for assessing
the quality of management functions that sup
port service delivery.
• Explain that if existing records do not supply
necessary data for monitoring, forms can often
be modified so that staff can record the infor
mation needed.
IB
Module 5: FG; session 4
Session 4: Choose data sources and collection procedures
(Step 4)
• Direct participants to complete Handout 5-6
by writing in the techniques appropriate for
each indicator and have them highlight
those techniques that will require either mod
ification of existing forms or development of
new instruments.
• Ask for questions and comments.
Transparency 5-12:
Steps in activities
monitoring
Module 5: FG; session 4
• Remind participants that this is the final step
in the planning phase. The next phase to be
discussed is implementation.
48
SESSION 4: OBJECTIVES:
• Choose information sources
• Select data collection techniques
Module 5: FG; session 4; transparency 5-19
49
DATA SOURCES
• Existing sources
• Routine records
• Logistics records
• Treatment registers
• Patient records
• Activity reports
• Population-based records
• Modification of current forms
• Develop new instruments
Module 5: FG; session 4; transparency 5-20
WORKSHEET TO SELECT DATA SOURCES
AND TECHNIQUES
Data Source:
Records
Provider- client
interface
Client
Health worker
DC Technique:
Review
Observation
Survey/
interview
Interview
PHC service indicator
Module 5: FG; session 4; handout 5-5
Management service
indicators
S) IB B IB fB /p
p ffl ® ffl
®
® ® ®
R
>
51
3 DATA COLLECTION
* TECHNIQUES
-3
it
53
• Record review
© Observation
© Rapid surveys
u/
© Exit interviews
@ Health worker interviews
DHC- fuc
r
'
10534
Module 5; FG; session 4; transparency 5-21
•3
■31 Session 5: Collect, tabulate and analyse
the data (Steps 5 and 6)
38
3U
Participants will follow the directions in the
Module 5 User’s guide to:
38
• Collect the data needed to monitor selected
activities.
Objectives:
• Tabulate and analyse the collected data.
3)
c
1 31
] 31
Session outline:
1. Introduction (5 minutes)
II. Collect the data (25 minutes)
III. Tabulate and analyse the data (45 minutes)
Materials:
Module 5 User’s guide
Transparency 5-22: Session 5: Objectives
Handout 5-6:
Worksheet for specifying
data collection, supervi
sion and processing
Handout 5-7:
Worksheet for comparing
actual with its standard of
performance
Equipment:
Flip chart, stand, markers, masking tape, over
head projector and screen
LjsjJ
if
if
i*
L*
1E 3
kf
h
Module 5: FG; session 5
54
Session 5: Collect, tabulate and analyse the data
(Steps 5 and 6)
I. Introduction (5 minutes)
Transparency 5-22:
Session 5: Objectives
• Review session objectives.
II. Collect the data (25 minutes)
• Describe steps that are recommended when
using new monitoring forms or instruments,
before actual data collection begins.
Pretest new instruments and revise as needed.
Select and train data collection personnel.
Establish procedures for supervising data
collection and processing forms and data.
• Explain the information contained in Mod
Handout 5-6: Worksheet
for specifying data
collection, supervision
and processing
ules 2, 4, 6, and 7 that can be guidelines for
setting up data collection procedures.
• Distribute Handout 5-6 and instruct
participants to complete it for one of the
techniques identified on the preceding work
sheet, such as a rapid survey, using actual
names, if they can.
• Ask for questions and comments.
Note: If data collection will take place after the
workshop, be sure that participants anticipate
problems that might occur and identify tentative
solutions.
III. Tabulate and analyse the data (45 minutes)
• Explain that data should be tabulated and
analysed to meet the objectives and scope
specified in steps 1 and 2.
• Review the procedure for calculating percent
of standard achieved.
Module 5: FG; session 5
55
Session 5: Collect, tabulate and analyse the data
(Steps 5 and 6)
Handout 5-7: Worksheet
for comparing actual
with its standard of
performance
• Distribute Handout 5-7 and direct partici
pants to complete it and verify the percent
achieved using the formulas.
Note: If data collection will take place after the
workshop, provide them with sample data with
which to practice calculating percent achieved of
standard.
• Explain that the worksheet shows one way
of displaying data for interpretation: compar
ing actual performance to standards. Other
ways are:
Comparing indicators over time.
Comparing performance between health
units or administrative districts.
• Explain that data are most commonly dis
User’s guide
played as lists, tables, and graphs.
• Refer participants to the examples provided
and explain the information in each.
• Ask participants to discuss what conclusions
they might draw from these tables and graphs
or what problems might be indicated by the
data displayed.
• Ask for questions and comments.
Module 5: FG; session 5
56
SESSION 5: OBJECTIVES
• Collect the data needed to monitor
selected activities
• Tabulate and analyse the collected data
iii
Module 5: FG; session 5; transparency 5-22
WORKSHEET FOR SPECIFYING DATA
COLLECTION, SUPERVISION AND
PROCESSING
Responsibility
Staff
Collection
Supervision
Processing
WORKSHEET FOR COMPARING ACTUAL
PERFORMANCE WITH ITS STANDARD OF
PERFORMANCE
Components
Actual
performance
Standard
performance
% Achieved
of standard
Action
to be taken
PHC service (indicators)
Module 5: FG; session 5; handout 5-7
Management
service (indicators)
/.T 6. 3 Q. B fl? ffl B B *9 B B B B B B B ® B I® S B B .*F
59
Session 6: Present findings and take
action (Steps 7, 8 and 9)
Objectives:
Participants will follow the directions in the
Module 5 User’s guide to:
• Report findings
• Take appropriate action
• Decide on continued monitoring
Session outline:
I. Introduction (5 minutes)
11. Report findings (20 minutes)
III. Take action (40 minutes)
IV. Decide on continued monitoring (10 minutes)
V. Conclusion (5 minutes)
Materials:
Module 5 User’s guide
Transparency 5-23: Session 6: Objectives
Transparency 5-24: Report findings
Transparency 5-25; Take action
Transparency 5-26: Monitoring system review
guidelines
Equipment:
Flip chart, stand, markers, masking tape, over
head projector and screen
Module 5: FG; session 6
60
Session 6: Present findings and take action (Steps 7,8 and 9)
I.
Introduction (5 minutes)
Transparency 5-23:
Session 6 objectives
• Review session objectives.
Report findings (20 minutes)
II.
Transparency 5-24:
Report findings
• Discuss briefly each of the key considerations
it
in reporting findings of the data collection and
analysis steps.
• Direct participants to review the users they
identified in Step 1 on the Worksheet for
specifying the monitoring objectives and to
consider;
What information would each user need
from the data that were collected (or planned
to be collected) in Steps 1 through 6?
What is the best way to present the infor
mation to each group of users?
What is the best way to seek their contribu
tion to identifying problems and solutions?
(gE.j
• Ask for questions and comments on reporting
findings.
III.
J
Take action (40 minutes)
• Say that the final activity of the session will
give participants an opportunity to think
about and list the actions they will need to take
to continue the work they have begun on
improving monitoring of their PHC pro
gramme.
Transparency 5-25:
Take action
J
K.
• Review the sequence of key events that fol
low if a problem is identified from the analysis
of monitoring data.
©•
if
Module 5. FG; session 6
j;
61
Session 6: Present findings and take action (Steps 7,8 and 9)
• Mention the Problem-Solving guide and PRI-
2*
COR Operations Research manuals as re
sources for help in pinpointing causes of prob
lems.
36
Note: If the workshop has included actual data
collection, the participants’action plans will focus
on improvement in services that are indicated by
the monitoring data they collected. If participants
will collect monitoring data after the workshop,
the action plan should focus on what steps they
must take to implement the monitoring system
planned during the workshop.
4
J
Ij
4
• Direct participants to review their completed
worksheets and indicate where they need to
get additional information or take some action
(as in Session 4, where they may have iden
tified new data collection instruments to
develop).
46
46
Handout 5-8: Worksheet
for developing action
plans
4
48
48
anticipate any problems that might occur, and
suggest a way to deal with each problem.
0 Ask for questions and comments regarding
48
-48
48
taking action.
IV. Decide on continued monitoring (10 minutes)
Transparency 5-12:
Steps in activity
:4fi
40
do to continue planning and implementation
of activity monitoring and then to develop an
action plan. The plan should include:
Tasks in order of accomplishment.
When each task must be completed.
Who will be responsible for each task.
• Ask participants to review their action plan,
-48
4*8
• Direct participants to list what they need to
Module 5: FG; session 6
3 Review the steps in activity monitoring,
highlighting the outcome of each step.
IB
Session 6: Present findings and take action (Steps 7,8 and 9)
• Emphasise the importance of periodically
Transparency 5-26:
Monitoring review
guidelines
reviewing the procedures used to monitor
PHC programme services and management
functions.
• Review the guidelines for changing monitor
ing procedures.
• Ask for questions and comments.
V. Conclusion (5 minutes)
Flip chart of
expectations from
Session 1
• Review the expectations participants ex
pressed at the beginning of the workshop and
ask them to assess informally how well they
think those expectations have been met. If
any were not met, ask how the workshop
could be improved to make that happen.
• Thank participants for their time and atten
tion, congratulate them on the results of
their work, and wish them good luck in im
plementing their plans.
Note: Indicate what follow-on assistance will be
available, if appropriate.
f l 1$ f l f l .fl f l f l f l f l f l f l f l f l f l f l f l f l f l
62
!?••••
JT
fiE-
u
Module 5: FG; session 6
k'
63
SESSION 6: OBJECTIVES
• Report findings
• Take action
• Continue monitoring
Module 5: FG; session 6; transparency 5-23
if
Module 5: FG; session 6; transparency 5-24
fl fl fl
%
B
f p
IB
f fl
• Seek input on causes and solutions
fl fl
• Highlight major findings
IB
• Use graph and tables
IB IB
1
• Adapt report to user needs
IB
• Review users and . their needs
(from Step 1)
lf>
REPORT FINDINGS
If. IB IB
f
64
65
TAKE ACTION
If problem has been identified:
• Identify cause(s)
• Describe obvious cause(s)
• Collect data to find unknown cause(s)
• Plan and implement solution(s)
• Monitor results of action for proper
implementation and intended effect
Module 5: FG; session 6; transparency 5-25
WORKSHEET FOR DEVELOPING ACTION
PLANS
Action to take
Responsible
Dates
Other
(What)
(Who)
(When)
(Where, how, resources)
Module 5: FG; session 6; handout 5-8
F F F FP. w w iv >* psi (s’
jr nr.
nr [f
n»n ps
67
MONITORING REVIEW
GUIDELINES
As programme operations improve:
• Need for data on inputs becomes less
important
• Need for data on effects becomes more
important
As problems are solved:
• Number of indicators can be reduced
• Frequency of data collection can be
reduced
Module 5: FG; session 6; transparency 5-26
g
■g
PHC MAP MANAGEMENT COMMITTEE
-3
Dr. Ronald Wilson • Aga Khan Foundation, Switzerland (Co-Chair)
Dr. Jack Bryant • Aga Khan University, Pakistan (Co-Chair)
Dr. William Steeler • Secretariat of His Highness the Aga Khan, France (Co-Chair)
Dr. Jack Reynolds • Center for Human Services, USA (PHC MAP Director)
Dr. David Nicholas • Center for Human Services, USA
Dr. Duane Smith • Aga Khan Foundation, Switzerland
Dr. Pierre Claquin • Aga Khan Foundation, Switzerland
Mr. Aziz Currimbhoy • Aga Khan Health Service, Pakistan
Mr. Kabir Mitha • Aga Khan Health Service, India
Dr. Nizar Verjee • Aga Khan Health Service, Kenya
Ms. Khatidja Husein • Aga Khan University, Pakistan
Dr. Sadia Chowdhury • Aga Khan Community Health Programme, Bangladesh
Dr. Mizan Siddiqi • Aga Khan Community Health Programme, Bangladesh
Dr. Krasae Chanawongse • ASEAN Institute for Health Development, Thailand
Dr. Yawarat Porapakkham • ASEAN Institute for Health Development, Thailand
Dr. Jumroon Mikhanorn • Somboon Vacharotai Foundation, Thailand
Dr. Nirmala Murthy • Foundation for Research in Health Systems, India
3
g
3
3
3
PHC MAP TECHNICAL ADVISORY COMMITTEE
3
Dr. Nirmala Murthy • Foundation for Research in Health Systems, India (Chair)
Dr. Krasae Chanawongse • ASEAN Institute for Health Development, Thailand
Dr. Al Henn • African Medical and Research Foundation (AMREF), formerly of
the Harvard Institute for International Development
Dr. Siraj-ul Haque Mahmud • Ministry of Planning, Pakistan
Dr. Peter Tugwell • Faculty of Medicine, University of Ottawa, Canada
Dr. Dan Kaseje • Christian Medical Commission, Switzerland, formerly of the
University of Nairobi, Kenya
ia
KEY PHC MAP STAFF AT THE CENTER FOR HUMAN SERVICES
i)
3
3
a
a
'3
ia
Dr. Jack Reynolds (PHC MAP Director)
Dr. Paul Richardson
Dr. David Nicholas
Dr. Wayne Stinson
Ms. Maria Francisco
Dr. Neeraj Kak
Ms. Lori DiPrete Brown
Ms. Pam Homan
Dr. Lynne Miller-Franco
Ms. Mary Millar
MODULE 5
FACILITATOR’S GUIDE
Primary Health (are Management Advancement Programme
Position: 4604 (1 views)