10810.pdf
Media
- extracted text
-
PARTICIPATORY
TRAINING
EVALUATION
MODULES
Ai
MARIE
THERESE
FEUERSTEIN
1984
1J
PARTICIPATORY
TRAINING
EVALUATION
MODULES
1 Participatory Evaluation - What Does It Mean?
3 What Participatory Evaluation Can - and Cannot Do.
3 Who Can Evaluate?
4 VZho Is an Evaluation For?
5 Planning an Evaluation.
6 Deciding When and Where to Evaluate.
7 Choosing Appropriate Evaluation Methods.
8 Using Existing Materials and Records.
9 Resorces- What Have We Got, What Else Do We Need?
10 Using Surveys.
11 Interviewing - An Essential Skill.
12 Questionnaires, Deciding What to Ask and How.
13 Questionnaires, Recording the Answers and Analyzing Them.
14 Indicators.
15 Some Simple Ways to Assess Health Status.
16 Assessing Health Impact.
17 Baseline Information.
18 Looking at Knowledge, Skills and Attitudes.
19 Improving Monitoring and Record Keeping,
20 Supportive Supervision.
21 Reporting The Results of Evaluation.
22 Ways of Presenting the Results.
23 Using the Results of Evaluation.
MARIE THERESE FEUERSTEIN
1984
iI
. r-
OBJECTIVES OF WORKSHOP ON METHODS ARD APPROACHES TO
EVALUATING CO^UNITi BASED HEALTH PROGRAMS
c
€
C
T.
To consider the general principles and approaches of
participatory evaluation.
2.
To consider such principles and approaches in the
context of the CBHPs in the Philippines and to build
on past and present customary evaluation methods.
3.
To identify the main components of an appropriate
design for participatory evaluation of the CBHPs.
4.
To identify the main objectives of such an evaluation.
5.
To consider a range of participatory evaluation methods
to be employed in the evaluation.
6.
To engage in a small scale field exercise during the
wokshop to enable participants to develop practical
skills in this area.
7.
To keep the costs of the workshop as low as possible
and to utilize only locally available materials in
the production of selected visual aids and during the
field exercise.
8.
To begin the workshop with a participatory planning
exercise to enable participants to develop practical
skills for training and for evaluation purposes.
9.
To use a simple on-going monitoring system daily during
the workshop to assess progress and to identify areas
needing attention/improvement.
10. To organize the workshop notes and proceedings into
an action oriented report for further use by partic-ipants and for eventual publication.
11 . What else?
<«•
V\ -D
*
3
v>
□
fcS
0
5§
a 62 dA Y 5
2 i 1
3
4
5n
yp
j
4/J
i
VJ
Ul
2
,«
a
a.
■fl
X
■i
i/i
< Sk c
JV V)2J Q£2 zv4 3J
T i V”
o.
5 2 fl
■4
o 5
0^
"R i
tn
X’
7
V)
3
Z
3
r
a.
r* •
§
v> g.
Jf
fi 5^
5c
I
s
Q
5
z
r
1
3
O'
r
25
2
3
fr
2?
‘ 1
M c
5(3
2^
I
*J
tl
0
(I
2
0 7
*d
I1b
r
Q .
'■•
r
>1
to
111
3§
3 d S£
0
d
rd n
M
i.d 01 311
2 I 2
0 4
2
cC<£
Z1“L 6
S
31
2 2 2
. <£
n c) 5
<
•i
A
%
iJ
2
c
s5
'b
i
r °J
UJ'S
UJ
IS
IS 1 i
■q
c
J
vT
--
■?r-
VO
8 >
:
84
6
vH
f
li fh
cn
il
c
*1
2 °
i
(3
AT
2
X
v!
.Q
J
<>
*
1^-
H d?
J z>
h
vH i j
J--1
• 2^
4a
o.
£
id, I
(
e • li
ir
2
'S
3
to
<Z)
2
a
— ^A
u-a
a.
!
-8
2^3
2
1
fi 1
h5 o
Ql.
?
tT'A <a’
■2
c
o
•u
£ 8
2—
d
5
6
*oJ
J
C
0-
-5
-3
</l
5□ o
o
li z
b
l (i
,' -3
(y
' 9
*
17
sj
jf
% 3
f-
a4
3
o
J
o
o
Q
2
ru
uJ 4
tA -q
*
i
o
Q
o
1
3
<_
<
0
2
A
<
oo
Q
A-
vl
0
J
MOOULE
PARTICIPATORY EVALUATION
- WHAT DOES IT MEAN ?
Learning Objective:
To consider what evaluation means, and the type of evaluation
where staff, community health workers and community participate
from the beginning and throughout different stages of the eva
luation.
1.
FIRST OF ALL, LET*5 LOOK AT WHAT EVALUATION MEANS
(Ask what does evaluation mean.
mean, to you? Then , record the answers on the
blackboard, or use newsprint hung up in the room which participants can
write IN A FEW WORDS what evaluation means to them. Then, have a look
at all the different meanings and make a SHORT LIST.)
1
u
2.
EVALUATION USUALLY MEANS THESE THINGS
*
*
*
*
*
Assessing or monitoring progress
Surveys
Measuring progress
Seeing what lias been achieved
Seeing if you are moving in the right direction
Looking for success/failure
To help decision-making/planning
To justify past or proposed expenditures
To assess both financial and human costs
To help make work more effective
To collect information
To be able to share positive experience
To gain support for program expansion
To compare different types of programs
•k
k
k
*
k
•k
k
k
3.
WHAT ELSE DOES EVALUATION MEAN?
(If not already included , mention the following:
EVALUATION OFTEN ALSO MEANS THESE THINGS:
* Responding to funding agency request
* Responding to govemment/ministry request
* Demonstrating to others that the program is worthwhile
k
Providing information for decision-making on further support/
funding
Enabling research workers to try out new techniques
Providing new materials for publicity purposes
•k
*
(Having looked at the most couimon meanings of the word ’evaluation’
has helped to show WHY evaluation is carried out. But there are
usually SEVERAL reasons why the same evaluation is carried out.
Not all may be made clear — intentionally.)
4.
SOME ’HIDDEN* REASONS FOR EVALUATING
*
Evaluation results are SUPPOSED to decide future funding BUT the
decision may already have been taken before the evaluation starts
*
Evaluation may be used to cover up faults and failures and only
looks at the strong part of the program
*
Evaluation may be used to gather ’ammunition’ to achieve personal
or group ambitions.
*
Evaluation has even been used to destroy programs which somebody
wanted to get rid of’.
1
5.
$£»
HHAT DOES ’PARTICIPATORY’ EVALUATION MEAN AND HOW IS IT
DIFFERENT FROM MORE TRADITIONAL KINDS OF EVALUATION?
Participatory^ evaluation means that community participation is sought
at different stages of an evaluation and not, for example, only in
answering questionnaires.
7CT? ■
. 7-.L1
h
6.
WHAT ARE THE MAIN STEPS IN PREPARING FOR PARTICIPATORY EVALUATION
WHICH YOU WILL NEED TO REMEMBER?
1)
HELP THOSE INVOLVED WITH/RELATED TO PROGRAM TO SEE THE NEED
FOR AND DECIDE TO EVALUATE
2)
DECIDE WHO WILL BE RESPONSIBLE AND INVOLVED
3)
IDENTIFY OVER-ALL AND SPECIFIC OBJECTIVES
4)
DECIDE WHAT TO EVALUATE AND HOW (e.g., IMPACT, TRAINING, ORGANIZA
TION)
5) IDENTIFY TYPE AND SOURCE OF INFORMATION NEEDED
6)
DECIDE WHICH METHODS TO USE IN OBTAINING IT
7)
DECIDE WHO WILL OBTAIN IT, HOW AND WHEN
~
< 5
I
I.
T-!'■;
-
8)
DECIDE HOW THE RESULTS WILL BE ANALYZED AND REPORTED
9)
PREPARE AND FIELD-TEST MATERIALS
10)
PREPARE A DETAILED EVALUATION PLAN AND BUDGET ON PAPER
11)
DISCUSS HOW THE RESULTS CAN BE USED
12)
PREPARE AND TRAIN ALL THOSE INVOLVED FOR THEIR ROLES IN THE
EVALUATION.
WHAT PARTICIPATORY
EVALUATION CAN
-AND CANNOT DO
Learning Objective:
To consider the expectations, potentials and limitations of
evaluation. Also, to look at the difference between quali
tative and quantitative evaluation.
There are often too many expectations of what evaluation can do.
It is almost expected to provide the answers to all questions and
the solutions to all problems. It is often very useful but cannot
do all these things.
c
EVALUATION IS NOT A MEDICINE
TO CURE ALL ILLS.
1.
What do you expect of evaluation?
2.
So, what can participatory evaluation do?
It can show
* progress and failures
* strengths and weaknesses
* where changes are needed
* how changes can be made.
But, most importantly, it
* involves all program participants in the evaluation
* is an educational process
* enables the participants to see the wider view of their own work
* increases self-reliance in program development.
3.
What it cannot do?
It is often difficult to shew success very clearly, It is often
easier to show failure. But people may try to ” cover up” failure
especially if they want continued funding.
Also, success means different things to different people,
failure.
So does
For example, what is regarded as failure now, may later be seen as
a type of success, and the reverse.
M.T
H
He re are two examples from South America where
a latrine (comfort room)
bui Idi ng p reg ram ai me d to improve community health.
i mt
/
At first the latrines were considered a
success. But,as time passed, the latrines
were used less and less, and never cared
for.
They fell into disuse and decay.
Eventually they became health hazards
dangers — instead of health benefits.
c
'k}
(
In one latrine building project,
the latrines were so well built
of bricks and had locks on their
doors, that people used them not
as latrines — but to store their
valuables, such as bicycles and
chickens. As far as the people
were concerned, the latrines were
a great success — for storing
valuab les.
4.
What is difficult to do — but essential?
Evaluation involves counting and measuring things, such as
numbers of children vaccinated, amount of medicines and her
bal treatment used monthly, area of land cultivated, cost of
materials used, number of TB case findings, number of patients
treated and others.
These activities provide information for the QUANTITATIVE part
of an evaluation. Evaluation also involves analysis of things
which are harder to count or measure, such as people Ts attitudes,
opinions, values, motivations, behavior, expectations, preferences,
and prob lems.
These things are often called QUALITATIVE because they are connected
with quality of people’s attitudes, behavior and program development.
They are equally as important because they can reveal WHY programs
are succeeding or failing, and how they can be improved.
They relate
to PROGRAM GRCWTH.
EVALUATION HELPS YOU TO SEE
HOW YOUR PROGRAM IS GROWING!
H
WHO CAN EVALUATE
I
Learning Objective:
To look at the different people who can be involved in
participatory evaluation.
1.
USING EXPERT EVALUATORS
For too long, it has been assumed that only “experts” could carry
out evaluation. To be sure, there are some highly-specialized
evaluations like those carried out in laboratories, hospitals, re
search institutions and universities which are best carried out by
expert evaluators. But evaluation of a community-based health program
can be carried out by program staff, community health workers, and the
communities themselves.
EVALUATION IS
FOR THE PEOPLE
EVALUATION IS
FOR EXPERTS
CHWs
.
STAFF
COMMUNITY
^EXPERTS
WHAT DO YOU THINK?
/
2.
USING PEOPLE ’’INSIDE” AND ’’OUTSIDE*OF A PROGRAM
Those who are involved in a program usually know a great deal about it.
It is important that evaluation ’’taps” and uses this ’’inside" knowledge.
But sometimes, those involved in the program are afraid to be critical
of the program or afraid to lose their jobs, or want to hide areas of
failure. They may find it hard to be ’’objective”. It is important that
they succeed in being objective — in standing back to critically analyze
what they are doing. If they do not, the information they provide maybe
’^biased” or too influenced by their own opinions.
H
To avoid this bias, "external” evaluators are often used. These are
people trained in evaluation methods and who are not usually a normal
part of the program.
They are supposed to be able to provide infor
mation which is less biased because they can more easily take a fresh
look at a program because they are not personally involved.
However, they cannot know’ some of the ’’inside information which is
also important to an effective evaluation, They may also appear
threatening to a program because they are not part of it. Some evaluations are carried out by both those who are involved in a program
AND a person who is not.
In participatory evaluation, the aim is to increase self-reliance in
evaluation among all those who are involved in a program. For this
reason, the less "outside” involvement, the better.
But .do those involved in the program feel competent to carry out
such an evaluation?
3.
BUILDING THE CONFIDENCE TO EVALUATE
Sometimes, health workers feel they may not be competent to carry
out an evaluation. UTiat about community level workers, what do
they feel?
(Divide into groups and discuss how health workers of different types,
from professionals to community workers feel about carrying out eva
luation under the following headings:
TYPE OF WORKER
EXPERIENCE
IN
EVALUATION
ATTITUDE
TO
PARTICIPATORY
EVALUATION
ROLE
TRAINING
REQUIREMENTS
Then form one group and consolidate findings.)
4.
THE CONTRIBUTION AT COMMUNITY LEVEL
At communi ty
★ knowledge
★ kn ow le dge
* knowledge
level, there is already
about the area
about community structure, customs, leadership, behavior,etc.
about community level opinions, expectations, fears, needs
All this knowledge can be harnessed for planning evaluation, carrying it
out, and analyzing the information collected.
The challenge is to find the best ways in which people can do these
things.
EVERYONE HAS SOMETHING
TO CONTRIBUTE IN EVALUATION.
u
kobULE L|_
WHO IS
AN EVALUATION FOR ?
Learning Objeotive: ’
To identify and consider the various interest groups involved.
FROM OUR ElZALtMTOA/ VW£
CAN J*^E TWAT UzE VFED MPKJ:
ACTION ON THE HEALTH
Dp THE YOUND CH I LOKEN /
€
ii
X-
7
1.
FOR WHOSE BENEFIT IS THE EVALUATION CARRIED OUT?
Is it for a ministry or national agency, so that they can assess their
policy and progress? Is it for a funding agency who wants to assess
progress and cost-effectiveness so that they can give account to the
donor(s) of the funds? Is it a university which hopes to gain new
knowledge about evaluation methods and want to publish the results
of the evaluation? Is it for a community level program itself who
want to assess progress in order plan for the future?
Who else maybe involved?
H
2.
MANY GROUPS MAY BE INVOLVED
Usually, when an evaluation takes place, there are various groups
involved who may have different motives for being involved and have
different needs and expectations.
Does
this matter?
Well, if we compare the progress of an evaluation to the progress of a
jeepney, it is clear that some confusion will follow if people start
off with different ideas about why they are there, what they expect
and where they want to go.
BEFORE EVALUATION BEGINS,
DECIDE EXACTLY WHO WILL BENEFIT.
3.
GIVING PRIORITY TO COMMUNITY LEVEL NEEDS.
Here, ’community level’ means that part of the program which exists
at community level.
Traditionally, community level evaluation needs have been neglected.
Limited or often no feedback of the evaluation findings has been pro
vided to that level.
In participatory evaluation, all those involved in a program take some
part in making the over-all decisions concerning the progress and future
direction of program activities.
b
PLANNING AN
EVALUATION
Learning Objective:
To consider why and how to plan an evaluation and to draft
a pre li men ary plan for evaluation of an individual program.
FV/A LU Ar* (j N FLA hjj.
WHY DO YOH N£ED TO PLAN AN EVALUATION?
An evalui.tiou plan will help you to:
* SELECT THE PRIORITIES AND OBJECTIVES OF THE EVALUATION
* SEE WHAT KINDS OF EVALUATION METHODS YOU NEED TO USE
* PLAN WHAT YOU NEED TO DO IN DETAIL
* DECIDE WHO WILL PLAY WHAT PART IN EVALUATION
* INDICATE HOW LONG THE EVALUATION WILL TAKE AND HOW MUCH
IT WILL COST
A
INCREASE SKILLS IN PLANNING AND ORGANIZATION
z
A
THE BEST WAY TO START PLANNING AN EVALUATION
IS TO LOOK AT A PROGRAM IN ITS OWN SETTING.
u
r
HoOu l—
DECIDING WHERE AND
WHEN TO EVALUATE
Learning Objective:
To consider what needs to be remenbered in deciding participatory
evaluation. The main questions are when & where to evaluate, how
long will it take, and how much will it cost.
J
AiAyfiE WE rHOULU
CCHEDULEV THIS FOR
the duh season •
I
/'
I
I
I
I
when to Evaluate:
Programs who regularly monitor and assess their program may feel that
they are evaluating all the time. Bi^t, there is much to be gained by
a periodic, deeper, wider, and perhaps more systematic evaluation.
This also avoids the piling up of records and reports.
Deciding when to evaluate will depend on the many things which affect
a particular program. These include:
* Time Program has been Operating
For example, before two years few changes may be evident, unlike in
short term vaccination campaign of two months.
I
I
I
H
* tyP6 of Existing Monitoring Methods
For example, do records need to be gathered from far, and what
amount of records are there.
* Climate and Seasons - The rainy season may isolate some communities
or in a city the hot season may make it hard to concentrate on an
evaluation.
/c
People *8 Time at Community Level
Harvest time or a time of food shortages, festivals, and similar events
will take up most of people’s time, At other times they will be more
able to participate in evaluation.
* Time of Program Staff
They also have particularly busy times and times when they are less
b usy.
* Involvement of Outside Agencies
Funding agencies, minist ries , departments , local organizations may also
have specific ideas about timing.
★ What Else?
2.
Where to Evaluate
Tn participators’ evaluation the objective is to involve as much as possible
the program participants in the planning of the evaluation.
(This is in
contrast to planning an evaluation elsewhere and ariving for example with
ques tionaires to be answered).
Planning the evaluation should happen where the program is located. ]Not
all participants will be involved in the same way in all the planning
stages.
For example, a small group will need to be selected to coordinate
the whole evaluation.
With community level participation from the beginning, there will be
greater understanding of the reasons for
j
& objectives of the evaluation.
Also, the questions which are asked in the evaluation are more likely to
reflect community concerns and provide some of the answers to community
level problems.
There may be other questions and other kinds of information which will
need to be collected in response also to the needs of others involved
in the evaluation, such as a funding agency or perhaps a national organization.
Not only the planning & implementing but also analysis of the evaluation
results should be carried out at program level.
In many evaluations
the results are removed for computer analysis elsewhere.
When this
happens participants are ,preventing from taking part in one of the most
important and interesting stages of evaluation-looking; at the results.
- Some documentation and materials used in the evaluation may come from
outside of the area. The less this occurs the better as it may un de r-
U'
mine local self confidence in what is already available at program
level.
3.
How Long Will Evaluation Take?
Some evaluations have taken only few days,
and some have even taken years.
Others have taken many months.
How long fevaluation will take depends on:
* How long the program has been operating
* Whether program is spread over a wide area:
* How much time staff will be able to devote to the evaluation
* Time necessary to prepare for methods such as questionnaires.
* availability of basic resources-financial. material, transportation etc.
* Desire for fast results for planning, funding etc.
* What else?
4.
How Much Will It Cost?
Some evaluations have costs thousands of dollars, francs, pesos etc.
Some evaluation reports do not even mention cost.
Participatory evaluation seeks to keep financial costs as low as possible.
How much an evaluation will cost depends on:
* How much money is available and from what source.
* Objectives and scope of the evaluation
* Material Resources Required
* Whether extra pay is expected for extra work
A
Whether external people are to be involved
The financial cost of an evaluation is only one of its ’costs’. Evaluation
should also be costed in terms of the labor of the people involved.
EVALUATION ’COSTS’
A LOT MDRE THAN MONEY.
H
CHOOSING APPROPRIATE
EVALUATION METHODS
Learning Objective:
To consider a range of both commonly used and innovative
methods by which programs can monitor progress and evaluate
the effectiveness of program organization and activities.
/
SfoT MAP 6F
Tl*
c
1.
HOW IS EVALUATION LIKE DETECTIVE WORK?
It has been said that both evaluators and detectives search out information,
analyze what they find, and then reach conclusions based on their analyses.
Detectives have their own ’tools of the trade’ — but what do evaluators
have?
2.
WHAT ARE SOME FAMILIAR EVALUATION 1 TOOLS OF THE TRADE1?
There are many evaluation tools which are already familiar to the CBHPs.
In different programs, these are already being used. These include:
□
REGUIAR MEETINGS in which staff discuss with the
CHWs and community representatives different
aspects of program activities. There may be
orientation meetings to inform and mobilize.
u
planning meetings to prepare for action, or
progress meetings to assess how the program is
functioning and how effective it is. There are
also meetings of staff, meetings, of CHWs, and
rnmmunitv meetings.
REGULAR ANALYSIS OF RECORDS by CHWs with staff, This
includes analysis and discussion of the treatnent
records kept at community level such as the CHWs’
patient profiles, and the ’’under- fi ves” record
card kept by mothers. The extent to which the
CHWs analyze these records probably varies from
program to program.
In a participatory evaluation
approach, the mothers would also be involved with
CHWs and staff in analyzing the cards of their
own children and others in the community.
□
□
□
□
3.
REGULAR REPORTING like preparing a monthly report
of work or an annual report.
MAPPING AND ’SPOT MAPS’ where the CHWs and staff map
the community and indicate the number of houses,
location of water sources and comfort rooms, public
buildings such as the school, roads, and other geographic features such as mountains and rivers.
SURVEYS AND QUESTIONNAIRES are often used by CHWs and
staff to collect community information relative to
the health status of the community, in order to plan
future program activities based on problems identified
and to plan trainings relevant to community needs.
DRAMA/ ROLE PLAY are used to invoke from the people
certain issues/prob lems — nature of the problem, why
they exist and what to do about them. They draw out
conclusions and the necessary steps to be taken.
They are also used to show the people the progress
of the program — initial stages of the program and
where it is at the moment.
WHAT OTHER EVALUATION METHODS CAN ALSO BE USEFUL
□
□
ATTITUDE SCALES can reveal what people value and what
they value influence their behanor and actions.
These can also shot? what people feel and how strongly.
In group sessions, these can help assess how an indi
vidual or group feel about particular issue. They
then can express themselves without being influenced
by the opinion of others.
PHYSICAL EXAMINATIONS when forming part of a survey can
give a clearer picture of the health status and needs
on a particular time in a particular
j
community. Another simple physical techniqlue used to assess nutri-
H
tional status is the "Shakir strip" for measuring the
mid arm circumference of the child from one year to
five years. The age /height/weight charts used in the
Under Fives Clinics also assess the status and progress.
□
ACTION CARDS provide a simple way of noting steps taken
towards a goal and problems which were encountered.
Blank cards or sheets of paper with few words of
sentences are used by individuals or groups to keep
a record of how their activities begun and progressed.
These can be used at meetings to assess progress and
shared with other groups or individuals who may be
wanting to start similar activities.
□
INTER-BARFIO CASE STUDY SESSIONS between CHWs themselves
enable them to collectively discuss individual problems/cases and a plan of action to be taken. They
are able to assess the success/failure of the plan of
action of a particular case. Also through these sessions,
they can assess their cwn progress in coping with such
problems.
□
PICTURES, PHOTOGRAPHY, DIAGRAMS can vividly record pro
gress and outcomes of program activities. Good photo
graphs, for example, can convey emotional as well as
informative messages and may reveal aspects not evoked
by surveys or other methods.
u
MODuL-GT §
USING EXISTING
WRITTEN MATERIALS
AND RECORDS
r
Learning Objective:
A first step in evaluation is to look at a program in its
own setting. A second step is to look at existing infor
mation relating to the program and its setting. These
steps should be carried out before considering what addi
tional information needs to be collected.
c
\
Hi
MAP w
lANCCAHAN
r
I
1,1
X
I
/
t
I
V
c
1.
WHAT KINDS OF EXISTING MATERIALS ARE USEFUL?
There are three main types of materials: first, the program documents and
records kept by itself; second, what may have been written about it; tand
third, what is written about the area in which the program is located.
2.
WHY ARE THESE MATERIALS USEFUL?
From looking at program documents, the origins and purpose of the program
can be understood (and later be briefly described in the evaluation report).
Programs often change over time in response to different circumstances.
For example, goals can ’evolve* as more important areas of work become
clear. It is important to record whether and why program goals have
changed. More effort and resources may have been shifted towards these
altered goals. In which case, it is not going to be useful to compare
program ’efforts’ to the original program goals.
u
Program records, es pecially health records, can be studied to find answe rs
to questions such as 'How
'
many patients does a health promotor see in a
month, for what reasons and with what results?' An idea of the annual
pattern of morbidity and mortality can be seen,
Changes in these patterns
may also emerge.
Such analyses will help to decide_ what
future
----------------u program
goals there should be.
The organization end administration of a program can also be studied.
Also, patterns of training and supervision. The effects of program
activities and their costs can also be studied.
Materials written by others about the program may contribute to all of the
above points.
They may also have an advantage of the writer not being
personally involved in the program and perhaps being more critical.
Materials relating to the program area can help participants to get a wider
view iof the setting of their own program.
This can help them to assess
thei r cx^zn progress,^0 plan activities for the future, and to see how
their activities relate to others
in- that
------------- j area.
i
3.
A CHECKLIST HELPS TO DECIDE WHAT YOU NEED
PROGRAM DOCUMENTS
Original Program Proposal
Original List of Program Objectives
Reports of Progress
Reports of meetings
Program Records
Program Budget and Financial Reports
DOCUMENTS RELATING TO
THE PROGRAM
Reports on Program Progress not written by
the staff
Reports on Meetings attended by p rogram
s taff
Articles, News clippings. Press releases. etc.
INFORMATION ABOUT THE
PROGRAM
Surveys/reports on that area by others
relating to activities of program interest
National Health Progress Reports and Plans
Map of the area
H
HOOuUfc
RESOURCES
- WHAT HAVE WE GOT ?
- WHAT ELSE DO WE NEED ?
Learning Objective:
To keep evaluation coats as low as possible, it is necessary
to look at existing program resources and to purchase only
what extra resources are really essential.
CJ
1.
WHAT RESOURCES HAVE WE GOT?
A program already has resources of manpower and materials which can
be used for evaluation purposes.
The design and objectives of the evaluation will indicate who will
be involved and how in the evaluation. Different people and groups
will have different tasks and responsibilities.
The material resources which will be needed should be prepared before
the evaluation begins. Perhaps, the following checklist will help
you to decide what will be useful in your own evaluation.
PAPER
For reports and questionnaires,
what have you got? What kinds of
paper are available and what do
they cost. Certain sizes can be
photocopied more easily.
CARBON PAPER
For taking copies
POSTER PAPER
For making posters, plans, charts,
and folders for covering written
reports or keeping papers in during
the evaluation.
PENCILS, PENS,
FELT-TIPS, PAINT
BRUSH
CHALK
Pencil writing can be erased to
make corrections easier and to save
paper. Felt tips are good for making
posters, plans, charts, etc. A fine
pen is good for maps. A paint brush
may also be useful to use with ink or
paint.
White and colored chalk for use with
chalkboard.
ERASERS
To correct pencil writing or liquid
eraser to correct typing or stencils
RULER
A ruler or piece of wood (30 cm. long
and 5 cm. wide) is used for drawings,
diagrams, graphs, etc.
DRAWING PINS,
NAILSi ROPE
OR STRING
GLUE/ADHESIVE
CLIPS, PEGS,
CLIPBOARDS
PLASTIC
SCISSORS,
STAPLERS
FLIP BOARD
TYPEWRITERS
TAPES
DUPLICATING,
COPYING, OR
MIMEOGRAPHING
MACHINE
CALCULATOR
BATTERIES
CAMERA, FILM
TAPE-RECORDER
TAPES, BATTERIES -
or what you usually use to display charts,
posters, pictures, etc.
or what you use to stick paper together.
or what you use 1to keep paper together.
A board is useful for nesting; ppaper on
to write when in the community.
or waterproof covering or bag to keep
papers dry and clean when travelling.
scissors for cutting papers and a stapler
to staple papers together.
And other display and teaching^aids
which you normally use.
For typing the evaluation report and
other evaluation materials
For mass production of evaluation
materials such as questionnaires,
and for making copies of the evalua
tion report. Check if you have enough
ink, powder, spirit, etc. to use the
machine.
A simple and inexpensive calculator
helps to save time and to check your
figures and tables.
A simple inexpensive canera maybe
useful for reporting, illustrating
the evaluation
-a report, producing
teaching or publicity materials.
For interviews, recordings,
meetings, etc.
u
MoDut_& ] o
USING SURVEYS
Learning Cbjective:
To consider the use of the survey in participatory
evaluation.
1. • What is a survey?
e
To ’’survey” something means to look at it in an organized way. For
exanple^ if you want to survey health conditions in a community, you
decide who and what you will look at and how. Perhaps you can look
at all the families in the community. Perhaps, the community is too
big and you don’t have the time or resources to look at every family.
In this case, you will have to choose a ’’sample” or section of families
to look at or surveyed.
2.
Choosing a sanple.
Choosing who will be surveyed in a survey is called "sampling”. It
is not always easy to decide who will be in a sample. How you sanple
will determine the quality of your survey results. For example, if
you choose only the higher income families in a community or an urban
neighborhood, then you may find that their health is quite good. But
you cannot say that the health of all the community is good — because
you never looked at the poorer families of the community who are usually
more sick.
11
There may also be many communities in a program and you have to sample
only some* Perhaps you will choose those communities which have been
involved in the program the longest, Or maybe you choose two which are
easily accessible and two which are very difficult to reach. This way
in selecting certain communities and not
you have a definite
c
.purpose
.
others.
If, in an urban neighborhood for example, there are many hundreds of
families involved in the program, you will have to decide which ones
In doing this, it is necessary to give them an ewill be surveyed.
■
*
qual chance of being chosen for
the
survey,, In such a case, each family
may be given a number and then, the nunbers for the survey drawn "like
This avoid "bias" or prejudice in sampling.
a lottery”.
3.
Planning a survey.
The five main questions to ask when planning a survey are:
★ Why is the survey being done?
•k Where and when will it take place?
* Who is to be interviewed in the survey?
* What evaluation tools will be used(like questionnaires, physical
examinations, etc.)?
* What will happen to the results of the survey?
4.
Using a survey in participatory evaluation.
In participatory evaluation program, participants are involved m de
ciding to carry out a survey, in identifying the objectives of the sur
vey, in planning and carrying it out, and in analyzing the results.
In principle, as many participants as possible should take part m all
these steps.
In practice, the coordinating group for the evaluation
will take a greater part in some of those steps, such as preparing and
field-testing the questionnaires and even in doing an initial analysis
of the results of the survey.
x
HELP THE COMMUNITY
TO PARTICIPATE IN THEIR SURVEY.
11
Mobui—El
11
INTERVIEWING
- AN ESSENTIAL SKILL
Learning Objective:
To consider the principles and practices of interviewing.
o
1.
WHAT DOES INTERVIEWING MEAN?
Interviewing happens when one person meets with another person or group
with the purpose of obtaining information for a particular purpose. Just
dropping in for a chat is not interviewing. However, an interviewer can
be very informal, as where the mother of the family may be preparing the
food at home and the interviewer talks with her and asks questions while
she continues to work. This way she doesn’t lose any tine.
I
Sometimes a questionnaire is completed during an interview. Sometimes
the interviewer uses a few questions chosen to guide the conversation
and to obtain the information needed.
People who are already respected, like CHWs, teachers, etc. often make
good interviewers.
2.
LEARNING HOW TO INTERVIEW
If an interview is carried out well, good information can be obtained
by the interviewer. TLearning to interview correctly is not difficult,
but it requires some basic training,. Perhaps the following points will
remind you what the interviewer needs to do.
12
Useful
Tips
a-T-
PGueRsreiKJ tATV
1)
DECISIONS ABOUT THE TIME AND PLACE of the inter
view will already have been taken through discus
sions with the community. Sometimes, evenings
are chosen as the family will have returned from
the fields.
c
h
2)
DOES THE COMMUNITY KNOW THE INTERVIEWER? If he/she
is not familiar to the community, they will have
to take time to introduce themselves carefully.
An identifying card, letter or badge can be useful.
3)
EXPLAIN THE PURPOSE OF THE SURVEY and what will
happen to the information collected.
4)
MAKE PEOPLE FEEL COMFORTABLE they are being inter
viewed. Sit down and take the time to be friendly.
You may be offered drink or food. People will feel
more able to talk to a friendly interviewer,
Listen
carefully, don’t do all the talking.
5)
FILL IN RESPONSES AT ONCE. Don’t rely on your
memory.
Use pencil. If you want to correct an
answer, cross it through and write underneath or
ah ove.
If you try to erase, it may be confusing
later.
6)
IF THE ANSWER IS UNCLEAR OR VAGUE, ASK IT AGAIN.
Try to get clear answers.
7)
IF RESPONDENTS START TALKING AT LENGTH ABOUT OTHER
THINGS, politely indicate that you are interested;
BUT gently bring them back to the question.
8)
CHECK THAT ALL RESPONSES ARE FILLED at the end of
the interview. If not, fill them in before you
leave.
Sometimes you can begin to analyze infor
mation before the whole survey is completed.
9)
LACK OF COOPERATION OR HOSTILITY MAY BE CAUSED BY
SUSPICION of the purposes of the s urvey (pe rhaps
people think they may have to pay more taxes) ,
TIREDNESS with surveys (maybe others have been
done in that area) and similar causes.
10)
PRIVACY IS UN COMMCN at community level,
Us ual ly,
Try to prevent them in
other people are present,
fluencing the respondent^ answers too much,
much. Sometimes, though, they can validate information.
11)
GETTING TRUE ANSWERS MAY BE DIFFICULT,
People may
not know something ( like his own age); they may
prefer to distort the reality so as not to offend
a visitor; they may be ashamed and claim more than
they have.
1Z)
THANK THE RESPONDENTS for their help and time and
explain what part they will have in analyzing and/or
taking action on the information collected.
S'
A GOOD INTERVIEWER CAN GET
GOOD INFORMATION.
i
u
MoDULEl
QUESTIONNAIRES
ia
DECIDING WHAT TO ASK AND HOW
Learning Objective:
To consider the basic principles of Questionnaire design
and construction.
'
e
■ c ._2_ E
JI ; r 4-1-t
I
(
©
1.
c
H
WHAT IS A QUESTIONNAIRE?
A questionnaire is a group of printed or written questions which is used
to gather information from respondents who provide answers to those ques
tions.
A questionnaire may be as short as one page or as long as a small report.
.The answers maybe written by respondents or filled in by an interviewer.
In participatory evaluation, it is more appropriate to use interviewers
who record the answers. Some people at community level may not be lite
rate.
Sometimes, one person only answers the questions. For example, if a sur
vey is trying to get information on the work of the traditional birth
attendants (TBAs) , a selection of the TBAs might be interviewed and ques
tioned. If a survey is seeking information on the health and development
of a whole family, then the interviewer may question an adult member of
that family but record information about the whole family.
There is also a group questionnaire method when one large specially de
signed questionnaire is used to record information from a group of fami
lies at the same time.
2.
DECIDING WHICH QUESTIONS TO ASK
This is not always an easy task.
L^lV
Sometimes, people want to find out as
o
H
much as possible, so they ask hundreds of questions.
This can exhaust
respondents and often results in an enormous mountain of information —
much of which may never be used.
Decide what are the objectives of the questionnaire and what are the
PRIORITY questions. Then break these down into sub-questions which you
will need to ask in order to get adequate answers to the priority ques
tions .
For example, a priority question might be '*How effective has our health
program been?”
The sub-questions:
a.) What health imp rove men ts have there been at community level in
the program area since the program began?
What
were the goals of the program and how far have they been
b.)
achieved?
?•) Is program management and development closely related to
responding to community level needs and problems?
d.) What other health resources are there in the same program
area and what effect have they had on health changes?
r
HELP THE COMMUNITY TO TAKE PART
IN DECIDING WHICH QUESTIONS TO ASK.
3.
17 BASIC STEPS IN PREPARING QUESTIONNAIRES
(1)
(2)
(3)
(4)
Decide exactly what you want to find out.
Identify the MAIN QUESTIONS and the SUB-QUESTIONS.
Use short questions and simple words not complicated ones.
Decide on the NUMBER of questions and the LENGTH of the questionnaire;
keep it short.
Ask EASY QUESTIONS at the beginning and more DIFFICULT QUESTIONS
later to help respondent and interviewer establish a good relation”
ship.
AVOID questions which may appear like criticisms.
Avoid questions which may make people feel sad. (Like, which school
grade did you complete? — maybe, they never went to school.)
Decide on HOW YOU WANT THE ANSWERS, either a series of boxes where
one or more are ticked, or an open-ended question.
Make the questionnaire EASY TO COMPLETE by careful and clear lay-out.
WATCH THE ORDER of the questions. See that earlier ones do not in
fluence the respondent’s later ones.
Allow for "NO ANSWER", "DON'T KNOW", and "NOT APPLICABLE".
Include ADDITIONAL INFORMATION QUESTIONS like,"If YES, did you..."
Give EXACT INTSTRUCTIONS on how to record the answers.
Leave enough SPACE for answers.
MARK EACH PAGE so that if the pages are separated, you can identi
fy them.
Write INTERVIEWER'S NAME or NUMBER on questionnaire:
for exanple.
(5)
i
(6)
(7)
(8)
(9)
(10)
(11)
(12)
(13)
(14)
(15)
(16)
(17)
/I //8 //3 //6 /
(This means the 36 th questionnaire couple ted by interviewer no. 18.)
The questionnaire must be PRE-TESTED or tried out in practice to
see how well it works and what changes may. be necessary before use.
‘LIBRARY
AND
information
CENTR£
J
i
"•.
P-S 'IOO
11). ID
H
koOULE
QUESTIONNAIRES
t
RECORDING THE ANSWERS AND
ANALIZING THEM
Learning Objective:
To consider alternative ways of recording information
and analyzing it with participants including those at
community level.
1.
DECIDING HOW TO RECORD THE ANSWERS.
At the same time as planning which questions to ask and how, you
need to plan how to record the answers.
There are many different ways of recording the answers,
you can use:
For example.
CHECKLISTS
The interviewer checks through a pre-selected
list and the respondent checks or marks one
or more items. A good checklist must contain
all possible responses.
e
TWO-WAY QUESTIONS
These are questions by which the respondent
chooses only one of only two responses like
agree or disagree, or like or dislike. The
responsdent also needs the chance to say I7HY
he/she feels that way.
MULTIPLE CHOICE QUESTIONS
These are useful when there are several possi
ble responses and you want to make sure that th
the respondent is aware of them. For example,
the respondent may cons'ider using slides as a
learning aid as very useful, useful, not very
useful or useless. Make sure only that one
idea is given at a time and do not let them
overlap, like "useful” and "quite useful".
SCALES
These are used to find out how the respondent
rates, assesses or ranks several things in re
lation to each other. Like: ’’Were the workshop
participants actively involved?"
(Place an X on the scale to show where your
opinion lies.)
Too little
Too much
13
ii
OPEN-RESPONSE QUESTIONS
These are questions which allow the respondent to
say whatever is in his/her mind, like: What do you
consider as the main problems in this community?
They are also useful for adding more information
when used with two-way questions.
For example.
If you dislike it, why?
2.
DECIDING HOW TO ANALYZE THE INFORMATION
At the same time as planning which questions to ask and how to record
the responses, you must plan to analyze the information which you will
collect. Sometimes, the answers can be coded to facilitate the conso
lidation. In participatory evaluation, one objective is to help respon
dents themselves and menbers of the community to take a part in consoli
dating-and analyzing the information collected. The two main ways by
which this can be done are by ’’hard" or by ’’machine”.
3.
ANALYZING INFORMATION BY HAND
There are various ways in which this can be done:
((
USING A TALLY SHEET
This shows all possible responses and you fill in
the nunier of responses received in the different^
categories, using a single stroke to record the
response or information (as in analyzing records).
This is particularly useful where non-literate part
participants are also to be involved in analysis.
Paper or chalkboard are most often used for this.
A-r 1^5
s
UNDE
y
AGE (N
I - 4_
lliiliiili
iliiiii/
M
mu
Illi'
nimiiiiii/ i|||l|llllll
iiiiui1 UH
M COUGH
s
ll/HH
7
/i
I'/
wirH
FEVER. (lllll/llll/l Dllllllllll
/ '1
il h
|P~ 14-
HI
lll
III
II
II
I
u
USING A SUMMARY SHEET
This is a simple method of recording on a board
or large paper some or all of the responses to
questionnaires. A matrix shows the specific
answers inserted vertically in relation to speci
fically coded families or individuals. This is
good for recording horizontally numerical infor
mation.
Then, all the responses are added and
an average can be made from the total nunber
of responses.
4.
ANALYZING INFORMATION BY MACHINE
When there is a lot of evaluation information to analyze, computers are
often used for this task.
In participatory evaluation, however, the
aim is to use the ’’mini-computers” (or brains) in people’s heads to ana
lyze themselves much lesser qualities of information. The process of
doing this helps them to learn how they can best help themselves and
each other to solve the problems which prevent them from having healthy
and satisfying lives.
7
Pocket calculators maybe used to speed up analysis of the greater amount
of information at the next level above the community level.
5.
HELPING THE COMMUNITY TO DRAW CONCLUSIONS FROM THE INFORMATION
The information has been collected and consolidated so that it can lead
to action. If community-level participation takes place at different
stages of evaluation, there will already be awareness of why the informa
tion is being collected. By helping them also to consolidate and analyze
it, the community can also participate in planning for future action.
They will then feel more committed to carrying out action plans which
they have made themselves.
u
kodULG- ll_|_
IMDICflTORS
LEARNING OBJECTIVE: To consider which indicators are needed to monitor and
evaluate progress in PHC, particularly at community level.
WHAT ARE INDICATORS AND WHY ARE THEY IMPORTANT?
’Indicators1 are like ’Kilometre signs’. They show you how far you have
travelled and how far you still have to go in order to reach your
destination or objective.
r
v \ I
f
PHO
-4
2.
THERE ARE SEVERAL KINDS OF INDICATORS
□ INDICATORS
OF
AVAILABILITY
These show you whether
something exists and is
avazldbZe. Eg. One
indicator of progress
in malaria control is the
availability of trained
personnel to take slides
for blood examination.
Another is the. .ratio of
population to a health
worker, eg. CHW.
□ INDICATORS
OF
□ INDICATORS
OF
QUALITY
RELEVANCE
These show what is the
quality or standard of
something. Eg. One
indicator of quality of
water is whether it is
free from pathogenic
organisms and toxic
disease.causing substances.
These show how relevant
or appropriate something
is. Eg. One indicator
of relevance is whether
the training of CHWs
is related to the common
disease and conditions
which they will
encounter at community
level.
H
2
□ INDICATORS
OF
c
□ INDICATORS
OF
□ INDICATORS
OF
ACCESSIBILITY
COVERAGE
UTILIZATION
These show whether
what exists is actually
within reach of those
who need it. Eg. A PHC
post may be available
in one village but due
to mountains, flooded
rivers, lack of trans
port, or people’s
poverty, may be out of
reach of other villages
in the area.
These show what exists,
such as a service, a
structure or health
worker, in relation to
the population in the
area. They also show
what percentage of a
specific group are
receiving specific care.
Eg. The percent age
barrws with a 6HW in
relation to all barri-es
in that area and the
percentage of popi^^ion
thus covered by CHtfs in
that area.
□ INDICATORS
OF
EFFORT
c
These show how much and
what is being invested
in order to reach your
objectives. Eg. Amount
of CHW training sessions
or supervisory visits
within a given period.
These show to what
extent a service,
structure or health
worker is being used
by the population.
Eg. An indicator of
utilization is the
amount of patient visits
to a
in a month.
1 You can also see from
analyzing this the
' most common problems
for which treatment is
sought and the outcome.
□ INDICATORS
OF
□ INDICATORS
OF
IMPACT
EFFICIENCY
These show whether the
resources and activities
which you are investing
are being wisely used to
reach your objectives.
Eg. You may know that upper
respiratory diseases are a
major problem but are still
directing resources and
activities at problems
which are not so major.
These show whether what
you are doing - or
trying to do - is really
making any difference.
Eg. After an immunization
campaign the effect
should be that the
incidence of certain
diseases is drastically
reduced. You can thus
see a change in the
disease pattern.
When you want to use existing indicators or have to make your own (to fit
your program) think about each activity, educational or organizational aspect^in
relation to the nine kinds of indicators which have been outlined.
This way you
can more easily turn evaluation questions into indicators, so that you can ’answer
H
3
the questions’ more systematically and scientifically.
There are many kinds of indicators for looking at education and organizational
activities.
Also, for socio-political and economic aspects of programs.
To look aow
more deeply into ’indicators’ one part of program activities has been selected PHC in relation to ’eight essential components*.
3.
WHICH INDICATORS ARE USED TO MONITOR AND EVALUATE PROGRESS IN PHC?
There are specific indicators which relate to the ’essential components’ of
PHC.
These are:
□
FOOD AND NUTRITION - Nutritional status is a positive health indicator.
Measurements to assess growth and development, particularly the physical growth
and development of young children, are the most widely used indicators of
nutritional status in a community.
Birthweight is also an important indicator of community nutrition.
Low
birthweight may however also be related to certain diseases such as malaria
f
or to specific nutritional deficiencies such as goitre.
An objective of PHC
is that 90% or nine out of ten babies bom have a birthweight of at least
2500g.
The birthweight indicator is expressed as the number of children per
1000 live births whose weight is lower than 2500g.
Other common nutritional indicators is weight-for-age, weight-for-height
and height-for-age.
A positive indicator for example is that 80% of children in a
community are of the correct weight for age.
The Shakir strip measurement of mid-upper arm circumference is a useful
indicator if used with the correct sample size.
Other important indicators (and ones which you may need to construct
yourself) relate to the amount and type of food available in a community
their nutritional state.
relation to rising costs.
Also, the ability of people to purchase food in
11
- 4
□
WATER - One indicator used is the percentage of houses with a sufficient
volume of water for drinking purposes and for keeping the house and its
immediate surroundings clean.
However the existence of a water outlet in a
household does not always mean that the water is safe.
requires drainage.
Also,a water outlet
Where there is no water outlet (like a faucet) in the
house an indicator is the AVAILABILITY OF A WATER STANDPOINT OR PROTECTED
WELL within a given time, for example 15 MINUTES WALKING DISTANCE.
Proper
storage of drinking water also needs to be considered.
□
SANITATION - The indicator often used is the proportion of households with
safe and adequate human waste disposal (comfort houses or similar).
However,
what is regarded as ’safe’ and ’adequate1 needs to be specified.
□
MATERNAL AND CHILD HEALTH - The birth rate is an important indicator.
It
expresses the number of births in a year per 1000 of the population.
High
birth rates together with short average birth intervals (between each birth)
are associated with higher mortality in mothers and children.
The maternal mortality rate reveals the number of deaths due to
complications of pregnancy and birth.
It is expressed as the number of
maternal deaths per 1000 live births in a year.
(Deaths due to abortion are
sometimes excluded).
Other important indicators are the percentage of eligible women (pregnant)
who receive antenatal care, and the percentage of women who are delivered by
trained personnel (including trained Hilot).
In addition the percentage of
women who receive post-natal care is also important and relates to the next
rate which reveals how many infants die.
The infant mortality rate is the number of deaths of infants up to the
age of 1 year per 1000 live births in a year.
To assess this rate a complete
5
record of all deaths is necessary.
Sometimes deaths are not registered if
the infant dies soon after birth, or the family cannot afford the registration
fee.
It is important to collect this information as an ’official’ IMR will
often relate only to hospital data or urban populations.
The real situation
in rural areas is often ’hidden’ by nationwide statistics.
The child mortality rate is the number of deaths in children 1-4 in a
year per 1000 children in that age group at the mid-point of the year concerned
(It does not include IMR).
The mortality rate of all children under 5 years reflects the IMR and
child mortality rates.
□
IMMUNIZATION AGAINST MAJOR INFECTIOUS DISEASES - Indicators here are the
percentage of children immunized against specific diseases in relation to
those eligible for immunization (e.g. in the correct age groups).
It is
also necessary to consider here quality of coverage in terms of the completion
or non-completion of immunization.
t
For example, what percent of those who
received the first dose also completed the other doses.
PREVENTION AND CONTROL OF ENDEMIC DISEASES
’Morbidity’ means the incidence (or amount) and/or prevalence of certain
diseases or disabilities.
It is usually expressed as a rate, e.g. the number of
cases of a particular disease or disability per 1000 persons at risk.
Patterns of morbidity are often reported from health institutions such as
hospitals and health centres.
It is harder at community level to know how many
persons are at risk and also hard to get accurate symptomatic diagnoses.
surveys sometimes give a clearer picture in this respect.
Household
6
Mortality rates for specific diseases are usually expressed as the number of
deaths of those diseases as a percentage of all deaths.
From the records kept by the CHW the percentage of specific diseases and
injuries
for which their help is sought
number of patients.
can be seen in relation to the total
(Be sure to record information so that a difference is made
between first and subsequent visits).
From the records the percentage of children
to adult patients can also be seen.
□
TREATMENT FOR COMMON DISEASES AND INJURIES - Indicators here include the
percentage of specific diseases and injuries seen by CHWs over a chosen period.
Indicators of quality relate to type and effectiveness of treatment.
Other
indicators may be based on the use of oral rehydration in diarrhoea. or access
to first aid which cannot be provided in the home.
□
AVAILABILITY OF MEDICINES
Indicators here relate to the existence of a
selected list of western (and for the CBHP herbal) medicines and the
availability of these when they are needed throughout the year.
For example.
some medicines such as antibiotics may not be available at health facilities
at certain times.
□
REFERRAL FACILITIES - Indicators include the existence and type of referral
mechanisms.
For example, the percentage of patients arriving at hospital
(or health centre) within one hour (or a specified time) of sustaining the
inj ury.
What also
needs to be considered here is the accessibility and
geographical distribution of referral facilities.
even though they exist?
Can people get to them
An indicator here is the percentage of the population
within a defined range of such facilities.
7
□
UTILIZATION OF SERVICES - Utilization is related to coverage, i.e. how many
people in need of a service use it, e.g. children for immunization and pregnant
women and antenatal care.
Sometimes services exist but people don’t use them -
like health centres with no medicines.
Sometimes the hours of the centre do
not fit in with the time people are free.
Sometimes people by-pass a
CHW
or health centre and go straight to hospital.
The number of patients who seek the CHW’s advice is an indicator of
utilization.
□
kitten
HEALTH EDUCATION - Just recording the type and number of health education^and
information given will indicate amount of effort.
What is also needed are
indicators of effectiveness relating to such activities.
did attitudes and behaviour change?
For example, how
For example, what percentage of the
population constructed toilets after a health education campaign?
Some attitudinal changes can be assessed through rating scales.
Where health education is provided by the mass media one indicator is
(
the number of households with such outlets as radios, TV etc.
□
OCCUPATIONAL HEALTH - Indicators here include the existence of facilities for
monitoring health hazards and methods of surveillance of the health of the
workers.
Indicators may commonly be expressed as a PERCENTAGE (amount of a total) such as
n Fifty percent
of the barrios in a specific area have CHWs”, as a RATE such as
Infant Mortality Rate - the number of children under 1 who die in relation to
1000 live births in a year, or a RATIO such as the CHW to population ratio is
8
1/70 population.
1-1
(Section 2^gives examples of other expressions of indicators).
A GOOD 'BASELINE' OF INFORMATION
IS ESSENTIAL FOR USING SOME INDICATORS
J
INDICATORS
/■
Learning Objective:
To consider which indicators are needed to monitor
and evaluate progress in PHC, particularly at commu
nity level.
✓
1.
WHAT ARE INDICATORS AND WHY ARE THEY IMPORTANT?
Indicators are like kilometer signs. They show you how far you have
travelled, and how far you still have to go in order to reach your des
tination or objective.
PHC
Km i
16
PHC
2.
THERE ARE SEVERAL KINDS OF INDICATORS.
Each kind can tell you more about progress in a specific way.
common kinds are:
The most
INDICATORS OF
AVAILABILITY
INDICATORS OF
QUALITY
INDICATORS OF
RELEVANCE
These show you whether
s ome th ing exis ts and is
available.
These show
quality or
something.
what is the
standard of
These show how relevant
or appropriate something
is.
E.g., one indicator of
progress in malaria con
trol is the availability
of trained personnel to
take
slides for blood
examination.
Another is
the ratio of population
to a health worker or CHW,
E.g., one indicator of
quality of water is whether it is free from pa
thogenic organism and to
xic disease-causing subs
tances.
E.g., one indicator of
relevance is whether the
training
curriculum of
CHWs is related to the
common diseases and condition which they will
encounter
at community
level.
H
3.
WHICH INDICATORS ARE USED TO MONITOR AND EVALUATE PROGRESS IN PHC?
There are specific indicators which relate to the essential components
of PHC. These are:
* FOOD AND NUTRITION
Nutritional status is a positive health indicator. Measurements to
assess growth and development, particularly the physical growth and
development of young children, are the most widely used indicators of
nutritional status in a community.
Birth weight is also an important indicator of community nutrition.
Birth weight may however also be related to certain diseases such as
malaria or to specific nutritional deficiencies such as goiter. An
objective of PHC is that 90% or nine out of ten babies bom have a
birth weight of at least 2500 g. The birth weight indicator is ex
pressed as the number of children per 1000 live births whose weight
is lower than 2500 g.
Other common nutritional indicators is weight-for-age, weight for
height and height for age. A positive indicator for example is that
80% of children in a country are of the correct weight for age.
The Shakir strip measurement of mid-upper arm circumference is a
useful indicator if used with the correct sample size (see hand out
on Simple Ways to Assess Health Status).
Other important indicators (and ones you may need to construct your
self) relate to the amount and type of food available in a community
with their nutritional state. Also, the ability of people to purchase
food in relation to rising costs.
€
* WATER
One indicator used is the percentage of houses with a sufficient
volume of water for drinking purposes and for keeping the house and
its immediate environment clean. However, the existence of a water
outlet in a household does not always mean that the water is safe.
Also, a.water outlet requires drainage. Where there is no water out
let (like a faucet) in the house, an indicator is the AVAILABILITY OF
A WATER STANDPOINT OR PROTECTED WELL within a given time, for example
15 minutes walking distance. Proper storage of drinking water also
needs to be considered.
* SANITATION
The indicator often used is the proportion of households with safe
and adequate human waste disposal (comfort houses or similar). How
ever, what is regarded as safe and adequate needs to be speci fied.
* MATERNAL AND CHILD HEALTH
The birth rate is an important indicator. It expresses the number
of births in a
per 1000 of the population. High birth rates
together with short average birth intervals (between each birth) are
associated with higher mortality in mothers and children.
H
\
Indicators may commonly be expressed as a PERCENTAGE (amount of a total)
such as Fifty percent of the barrios in a specific area have CHWs* , as
a RATE such as Infant Mortality Rate — the nunber of children under
one year who die in relation to 1000 live births in a year, or a RATIO
such as the CHW to population ratio 1/70 population.
(Section 2 gives examples of other expressions of indicators.)
A
A GOOD BASELINE OF INFORMATION
IS ESSENTIAL FOR USING SOME INDICATORS.
o
Mo QjuL-^
15
SOME SIMPLE WAYS
TO ASSESS HEALTH STATUS
f
\
Learning Objective:
To consider why assessment of health status of individuals and communities
is important and some simple ways in which such assessments can be made.
1.
What Does Health Status Mean?
Health status means hew healthy an individual or commmity is at a particular
time. By trying to find this out health problems can be identified & plans
for training and action can be made.
e
One way of assessing the health status of a community is by doing a survey.
This can give you a picture of how healthy people are. But, if this is
to be an accurate picture, physical examinations will also be necessary.
Sometimes laboratory examinations of blood, foeces etc are also used to
give a more complete picture.
Laboratory examination may be costly and impractical for many programs.
Physical examinations may also be regarded as difficult. But, there are
some basic physical examinations that CHWs can make either together with
staff, or if trained they can make them alone.
2.
How can the CHW Assess Health Status?
The CHW can be involved in surveying the families in their own area using
prepared forms.
c
They can also carry out some important observations and examinations both to
assess health status & as a normal part of their work by using their own
experience, and their own senses. For example, they can use:
EYES
To note anemia, skin
disease, lumps, sweating,
dark urine, dehydration,
dry skin or lips, coboar of
hair, thin body, swollen
body, red eyes, colour of
faeces, thickness & colourof sputum.
NOSE--^
To note smell of pus or dis
charge, acid or smelly urine,
badly cleaned comfort room,
dirty clothes, dirty skin,
smell of faeces.
EARS
To note type
of cry of baby
cry of pain, type
of breathing,heartbeat,
condition of lungs.
HANDS
To note fever, dry skin, lumps,
height of pregnant abdomen,
skin pirch of dehydration,
swollen body, broken bones,
to use in Shakir nutritional
status measuring strip, to
weigh and/or measure a child.
ii
3.
Thinking About PHC Helps to Plan How to assess Health Status in a
Community
For exanple> assessing whether people have as "Adequate Food Supply & Proper
Nutrition” will involve actions such as assessing the nutritional status of
children & examine the sources, amount, quality and availability of adequate
food for different types of families such as very poor or high risk families
Assessing the availability of ’’safe water” will include looking at the various
sources & quality of water, what it is used for (drinking,washing clothes etc.),
how it is Stored in the hone etc.
(Continue to think how to assess the other element of PHC)
4.
A Simple Way to Assess Malnutrition
If you want to know how many children under 5 years old are malnourished,
the nunber of children you must measure depends on hew many people live
in the community. Here is a list that will help you know which houses
to visit in the community.
Children
People
in
in
Community Community
100
500
1,000
2,000
5,000
8,000
10,000
20,000
50,000
20
100
200
400
1,000
1,600
2,000
4,000
10,000
Children
to
Measure
Houses
to
Visit
20
100
200
200
200
200
200
400
400
all houses
all houses
all houses
every 2nd house
every Sth house
every 8th house
every 10th house
every 10th house
every 25th house
It is often better to visit houses to assess malnutrition because if you
ask people to come to a place for assessment, some children may not cone.
Some children may be sick. Some families may have too many children to
bring.
A SHAKIR STRIP
WILL REVEAL MALNUTRITION
IN THE UNDER FIVES
o
IJ
1 t
‘SHAKIR"
Measure the left arm of a child aged 1-5 years, half way between his
•houlder & his elbcw. Do not squeeze the strip. If the black line
touches the RED part of the strip-the child is malnourished. If it
touches the yellow part it may later become malnourished if not attended
to. If it touches the green part, the child can be considered adequately
nourished.
1i
ASSESSING
HEALTH IMPACT
Learning Objective:
As community health programmes develop, they need to be continuously
monitored and periodically evaluated in order to assess what health
impact is being made. Focusing on family health needs assists
in
understanding this process.
1.
c
What does monitoring mean?
This means looking carefully and systematically all the time what you
are doing to see whether progress is being made towards your specific
objectives.
h
2.
What does progress ire an?
Progress means making gains in travelling towards your objectives.
How <can you tell if you are making gains? By using "indicators"
which are like"milestones"
------------or special signs to show you how far
you have travelled and how far you still have to go.
Ask participants to draw a family on the blackboard and ask them
to draw essential elements of PHC in a wide circle around the family.
]
\ . Keep
the drawings small,as you will also need to write beside each picture.* Then step by step, discuss some specific indicators such as those -shown
the page.
— in next
3.
€
What is the connection between these kinds of indicators and
health statistics?
““
The indicators of child health (like immunization coverage) are
linked to the CHILD MORTALITY RATE (1-4 years) and several
coubined indicators — child mortality rate and infant deaths
from birth to one year — make up the INFANT MORTALITY RATE.
The nuirber of women who die from complications of pregnancy
and delivery make up a MATERNAL MORTALITY RATE. These are some
of the main NATIONAL LEVEL INDICATORS OF PROGRESS in PHC.
COMMUNITY LEVEL INFORMATION CONTRIBUTES TO THE
NATIONAL INDICATORS FOR PROGRESS IN PHC.
4.
How is information collected at community level and what
happens to it?
By keeping routine records on for example, child growth, maternal
deaths, immunization coverage, treatment of common diseases. The
community health worker can be helped to consolidate and analyze
these records and pass them on up to the next level, like to pro-
It
u
!
gram staff or rural health center.
This in turn keeps its own re
cords and so passes information from both the community and first
referral level up to the next level — probably the district hospitai.
GOOD INFORMATION IS NEEDED FOR
GOOD MONITORING.
FOOD/NUTRITION
-availabily of food
-90Z newborn have
at least 2500 g
-90Z children have
correct weight for
age
PREVENTION AND CONTROL
OF LOCAL DISEASES
-% of houses sprayed
against malaria
-% blood slides
positive
IMMUNIZATION AGAINST
MAJOR INFECTIOUS
DISEASES
-X eligible children
immunized against
specified diseases
MArrERNAL AND
G LD HEALTH
-l i ained health
workers available
give prenatal
> d attend deli
ve ry
-i re after birth
i d for children
one year
-c^re for children
i der five years
FAMILY
^ATER
-in the home
within 15 minutes
walking distance
-proper storage
-drainage
CURATIVE CARE
-available at
community level
-herbal medicine
-other medicine
-referral available
HEALTH EDUCATION
-not just the amount
given but the results
SANITATION
-adequate in home
or immediate vicini ty
(N.B. The above are only some of the indicators which can be used!
Ask participants to add others.
F
‘
For example,
relating perhaps to
diarrheal diseases, eye diseases, upper respiratory infections,» TS,
dental health, etc.)
h
(
moduue:
<
BASELINE
INFORMATION
J
Learning Cbjactive:
To consider the importance of baseline information, what
kinds of information need to be collected and how
such
information can be used in regular program monitoring and
evaluation.
WHAT DOES ’BASELINE INFORMATION’ MEAN?
Baseline information about a community is collected preferably at the
beginning of an activity or program. Sometimes such information is
not collected at the beginning. In this case, it can still be collected
later.
WHY IS SUCH INFORMATION IMPORTANT?
It provides a ’base.’or foundation which can be used to understand how
the situation was before the program started, It also bprovides a base
for assessing changes and program progress. For example, perhaps a base
line survey in a particular community shows that only 10% of the houses
have comfort rooms. By the next year in the same community, 70% of the
houses have comfort rooms. From the baseline survey, it can be seen that
60% more houses acquired comfort rooms during that year. However, 30%
of the houses still do not have □uiafuLf'T.ooms.
3.
HOW BASELINE—INFORMATION CAN HELP YOU
When a program is beginning, baseline information can give an overview
or clearer picture of how the situation is in the community. You can
also see where the main problems are. In this way, plans for training
and for action can be made which are based on analyses made by staff to
gether with the community.
4.
WHAT KINDS OF INFORMATION DO YOU NEED TO COLLECT?
A good baseline will include the following kinds of information:
j
* DEMOGRAPHIC
How many people live in the community, how
they are related to each other, and how old
they are. What is the average family size.
H
* CAUSES OF MORBIDITY
AND MORTALITY.
VITAL RATES.
What are the most common diseases and causes
of death by age and sex groups.
This will
help you to see, for example, how many children
die before one year and why and how many women
die of complications of pregnancy and child
birth.
★ STRUCTURE OF SOCIETY,
SOCIO-ECONOMIC CONDITIONS.
LEADERSHIP
What are the main means of livelihood, who
are the poorest families, who are the leaders
and what is the basis of their leadership,
whht are the costs of common commodi ties (s uch
as salt, kerosene) and fertilizer, fishing
nets, etc.
* FOOD SUPPLY, DIET,
WEAN NG PATTERNS
(
* CULTURAL PATTERNS,
COMMON BELIEFS AND
HABITS
What families normally eat, whether there are
shortages, how weaning is carried out, how
much malnutrition there is, and in which
groups(e.g., under fives and pregnant women)
what livestock do they own.
What people believe and do particularly as
they affect health and disease and development.
* ENVIRONMENTAL FACTORS
CAUSES OF ILL HEALTH -
Sources of water and uses. Existence and use
of comfort rooms. Existence of health hazards
(such as stagnant water).
Existence of vectors
of disease(such as rats).
Disposal of garbage.
* USE OF HEALTH SERVICES.
REFERRAL, FREQUENCY
AND METHODS
Availability of health centers and trained staff,
distances from community, me th o'ds of referral.
Use of services for which conditions. Availability
of medicines and sources.
* SELF-CARE.
COMMUNITY SERVICE
* EDUCATIONAL LEVEL
* EXISTENCE OF COMMUNITY
LEVEL GROUPS,
ORGANIZATIONS
What families do normally when they are sick.
Availability and use of home treatment such as
herbal remedies. Use of community resources
such as hi lots or similar practitioners.
How many grades were attained in which level of
school. What further education or training
exis ts.
What kinds of group activities take place,
such as associations. Green ladies, cooperatives,
religious groups, parent-teacher associations, etc.
I i
* CONTACT WITH DEVELOPMENT
AGENCIES
- What contact is there with government or non
governmental development agencies, e.g. , ministries
private organizations, etc.
* SOCIAL LIFE.
RELIGIOUS LIFE.
What are the main social and religious events
in the community.
(See the Research and Development - R&D Form attached as one example of
how to collect information)
5.
WHY COMMUNITY LEVEL PARTICIPATION IS IMPORTANT IN COLLECTING BASELINE
INFORMATION
If the community just answers the questions you ask, they are cooperating
rather than participating.
Help the community to take part in deciding WHICH questions to ask, HOW
and WHEN. They help them to CCNSOLIDATE and ANALYZE, the information
collected. This way, they also learn from the process and will be more
interested and committed to future action plans based on the information
collected.
V’ROGRAM \\ '
04
J?
WEA
BUILD YOUR PROGRAMME
FROM A STRONG BASELINE.
H
M0DVUE1
LOOKING
SKILLS
1
A.T
AND
KNOWLEDGE
ATTITUDES
LEARNING OBJECTIVE; To consider why and how technical
knowledge and skills need to be regularly assessed.
Also, to consider how attitudes influence knowledge,
skills and action.
J
1ARE GENERAL KNOWLEDGE AND SKILLS USUALLY OBTAINED?
Everybody learns as they ao through iffe. Babies learn how to
walk. Children learn how to dress themselves, walk to school, feed
the animals and play games, Teenagers learn how to help in the cooking, the farming and fishing, They learn how to set up their own
homes as they join their parents and become adults. Adults learn
how to feed and clothe the?•r fami1ies and how to keep them healthy
and happy, especially during difficult times. Old people have learnt
all through their lives and their advice is often souaht by vounqer
people.
...
"
SOME OF THE BEST LEARNING
’ IS FROM LIFE EXPERIENCE.
There arc also
< '
special kinds of knowledge and skills
which are needed
ded in health and development work
How are
work.
these obtained?
2. WHAT ABOUT TECHNICAL KNOWLEDGE AND SKILLS?
4
They are often obtained through training and study. This may take
many years (like the training of doctors, nurses, social workers, vete
rinarians, etc.).
It may take a shorter time, perhaps several’months
or weeks (like CHW's, hilots), some basic training courses may only last
a few days (like the barangay health workers).
Sometimes technical knowledge and skills are also learnt from life
experience - like the Hi lot who learns from her own practice. Many
Hilots have undergone training to improve these practices, make them
safer and more effective,
Many other people learn technical knowledge and skills from their
own.life experience, The school teacher for example, learns to teach
more effeetive1y, for actually teaching the school children. The farmer
learns to farm better as he gains experience and learns from his neighhours.
There are many different ways of doing all these trainings. Every
one can always improve their own knowledge and skills as they learn
better ways to improve what they are doing. Often this involves learning
new ways and trying new skills.
BUILD ON PEOPLE'S OWN
KNOWLEDGE AND EXPERIENCE
;
1
H
3.
WHY DO TECHNICAL KNOWLEDGE AND SKILLS NEED TO BE ASSESSED
AND HOW CAN THIS~BE DONE?
'
Even if people start off with good knowledge and skills, they can'
sometimes slip into bad habits. Bad habits can be dangerous, like the
syringes and needle which is not properly boiled, and results in an in
jection abscess on a child’s arm.
It is important to assess knowledge and skills regularly, Some
of the ways in which knowledge and skills are assessed include:
D
WRITTEN TESTS - to see if the information has been absorbed.
This may happen after the training
ORAL TESTS - sometimes the trainer interviews the trainee and
asks questions to find out what the trainee has absorbed.
PRACTICAL TESTS - to see .whether the skills taught have been
. absorbed and can be performed by the trainee.
a OBSERVATI ON - sometimes the trainor or supervisor can assess
correct practice from observing practical skills, like giving
BCG - the anti-TB injection which is correctly given just
under the-skin. Sometimes the CHW can assess that a child is
malnourished - just by looking at it.
O MEETINGS - to assess progress and see how knowledge and skills
have been absorbed and how effectively, they are being put into
practice. This maybe community meetings, CHW and staff meetings.
What ELSE?
If you are going to assess technical knowledge and skills first you
must know if the trainee learnt those particular things. Maybe someone
else gave part of the training not you. Having a standardized writted curri
culum or outline of the course, and keeping a checklist of which top-ics
individual trainees have attended, will help you to check on this.
_________________ •
’
■
’ keeping knowledge and skills fresh I
IS GIVING A BETTER SERVICE TO THE
; '
PEOPLE
J
4.
HOW DO ATTITUDES AFFECT KNOWLEDGE AND SKILLS?
Attitudes are like joints of views' and they influence the way
a person feels about something
They also influence the way a
person behaves.
If for example a health worker is motivated to help the commu
nity to improve its health, that worker will seek knowledge and skills
by which this can come about.
It is sometimes helpful to find out about the attitudes which
people have in order to assess suitability for training and training
effect i veness.
u
Simple ‘Attitude Scales' can be used together with selected
questions-* like:
I
Example: "THIS.TOPIC WAS CLEARLY EXPLAINED" (place a cross
over the point which expresses your own point of
v iew).
I
I
VERY CLEARLY
CLEARLY
I
NOT ALL WAS
CLEAR
MOST WAS NOT
CLEAR TO ME
.■c.
This kind of method can be a fast tway of getting detai led feedback
from a group . 1You can assess the answers in different ways, including
giving
points for very clearly, 3 for clearly,, etc then add and discuss
[ I
15
nK)nitoring\gn;?iicord-kofT
l—
?ook°at\li ffcronJ^ ;
huts in which these things can be done more effectively.
1 . ’.VPY ARE
REGULAR
MONITORING
ANO. RECORD-KEEPING
IMPORTANT?
By using a carefully planned and systematic method
mOni tor i nq you can see whether progress is bet no made. of regular
You can also
identify program st renaths and weaknesses', and see’where the
program
needs to be improved in order to be more effective.
Good nyonitoring is partly dependent on qood record-keeping.
t
2. WHAT IS GOOD RECORD KEEPING?
Good record-keeping means that proaram oarticidants regu1 ar
and systematically record important facts about the work which they
are doing .
"
T
3. WHAT CAN GO WRONG IN RECQ-RD-KEEPI NG?
a
RECORDS MAY ASK FOR TQQ MUCH INFORMATION
§
. Program participants 1
“
like
staff and CHW’s are usually
very busy people.- ‘They don’tt have a lot of t ime to keep
very extensive records.
G
RECORDS MAY ASK FOR TOO LITTLE
INFORMATION
If records ask for too little information they will
not
be sufficiently useful in-order to assess progress and get
a clearer picture of what is really happening.
m.
WHAT HAPPENS
IF NOT ENOUGH
INFORMATION IS COLLECTED?
On treatment record kept by ai CHW
----- some usual headings are
are name
of patient, age, complaint or di agnos i s,, and treatment.
Some records
do not ask for 'Address'or ’Barrio’. Where patients consult CHW's but
come from rneighboring
' ' '
barrios this information is valuable.
It shows
how often people outside the bamiio seek a CHW and may indicates their
need for a CHW in their own barrio.
Some records do not ask for 'Date . Th i s is important for assessing how many patient come, or are visited. daily. weekly ana monthly.
Epidemics can be more c1 ear Iy recorded.
Recording 'Sex and marital status' and
‘occupation* is. reauested
on some records,
This mav also be useful i f not available from other
sources.
II
kF
8
¥ Wf € ■ £s £h £ I| re
. Hr
£
■.i
Z.-'
r.-JF
r
i*,
Some records do not ask for 'Results of Treatment1. Others do not
record when referral was recommended and its results. This information
is very important for assessing the effectiveness of different types of
treatment,
need1 to seek help outside the barrio
,
, the number of •patient who --and what happens to them. For example,
example, if a patient gets to a hospital and gets a consultation and |prescription, does that patient get
the medicines recommended.
recommended. PerhapsJ supplies have- run out, or the patient cannot afford to buy them. This is important information for
assessingJ the general health situation.
situati
• i
■ .
5- WHAT HAPPENS IF RECORDS ARE NOT KEPT REGULARLY?
G
Sometimes records are only kept from time to time.
It is not
to see the whole record of patient treatment. Maybe
cases never got reported. So, the records are only
possible therefore
the most i-mportant
partly usefu I .
Sometimes records are not kept at all. This is a problem because
thre is no way besides memory of assessing the vol ume and type of
sickness occuring in thei community for which the CHW's
-- - hef/p is sought.
Strictly speaking this is very hard to see anyway because only
some people who are sick go to the CHW. Others may treat themselves
.. at home. Others may go to a Rural Health Unit or Hospital.
If you want to assess this kind of 'pattern of ill health' you
have to have more information than CHW Patient Records can provide
With commun.ty participation this important pattern can be seen.
{Divide into groups to discuss.]
a. What information should be collected at community level
including births and deaths?
b. What usually happens to th i s information?
c. How can such information be collected and consolidated
to make the program more effective?
* C
u
HobULE Jo
SUPPORTIVE SUPERVISION
r
Learning Cbjective:
There are many ways in which supervision can be carried
out. The best way is where the the supervisor and the
CHW take time , share and discuss experiences, problems,
and progress. The best supervisors act as ’’advisers” and
give advice and support.
1.
c
HOW OFTEN DOES A CHW NEED TO BE SUPERVISED?
In some program where there is little supervision of the CHW,
the CHW can feel neglected, isolated and unsupported. In other
places, the CHW can even fell over-supervised — maybe, there are
are too many regulations and too many forms to fill.
The best supervision provides the CHW with reliable advice,
regular and appropriate supplies, and a reliable referral sys
tem for people needing further treatment.
c
How often supervision occurs depends on:
* How much community support the CHW normally gets from a committee
or community group.
* Distance and cost — how far and how expensive it is for the CHW
to go to the supervisor or the supervisor to visit the CHW.
* Seasonal factors, for example, whether it is planting or harvest
ing season or whether the rainy season has caused delay by floods.
* Reimbursement cos ts , whether the CHW is being paid for the
cost of the journey and time away from home.
* Length of visit, whether the visit is for a few hours, is
overnight or longer.
* Normal workload, whether the CHW works in a large community and
needs to make more frequent visits for advice/supplies or whether
the community is smaller. Also, how much time the supervisor has
if they are not supervising full-time.
HEALTH
SERVICES
OWN
COMMUNITY
MAKING SURE THAT THE
CHWs HAVE SUPPORT —
HOW THE SYSTEM CAN
FIT TOGETHER
OTHER SECTORS
u
/
f
A GOOD SUPERVISOR PROVIDES SUPPORT
WITHOUT TAKING CHARGE, AND WITH SKILL,
UNDERSTANDING AND PATIENCE.
2.
WHO MAKE THE BEST SUPERVISORS?
Those who trained the CHW and knew him/her make the best supervisors.
An important part of good supervision is friendship and trust.
In sone countries, primary school teachers have'been trained to su
pervise CHWs. They make good supervisors as they learn just what
health knowledge is necessary — not too much — and have teaching
skills to pass on to the CHWs.
Where the staff jsupervise the CHWs whom they have not trained and do
not know, it is difficult for both supervisor and
CHW.
__
----------The staff can
feel they have either not been trained for the job, have little time
for it, or little real interest. The CHW can feel that the staff do
not understand their needs, problems and working conditions. Super
vision becomes submitting forms and collecting supplies. There is
little exchange of experience, advice and information.
-(■
f
THE BEST SUPERVISORS DEVELOP THE CHW’S SELF-RELIANCE,
PROBLEM-SOLVING SKILLS, AND DESIRE TO LEARN
MORE.’
3.
USING EXPERIENCED CHWs TO TRAIN AND SUPERVISE OTHER CHWs
An experienced CHW can be trained to train new CHWs.
In some places.
they have become very skilled trainers,
An experienced CHW can supervise a local network of CHWs.
4.
USE THIS CHECKLIST FOR GOOD SUPPORTIVE SUPERVISION
DOES SUPERVISION INCLUDE OPPORTUNITIES TO LEARN MORE?
CHWs need to increase their knowledge and skills. A visit to or
from a supervisor is an opportunity to learn. Good supervisors
make a point erf giving new inf ormation/ski Ils.
DO CHWs GET A CHANCE TO MEET EACH OTHER?
This prevents them feeling isolated. In some places, CHWs neet
monthly and study new topic or learn a new skill. They understand
each others’ problems.
WHAT ABOUT BOOKS AND OTHER MATERIALS?
A manual maybe used as basic reference book. Pamphlets, news
letters and visual aids can also help the CHW to learn more(for
example, from the agriculture and education departments).
Can
the CHWs be helped to get low cost materials for making teachings
aids to help them in their teaching in their own communities?
H
A REGULAR NEWSLETTER HELPS CHWs LEARN
Sinply written and well-illustrated newsletters (4 or 2 a year)
containing health information and successful ideas and practices
can be produced to help CHWs learn.
CHWs themselves contribute
to the newsletter.
WHAT ABOUT REFRESHER COURSES AND FOLLOW-UP TRAINING?
These are often more important than the first training course.
Sometimes a 2 to 3-week refresher course once or twice a year
is used to continue training and increase effectiveness.
YOUNG CHWs CAN LEARN FROM OLD ONES
If a young CHW can spend a few days learning in the home of an
experienced one, this also helps those who find course learn
ing hard.
KEEPING IN CONTACT WITH OTHER CHW PROGRAMS
There are other countries who also have long experience with CBHPs
like ACHAN (Asian Community Health Action Network in Hong kong).
bfenbers attend regular meeting to discuss common problems and
experience and exchange staff as advisers and as part of further
training programs. Sometimes, one program has special skills to
teach others, such as working with women at community level or
how to work effectively with other development sectors such as
education, community development or agriculture.
4
I
<
1
«
P
TRAINORS ALSO NEED SUPPORT
AND LEARNING OPPORTUNITIES.
I
1
I -V l>o
H
RESULTS
EVALUATION
REPORTING
THE
1 NG_ OB J ECT IVE :
To consider how the results of par
ticipatory evaluation can be reported and what part the com
munity can play.
...
i.
„
HOW ARE
EVALUATION
RESULTS
USUALLY
REPORTED?
... Tjiey are often reported only by staff, researchers, experts, etc.
There -is usually minimal participation by CHW’s ior representatives from
community level regarding what finally goes into» a written evaluation
report.
-Often the evaluation report is full of complicated statistics and
technical terms. This is because it is usually intended for readers
fsTpm organizat ions, universities, funding agencies, etc.
2.
REPORTING THE RESULTS IN PARTIC1PATORY EVALUATION
-
•"
t When partic<pants have gone through the processes of planning and
imp 1 ementi-ng
evaluation they will be ready to report the results.
Some of the results will be reported in wrJt-fnq, some in figures.
There may be brief reports prepared by different individuals and groups.
There will also be consolidated resu1ts’such as those from surveys and
other similar information. Some of these wi11 be in the form of numbers.
In the basic analysis and consolidation of these numbers the community
can also be-involved most probably through the CHW’s.
j
IS: |
t-".
F-rom this consolidation simple percentages can be made by staff and
CHWs.
Whst is important is that the meaYiing behind these kinds of sta
tistics and percentages is clearly understood.
-r
GET THE MEANING -
7
:
NOT JUST THE NUMBERS
3.
....'. ‘i
WHY IS IT IMPORTANT TO INVOLVE THE COMMUNITY IN REPORTING THE RESULTS?
‘Reporting the results’ is one of the most important and most in
teresting parts of an evaluation. It-is also the part which is supposed
to indicate program strengths and weaknesses- The strengths can be
built upon, the weaknesses clarified and remedial action taken.
The resul ts<indicate what future pl-ans need to be made. 'They also
indicate the best ways to put the plans into action. Both community
and staff will be better able to plan and implement such action if they
have shared this decision-making for planning and action. Communities
will be more committed to planswhich they have had a part in making.
ii
>'
•
•
X
^7
T7
Reporting the results
of evaluation - - p. 2
4. WHAT TO AIM FOR IN PREPARING THE EVALUATION REPORT
Keep it SHORT - very long reports tend to be less used than short
ones. Who has the time to read a long report?
'
/y
;
.
Write it SIMPLY - it is supposed to be. read and understood. Avo i d
■‘.very techn i cal words or ‘jargon’. It will be easier to translate.
.
•»>
Use familiar and precise words.’
Use SHORT SENTENCES - not more than 20.words (and if possible less
than 16)77 You may also wish to write some parts in 'note' forms
to save spacfe and the reader's time. Don't put many ideas In one
sentence. . Use several sentences with not more than two ideas in each.
.
Present it in a CLEAR, LOGICAL ORDER
For example an introduction is useful, Then a brief history of the
program. Then the specific groups of evaluation results relating
- for example to community level effects, (impact) program planning
_and management (efficiency and effectiveness), Then final 1y con* S "z
elusions indicating future action. •
Give a LIST OF CONTENTS
quickly.
r
to help the reader find specific results
Include CHARTS, GRAPHS, TABLES etc. to present some of the results
in clear and concise form and to help the reader to absorb the in
formation. "Jt is.easier to absorb the results if such a visual
presentat ion -is used.
■ ’ ’
■
.
-
•
“5
Keep the production COSTS LOW by us[ng inexpensive and locally
available paper and materials, A thicker cover or folder is needed
to ensure that the report stays together under the handling of many
people.
What ELSE?.
w
REPORTING EVALUATION ^RESULTS
MEANS CAREFUL ASSESSMENT OF
THOSE RESULTS.
•-.M
H
WAYS
PRESENTING
RESULTS
OF
THE
V I
LEARNING OBJECTIVE:' " consider ways in which evaluation
results can be shared,, particularly at community level.
1.
WHY DO EVALUATION RESULTS NEED TO BE PRESENTED IN VARIOUS WAYS?
cThe evaluation results are designed to produce action,
But
the results cannot be acted upon if they are not fully understood.
When evaluation results appear only in complex statistics or
expressed in technical 'jargon' they are understood by only a few.
In participatory evaluation the objective is to share the results
with the majority of program, participants . In order to do this
the results need to be preserved in various simple and clear ways.
c
2.
a
HOW CAN WE HELP PEOPLE TO UNDERSTAND THE STAT I ST ICS?
Some of the information collected and consolidated during eva
luation consists of numbers. During consolidation these are turned
into simple statistics. This is necessary in order to summarize a
large amount of 'numerical' information. The final statistics help
to show the real meaning of those numbers. For example, we know
that there are 100 families in the community. The evaluation re
veals that 300 children under 15 years have some condition requi'ring
attention (such as adequate immunisation coverage, skin diseases,
intestinal parasites, etc.). If we say the average family has 4
children under ’5 years (may have more, some have less) then there
are likely to be 400 children in those 100 families. We can see
then if 300 require some kind of attention, that three quarters
or 75^ or 3 out of 4 of the eligible children (under 15) are needing
attention.
J
In this way the size of the problem can be more clearly seen.
Plans can be made for increased- action to improve child health.
3.
HOW CAN EVALUATION RESULTS BE PRESENTED CLEARLY?
There are usually several different ways of presenting the same
results. Choose the one which is most suited for your own purposes.
BAR CHARTS:
These are charts which present information
horizontally or vertically in the form of 'bars' which
.relate to the magnitude of the information given.
ueAbouq
exposes'
r
-/'zi
Z-'/Tz/Zj,
* A-v-\nrV4 c.
..^1
H»4 P etc. .
u
USING A SUMMARY SHEET
This is a simple method of recording on a board
or large paper some or all of the responses to
questionnaires. A matrix shows the specific
answers inserted vertically in relation to speci
fically coded families or individuals. This is
good for recording horizontally numerical infor
mation. Then, all the responses are added and
an average can be made from the total nunber
of responses.
4.
ANALYZING INFORMATION BY MACHINE
When there is a lot of evaluation information to analyze, computers are
often used for this task. In participatory evaluation, however, the
aim is to use the ’’mini-computers” (or brains) in people’s heads to ana
lyze themselves much lesser qualities of information. The process of
doing this helps them to learn how they can best help themselves and
each other to solve the problems which prevent them from having healthy
and satisfying lives.
Pocket calculators maybe used to speed up analysis of the greater amount
of information ht the next level above the community level.
5.
HELPING THE COMMUNITY TO DRAW CONCLUSIONS FROM THE INFORMATION
The information has been collected and consolidated so that it can lead
to action. If community-level participation takes place at different
stages of evaluation, there will already be awareness of why the informa
tion is being collected. By helping them also to consolidate and analyze
it, the community can also participate in planning for future action.
They will then feel more committed to carrying out action plans which
they have made themselves.
H
. [GRAPHS:] Presenting information this way helps to show whether
a situation is getting better, has now changed, or is getting
worse.
It helps to assess progress.
a
35
30
25
O-j
0
lo
?n
t -0 IS
o)
A C
1 Qj
2 Qi
7.^
(
io
5
0 ______ U._____ 3^.
Age. in
7^ars
bU
[pie DIAGRAM. :j This is round like ‘ or pie (or a round fruit).
It shows the parts of something in relation to the whole
thing.
It is usually better not to go beyond a 10% 'segment1
if the diagram is to be kept simple.
V
IQ2U
teACdA^
of
V^A-CXH a aAo-J
pnbc tA Tjavao CstH
■1—ei
[PICTOGRAM: J Th is uses*-simple pictures (usually outlines)
to show the meaning of the information presented.
ouve__
cruet to
vv\-^.tAeu^
p>/e. cUA4 AJ2. IA_
TABLES: } Most evaluations need to present some informat ion in
the form of tables, This is good for reference but not easy
for learning or sharinq. Keep your tables clearly presented,
Perhaps you can present some or all of this information in
pictures or another visual form. For example a list of current
costs of common commodities can be turned infcc pictures accompaniedby figures.
H
■
’ |P1CTURES; j Turning results into pictures helps people to
• absorb the information. It also helps to provoke analysis
and discussion. The pictures may be drawings, cartoons; even
photographs if costs.allow.
• RAPE RECORD I NG: | By turning results into a .’rad;o program4
(complete with musical interludes) people can be helped to
absorb the information, to discuss it and to plan for future
action. This way you can share results with a large croup,
like a community meeting. Use several voices and make the
recording interesting, amusing, dramatic.
’ ir OP-AH A: H This helps people to share evaluation results in an
act ive wav. .-They, remember ..what they see and hear in this way.
11 also provokes further analysis and discussion.’
.
HANDOUt S: |
If f-unds allow this method is very useful for pro
ducing short leaflets relating tc various evaluation 'results.
They can be used for.stucy, training, publicity, etci
What ELSE?
. A
/
■J: •
11
USING THE RESULTS
OF
EVALUATION
LEARNING OBJECTIVE
To consider different ways in which the evaluation results
should be used.
I. EVALUATION RESULTS -
chostng them or using them.
In some evaluations it takes a long time to report the results.
in one case it took a year for all the information col
lected to be computerized and the results finally written down, By
the time the results were reported
they were out of date and not so
useful anymore to the action program.
For example,
In participatory evaluation the results are designed to help participants seei their own progress and to take decisions concerning futore action.
RESULTS
DECISIONS
ACTION
When the participants themselves report the evaluation results
they can see immediately what the results are, and how they can be
used to increase program effectiveness.
2.
DIFFERENT GROUPS CAN USE THE RESULTS IN DIFFERENT WAYS.
Think about who can use the results.
One group are those at community level. They can use the results
in training sessions, and in community hea1 th education sessions (like
re-echo seminars).
If the results-are carefully presented and dis
cussed, they -can help the community to analyze their own health situa
tion to see the need for action and to plan for that future action.
Program staff can use the results for the same reasons, and also
to improve program organization and management.
Other groups such as funding or research agencies connected with
the program, can share the results in order to make decisions relating
to program support. Other selected organizations and programs nationally
and in other countries can also share the results in order to improve
their own programs.
If the results are shared in this way there may be
sections of the evaluation report which it is preferred NOT to share
publicly, such perhaps as details of funding and specific names of in
dividuals and places.
0
£K
or- t
'fert’'
r ’
£
. -1
rt/ e I (*'4 fei.- x
id ȣ l-
IA1V 4
t; t-
IV
Decide what you Wish to share with such groups and prepare their
copies of the report accordingly.
. J t—
.L
- . ; . ____ t______ J
FEEDBACK
INCREASES
AWARENESS - ■
OF
NEEDS --- AND - - SOLUTIONS
.r
:
L- r* ?
.r
-n': r- -
IN WHAT FORM can the evaluation results be shared?
The whole report of [he evaluation wi11, probab1y be distributed
among program staff in different places,
places,and
and to agencies or organizetions selectedby the program. ’Due. to costs of product!
production only a spe
cific number of the whole report is likely to be produced.
'£’’-
One form “i n wh i ch evaluation
<
results can be shared with a .wider
audience, is in the form of a regular newsletter. Maybe you are already
produc i ng one.
If not. maybe the evaluation will indicate the.need for
such a newsletter, Selected results can then be shared in an imaginative and clearly designed newsletter. For.example the evaluation in
1981 of the ChiId-to-Chi1d program resulted in a >very
’
successful news
letter which was easily mailed to program participants iin various
countries.
••
~
-.... 1
-
-
Another example of sharing results took place in Ecuador in South
Ame r i ca . The evaluation results were turned into simple pictures and
shared at a meet mg wi th''Sixty
i pan ts i nc 1 ud 5 ng CHW1 s ‘ “rura 1 •
physicians and un i vers4 ty ~s taf f
In the Philippines a group of farmers evaluated their first training
session by drawi’ng what they had learned during their training, This was
another way of-turning evaluation results into simple pictures.
-y.
Other -i-nnovat ive methods i nc 1 ude-us ing tape recordings, drama, etc.
[Additionai -suggestions a re-presented in "ALTERNATIVE WAYS OF PRESENTING
THE FINDINGS.'-]
11
LEARNING OBJECTIVE i
To look at what woxkshopping means to know how to prepare a work
shopping to Identify key people in a workshop
WHAT DOES "WORKSHOPPING0 MEAN?
1.
Workshopping is a group activity whichouses available resources to
discuss, plan, and produce certain outputs0For example these may be audio
visual or a plan or action. This output represents, explains, and/or
analyze one or more aspects of a chosen problem.
Group discussions are an important part of the process, These may
include reflections on past experiences.
c
.
In workshopping there are certain key people involved in seeing to
the smooth flow of the workshop. These are the FACILTTATOH the PROCESS*
OBSERVER and the RECORDER^ They are also participants in the workshop
discussion.
WHY ARE THESE PEOPLE IMPORTANT?
2.
A workshop is conducted the way music is produced,
key people help make the music sound better.
In a rondalla,
The FACILITATOR of a workshop is like the conductor, He helps to
make the guitars, banduria and drum flow smoothly together.
The PROCESS OBSERVER is like another musician who listens to the
music and observes the performance of the other musicians. He notes sounds
that are out-of-tune and helps the conductor. Process Observer in a
workshop observes group activity and takes charge of providing feedback
to the group on how the discussion process is going.
The RECORDER tapes the music and plays it back to the rondallaso that
they will know how they sound. The workshop recorder writes dowr^ the
workshop results.
PREPARING FOR THE WORKSHOP
3.
C l CHOOSE A GOOD TIME, Plan the workshop at a convenient time for
participants. Evenings or weekends are often best at community
level. Avoid times when people are busy planting, harvesting, or
having a fiesta.
□ CHOOSE A GOOD PLACE,
and comfortable.
|
How easily can people get there?
Is it suitable
I THINK ABOUT SIZE. How many will be present? 25 to 30 is a good
size. If more people are present it may require a different style
of facilitating. For example there may be more people who want to
express their views. This will need more time.
Hoxkshopping - p. 2
Cive
INTRODUCE THE WORKSHOP Keep objectives few and
workshop activities and questions.
simple instructions. Explain
1
4. TIPS FOR IMPROVING YOUR WORK AS I
a) FACILITATOR
BE CLEAR ABOUT. THE PURPOSE] OF THE VIQRKSHOP. Know why that particular
topic was selected and what the workshop is supposed to achieve?
Some may be shy or not readily
able to share their own knowledge and experience. Others may be
easily distracted and do something else.
O ENCOURAGE EVERYONE TO PARTICIPATE..
,
Encourage questions and
n show how they can achieve______________
a deeper discussion of the topic.
PROVIDE OPPORTUNITIES/TIME FOR QUESTIONS.
__________
BE OPEN TO ALL COMMENTS/REACT
TOl^S 0L Help participants to express their
Help
to
resolve
tensions or conflicts which may
own strong views.
f
arise.
When participants get too far away
from the topic usually help them to focus the group discussion and
get back to the main topic.
Q TRY TO KEEP ON THE MAIN TOPIC .,
{ ( KNOW THE PARTICIPANTS. Get information on their educational level,
their background experience and level of awareness. If possible,
know the participant^9 different personalities and pattern of beha
vior.
This helps you keep the discussion clear to all.
'.J
Q ENCOURAGE PARTICIPANTS T0_ INTRODUCE THEMSELITES. Include some personal
and work-related details.
BE SENSITIVE AND CREATIVE,
perhaps using visual aids.
Keep discussion lively and interesting,
Write down comments in order to focus attention
and stimulate discussion.
D USE A
BLACKBOARD.
Notice if the question is getting too long,
Q WATCH[ THE TIME.
the participants getting restless?
Are
___________
PLAN THE TIME, Have an introduction and middle section and leave
enough time for conclusions.
Q ACKNOWLEDGE PARTICIPANTS* CONTRIBUTIONS to the success of the workshopP
MAKE THE EXPERIENCE ENJOYABLE.
snacks, etc.
Encourage humurous and breaks for
u
woikshopping - p. 3
b) TIPS FOR THE PROCESS OBSERVERi
D NOTE THE FOLLOWING!
1.
Active and less active participants.
2.
Tendencies to stray
3.
Tendencies to stay too long on one topic.
from topic.
D FEEDBACK THIS I?lFOBM/kCION to assist the facilitator,
c) TIPS FOR THE RECORDER i
□ WRITE DOWN MAIN IDEAS, key words, aud. illustrative examples ■f'TXMn
the discussion.
□ ORDER YOUR NOTES INTO MAIN HEADINGS AND SUB-HEADINGS.
Q READ OUT NOTES to group to know if discussions have been
properly recorded.
Q WRITE LEGIBLY.
Seminar Objectives—continued
Course Chairman
the role of listening—the lost art of information gathering;
personal communications—communicating with
individuals face to face; organisational communications—
an analysis of the formal and informal information flows;
the physical environment and its role in facilitating or
hindering communications; practical guidelines for
effective communications in any organisation.
Dr Patrick T Kehoe
Chairman/Managing Director, Patrick Kehoe &
Partners.
Management of Change
The Manager as a change agent; behavioural,
psychological and social aspects of any change; predicting
degrees of resistance to change; how rigid individual
attitudes can be changed; how to liberate the rigid
organisation; successful change strategies identified; how
best to monitor changes.
The Management of Conflict
No organisation exists without a minimum level of
conflict—if no conflict a ‘dead’ organisation—if too much
conflict a ‘dead’ organisation; factors that cause most
conflict in organisations; an understanding of personality
conflict and role conflict in organisations (i.e. the
operation of the organisation’s structure); the way most
Managers handle conflict ineffectively; an identification
of the dominant way each Manager (participant) manages
conflict; minimum conditions for conflict resolution.
Management of Creativity
Dr Kehoe has been a management consultant and a
Professor of Business Administration in a number of
countries, most recently in Canada where he was
Chairman of the School of Business Administration at the
University of New Brunswick. He has had wide
management and marketing consulting experience in
North America, Australia, Europe and South East Asia.
He is a specialist in Organisational Effectiveness and has
completed Organisational Climate consulting
assignments in industry, banking, insurance, government
departments and in an international airline organisation.
He was a banker for some eight years.
I
I
i
I
!
I
.
■
How to be an
Effective
Manager
Dr Kehoe holds a Barrister-at-Law Degree, a B.Comm.,
a Diploma in Public Administration, and a MBA
(University of California). Fl is PhD which was completed
at Strathclyde Business School, Glasgow, was entitled
“The Effective Organisational Climate”.
o
Dr Kehoe has presented management and marketing
seminars for bankers and financial executives in Canada,
UK, Ireland, France, Singapore, Malaysia, Thailand
and Mexico.
Creativity is the generation of alternatives; each employee
has a level of creativity which is seldom utilized; the
barriers which prevent the employee being creative;
barriers which inhibit groups (teams/committees) from
being more creative; why organisations do not have a
creative posture—the competitive edge; some practical
creativity techniques; how the Manager can establish a
stimulating climate for the generation of creativity.
10-12 December 1985
lr
Management of Coaching
\
90% of an employee’s performance depends on
relationships with his/her superior; the Manager as a
coach or a poor player; coaching (performance) versus
counselling (personality); ensuring employees know what
is expected of them; how best to delegate, not abdicate;
the coaching process—an informal practice; coaching
guidelines for increased employee performance.
The University reserves the right to alter the programme
and venue as necessary.
THEmJCITY
UNIVERSITY
8-Vsims school
I
GM14
4
The Northampton Press Limited, The Oval, Hackney Road, London E2
i
11
J'J
J/?
^3
u
H
■4
£ A-
1
A•
d
2
iJ !
A
1
U"
a
Q
<r
•' k
Q
c^-
ijt
J
s
CL
_£
o
u
$
2
Q
2
2
(X
z
o
j
>
Lv
“T'
fJ
2 h?
u
k
Ja
Position: 2658 (2 views)