BASIC SENSORY METHODS FOR FOOD EVALUATION
Item
- Title
-
BASIC SENSORY METHODS
FOR FOOD EVALUATION - extracted text
-
-
Basic
sensory methods
for food
evaluation______
B.M. Watts, G.L. Ylimaki, L.E. Jeffery, L.G. Elias
I
I3L
K.
I
9^1
J
1
I'SPCLIC
The International Development Research Centre is a public corporation created
by the Parliament of Canada in 1970 to support research designed to adapt
science and technology to the needs of developing countries. The Centre’s
activity is concentrated in six sectors: agriculture, food, and nutrition sciences;
health sciences; information sciences; social sciences; earth and engineering
sciences; and communications. IDRC is financed solely by the Parliament of
Canada; its policies, however, are set by an international Board of Governors.
The Centre’s headquarters are in Ottawa, Canada. Regional offices are located
in Africa, Asia, Latin America, and the Middle East.
SOCHARA
Il existe egalemeni
La edicidn espanol
Community Health
Library and Information Centre (CLIC)
Community Health Cell
85/2,1st Main, Maruthi Nagar,
Madiwala, Bengaluru - 560 068.
Tel: 080-25531518
email: clic@sochara.org / chc@sochara.org
www.sochara.org
IDRC-277e
BASIC SENSORY METHODS
FOR FOOD EVALUATION
B. M. Watts
G. L. Ylimaki
L. E. Jeffery
Department of Foods & Nutrition,
Faculty of Human Ecology,
University of Manitoba,
Winnipeg, Manitoba,
Canada
L. G. Elias
Institute of Nutrition of
Central America and Panama,
Guatemala City,
Guatemala,
Central America
Prepared with the support of
The International Development Research Centre,
Ottawa, Canada
*
© International Development Research Centre 1989
PO Box 8500, Ottawa, Ontario, Canada K1G 3H9
Watts, B.M.
Ylimaki, G.L.
Jeffery, L.E.
Elias, L.G.
IDRC-277e
Basic sensory methods for food evaluation. Ottawa, Ont., IDRC, 1989. x + 160 p.: ill.
/Testing/, /food technology/, /agricultural products/, /consumer behaviour/, /nutritive
value/ — /planning/, /group discussion/, /work environment/, /hand tools/, /statistical
analysis/, /manuals/.
UDC: 664.001.5:339.4
ISBN: 0-88936-563-6
A microfiche edition is available.
This work was carried out with the cud of a grantfrom the International Development
Research Centre. The views expressed in this publication are those of the authors and do
not necessarily represent those ofIDRC, the Department ofFoods and Nutrition at the
University ofManitoba, or the Institute ofNutrition of Central America and Panama.
Mention of a proprietary name does not constitute endorsement of the product and is
given only for information.
CONTENTS
FOREWORD
ix
PREFACE
1
IbTRODUCTION
5
Using Product-oriented and Consumer-oriented
Testing...................................................................
7
1.1
1.2
CONSUMER-ORIENTED TESTING
PRODUCT-ORIENTED TESTING .
8
9
Chapter 2
Designing Sensory Testing Facilities
11
2.1
PERMANENT SENSORY FACILITIES
2.1.1
Food Preparation Area ....
2.1.2
Panel Discussion Area ....
Panel Booth Area ..............
2.1.3
2.1.4
Office Area........................
Supplies for Sensory Testing
2.1.5
11
12
13
14
18
19
2.2
TEMPORARY SENSORY FACILITIES
2.2.1
Food Preparation Area ....
2.2.2
Panel Area..........................
2.2.3
Desk Area..........................
2.2.4
Supplies for Sensory Testing
23
23
25
25
25
Chapter 1
2.3
Chapter 3
3.1
3.2
3.3
3.4
3.5
3.6
DESIGN OF A SIMPLE SENSORY TESTING
LABORATORY .................
26
Establishing Sensory Panels
29
RECRUITING PANELISTS............................................ 29
ORIENTING PANELISTS .............................................. 30
SCREENING PANELISTS FOR TRAINED PANELS . 31
TRAINING PANELISTS ................................................ 32
MONITORING PANELISTS’PERFORMANCE........ 33
MOTIVATING PANELISTS .......................................... 35
‘' ■■f
■
' • . Hi
Chapter 4
Conducting Sensory Tests
37
SAMPLING FOOD FOR SENSORY TESTING
PREPARING SAMPLES FOR
SENSORY TESTING .......................................
PRESENTING SAMPLES FOR
SENSORY TESTING .......................................
USING REFERENCE SAMPLES.....................
37
Reducing Panel Response Error
43
EXPECTATION ERRORS
POSITIONAL ERRORS .
STIMULUS ERRORS .. .
CONTRAST ERRORS ..
43
44
45
45
Collecting and Analyzing Sensory Data
47
6.1
MEASUREMENT SCALES
6.1.1
Nominal Scales
6.1.2
Ordinal Scales
6.1.3
Interval Scales
6.1.4
Ratio Scales ...
47
48
48
49
51
6.2
STATISTICAL ANALYSIS
52
6.3
STATISTICAL TESTS .........................
6.3.1
Statistical Tests for Scalar Data
54
54
6.4
EXPERIMENTAL DESIGN
6.4.1
Random ization
6.4.2
Blocking ....
6.4.3
Replication ..
56
57
57
58
4.1
4.2
4.3
4.4
Chapter 5
5.1
5.2
5.3
5.4
Chapter 6
iv
38
39
41
Sensory Tests: Descriptions and Applications
59
7.1
CONSUMER-ORIENTED TESTS
Preference Tests
7.1.1
Acceptance Tests
7.1.2
7.1.3
Hedonic Tests ..
60
60
63
66
7.2
PRODUCT-ORIENTED TESTS
7.2.1
Difference Tests ..............
Ranking for Intensity Tests
7.2.2
Scoring for Intensity Tests
7.2.3
Descriptive Tests ............
7.2.4
.79
.79
.86
.90
104
Planning a Sensory Experiment
105
Chapter 7
Chapter 8
107
APPENDICES
Appendix 1
Appendix 2
Appendix 3
Appendix 4
Appendix 5
Appendix 6
Appendix 7
Basic Taste Recognition Test........
Basic Odour Recognition Test ....
Training and Monitoring a Bean
Texture Panel ................................
Techniques for Evaluating Textural
Characteristics of Cooked Beans ..
Food References Used for Bean
Texture Panels ..............................
Line Scale Ballot Used for Bean
Texture Panels ..............................
Statistical Tables............................
109
111
114
116
117
118
119
REFERENCES
137
GLOSSARY
145
INDEX
157
v
LIST OF FIGURES
Panel Discussion Area with Panel Booth
Constructed Along One Wall................
15
Panel Booths with Individual Sections
for Each Panelist...............................
16
Figure 3
Disposable Sample Containers
21
Figure 4
Reusable Sample Containers
22
Figure 5
Typical Sample Tray Set-up for
Presentation to a Panelist........
24
Plan of a Simple Sensory Testing Laboratory Located
at INCAP, Guatemala.................................................
27
Figure 7
Examples of Commonly Used Sensory Scales
50
Figure 8
Ballot for Bean Puree Paired-Preference Test
63
Figure 9
Ballot for Bean Texture Acceptability
Ranking Test......................................
64
Ballot for Bean Varieties Hedonic Test
Using a 9-Point Scale.........................
71
Ballot for Bean Storage Pretreatment
Triangle Test...................................
85
Figure 12
Btillot for Sccdcoat Toughness Ranking Test
88
Figure 13
Ballot for Bean Hardness Scoring Test
Using a Line Scale ............................
92
Figure 1
Figure 2
Figure 6
Figure 10
Figure 11
vi
LIST OF TABLES
Table 1
Tabulated Ranking for Acceptance Test Data
67
Table 2
Tabulated Category Scores for the Hedonic Test
73
Table 3
ANOVA Table for the Hedonic Test
75
Table 4
Six Possible Serving Orders for a Triangle Test
82
Table 5
Tabulated Triangle Test Data
84
Table 6
Tabulated Ranking for Intensity Test Data
89
Table 7
Tabulated Scoring for Hardness Test Data
93
Table 8
ANOVA Table I Scoring for Hardness Test
98
Table 9
Data Matrix of Treatment Totals
for Each Panelist ......................
98
ANOVA Table II Scoring for Hardness Test
100
Table 10
LIST OF STATISTICAL TABLES
Appendix Table 7.1
Random Numbers Table
121
Appendix Table 7.2
Two-Tailed Binomial Test
123
Appendix Table 7.3
Critical Absolute Rank Sum Differences for
"All Treatment" Comparisons at 5% Level
of Significance............................................
124
Appendix Table 7.4
Critical Absolute Rank Sum Differences for
"All Treatment" Comparisons at 1% Level
of Significance............................................
125
vii
Appendix Table 7.5
F Distribution at 5% Level of Significance
126
Appendix Table 7.6
F Distribution at 1% Level of Significance
128
Appendix Table 7.7
Critical Values (Q Values) for Duncan’s New
Multiple Range Test at 5% Level of Significance
130
Appendix Table 7.8
Critical Values (Q Values) for Duncan’s New
Multiple Range Test at 1% Level of Significance
132
Appendix Table 7.9
One-Tailed Binomial Test
134
Appendix Table 7.10
Percentage Points of the Studentized Range
Upper 5% Points.......................................
135
Appendix Table 7.11
Percentage Points of the Studentized Range
Upper 1% Points.......................................
136
viii
Foreword
This manual is intended to provide a basic technical guide to
methods of sensory evaluation. It has been compiled particularly
with the needs of scientists in developing countries in mind. They,
unlike their counterparts in industrialized countries, often lack
adequate facilities and access to information sources.
The selection of materials included in this guide has been
influenced by the experience of the authors in setting up and
implementing sensory evaluation testing at the Institute of
Nutrition of Central America and Panama (INCAP) in Guatemala.
This experience was supported, in part, by the International
Development Research Centre (IDRC) through a research project
on beans that was intended to address problems of storage
hardening, lengthy preparation time, and nutritional availability.
The authors are to be congratulated on the production of a
comprehensive, practical guide.
In supporting food- and nutrition-related research, IDRC gives
ix
a high priority to ensuring that any new or modified products and
processes take full account of the likes, dislikes, and preferences of
the target consumer groups, and their acceptability requirements.
The objective is to help maximize the likelihood of achieving a
positive effect, particularly on disadvantaged producers,
processors, and consumers.
It is hoped that this manual will be usful to a wide variety of
readers, including researchers, students, government control
agencies, and others dealing with issues of more efficient and
effective food production and use within the context of clearly
identified consumer preferences and requirements.
Geoffrey Hawtin
Director
Agriculture, Food and Nutrition Sciences Division
IDRC
I
X
1
Preface
i
J
(
This manual arose from the need to provide guidelines for
sensory testing of basic agricultural products in laboratories where
personnel have minimal or no training in sensory analysis. It is the
outcome of a collaborative project between the Department of
Foods and Nutrition, University of Manitoba and the Institute of
Nutrition of Central America and Panama (INCAP). Included are
discussions of sensory analysis principles, descriptions of sensory
testing facilities and procedures, and examples of statistical
treatment of sensory test data. Examples presented have been
drawn from studies of the sensory characteristics and acceptability
of black beans. These studies were conducted as part of a bean
research network, funded by the International Development
Research Centre (1DRC) to increase the availability, consumption
and nutritive value of beans, an important staple food in Latin
America. Principles discussed, however, apply to the evaluation of
many other types of food and the methods described can be used to
measure and compare the sensory characteristics of both
agricultural commodities and processed foods.
I
2
This publication has been designed to provide an introduction to
sensory methods. For more thorough discussions of sensory
techniques the reader is referred to recent books by Meilgaard et al.
(1987), Jellinek (1985), Stone and Sidel (1985) and Piggott (1984),
to the ASTM publications STP 758 (1981), STP 682 (1979), STP
434 (1968) and STP 433 (1968), and to the classical work on
sensory analysis by Amerine et al (1965). The concise and widely
used Laboratory Methods for Sensory Evaluation of Foods
(Larmond, 1977) is also highly recommended. Statistical methods
for sensory data analysis have been explained in detail in books by
O’Mahony (1986) and Gacula and Singh (1984). Basic statistical
principles and methods are provided in many statistics books such
as those by Snedecor and Cochran (1980), and Steel and Torrie
(1980).
The support of this project provided by the International
Development Research Centre, Ottawa, by the Institute of Nutrition
of Central America and Panama, Guatemala City, and by the
Department of Foods and Nutrition of the University of Manitoba,
Winnipeg, is gratefully acknowledged. Thanks are due to many
individuals in each of these institutions for their encouragement
and assistance at each stage in the preparation of this work. The
authors are particularly indebted to the staff and students at the
Institute of Nutrition, and the University of Manitoba, who served
as panelists during the sensory experiments used as examples in
this book. Valuable editorial suggestions were made by Linda
Malcolmson and Marion Vaisey-Genser of the University of
Manitoba, by Gabriella Mahecha of the National University of
Colombia, Bogota, and by Dorien van Herpen of the Centro
International de
Agricultura Tropical
(CIAT),
Cali,
Colombia. The assistance of the Statistical Advisory Service of the
University of Manitoba is also greatly appreciated. Special thanks
are expressed to Angela Dupuis, Bill Lim, Derrick Coupland, and
3
Horst Weiss for typing, designing and illustrating the manuscript.
Thanks are also extended to the peer reviewers of the manuscript
for their useful suggestions. Support and encouragement were
provided by many other colleagues and friends, who cannot be
mentioned by name, but whose contributions are remembered with
gratitude by the authors.
Beverly Watts
Gladys Ylimaki
Lois Jeffery
Luis G. Elias
4
Chapter 1
l
Using Product-oriented and
Consumer-oriented Testing
Consumers’ sensory impressions of food begin in the
marketplace where visual, odour and tactile senses, and perhaps
taste are used in food selection. During food purchasing,
preparation and consumption, the product cost, packaging,
uncooked and cooked appearance, and ease of preparation
influence consumers’ total impression of a food. However, sensory
factors are the major determinant of the consumer’s subsequent
purchasing behaviour.
Information on consumer likes and dislikes, preferences, and
requirements for acceptability can be obtained using
consumer-oriented testing methods and untrained sensory panels.
Information on the specific sensory characteristics of a food must
be obtained by using product-oriented tests. The development of
new food products or the reformulation of existing products, the
identification of changes caused by processing methods, by storage
or by the use of new ingredients, and the maintenance of quality
8
control standards all require the identification and measurement of
sensory properties. This type of product-oriented quantitative
information is obtained in the laboratory using trained sensory
panels. When food formulas are being altered or new formulas
being developed, product-oriented testing usually precedes
consumer testing.
t
1.1
CONSUMER-ORIENTED TESTING
In true consumer testing a large random sample of people,
representative of the target population of potential users, is selected
to obtain information on consumers’ attitudes or preferences.
Consumer panelists are not trained or chosen for their sensory
acuity, but should be users of the product. For this type of testing
100 to 500 people are usually questioned or interviewed and the
results utilized to predict the attitudes of the target population.
Interviews or tests may be conducted at a central location such as a
market, school, shopping mall, or community centre, or may take
place in consumers’ homes. Because a true consumer test requires
selection of a panel representative of the target population, it is
both costly and time consuming. Therefore untrained in-house
consumer panels are commonly used to provide initial information
on product acceptability and often are conducted prior to true
consumer tests. In-house panels are much easier to conduct than
true consumer tests and allow for more control of testing variables
and conditions. In-house panels are, however, meant to augment,
not replace, true consumer tests.
In-house consumer panels (pilot consumer panels) usually
consist of 30 to 50 untrained panelists selected from personnel
within the organization where the product development or research
(
9
is being conducted. A group of panelists who are similar to the
target population of consumers who use the product should be
chosen. It is advantageous to use as large a panel as possible. This
type of panel can indicate the relative acceptability of products, and
can identify product defects. Results from in-house consumer
testing should not be used to predict product performance in the
marketplace, however, because in-house panels may not be
representative of the actual consuming population.
1.2
PRODUCT-ORIENTED TESTING
Product-oriented testing uses small trained panels that function
as testing instruments. Trained panels are used to identify
differences among similar food products or to measure the
intensities of flavour (odour and taste), texture or appearance
characteristics. These panels usually consist of 5-15 panelists who
have been selected for their sensory acuity and have been specially
trained for the task to be done. Trained panelists should not be used
to assess food acceptability. Their special training makes them
more sensitive to small differences than the average consumer and
teaches them to set aside personal likes or dislikes when measuring
sensory parameters.
■
1
I
■
♦
Chapter 2
Designing Sensory Testing Facilities
Sensory testing does not require elaborate facilities but some
basic requirements must be met if tests are to be conducted
efficiently and results are to be reliable. Although permanent
facilities, specially designed for sensory testing, will provide the
best testing environment, existing laboratory space can be adapted
for sensory use. The basic requirements for all sensory testing
facilities are (1) a food preparation area, (2) a separate panel
discussion area, (3) a quiet panel booth area, (4) a desk or office
area for the panel leader, and (5) supplies for preparing and serving
samples.
2.1
PERMANENT SENSORY FACILITIES
The design of permanent sensory testing facilities and
illustrations of possible layouts for sensory laboratories have been
12
presented in books by Jellinek (1985), Larmond (1977), Stone and
Sidel (1985) and ASTM publication STP 913 (1986). The types of
tests to be conducted, the amount of testing to be done, the space
and resources available, will be deciding factors in the design of
the laboratory.
Throughout the sensory area, walls should be painted in neutral
colours. Odour-free surface materials should be used in
construction of floors and counter tops. Some woods, rugs and
plastics emit odours which interfere with the sensory evaluations,
and should therefore be avoided.
2.1.1
Food Preparation Area
The area for food preparation should contain counters, sinks,
cooking and refrigeration equipment and storage space. The area
should be well lit and ventilated.
Counters. Sufficient counter area is needed to provide working
space for food preparation, and to hold prepared trays of samples
before they are given to the panelists. A counter height of
approximately 90 cm (36 inches) is comfortable for working.
Standard counter depth is approximately 60 cm (or 24 inches).
Sinks. At least two sinks with hot and cold running water
should be provided. It is also useful to have a source of distilled
water in the sensory laboratory. If tap water imparts odours or
flavours, distilled water should be used for panelists’ rinse water,
cooking and rinsing dishes.
f
13
Cooking equipment. Gas or electric stoves or separate heating
elements and ovens should be provided. Microwave ovens may
also be a useful addition to the food preparation area.
Refrigeration equipment. Refrigerated storage is essential for
keeping perishable foods and may be needed to chill samples to a
constant low temperature before serving. A separate freezer can be
useful for long term storage of ingredients, for storage of reference
samples and to enable foods prepared at different times to be stored
and evaluated together.
Storage space. Cupboards or closed shelves for dish and
supply storage should be constructed under the working counters
and also over the pass-through openings to the panel area. An open
shelf over the pass-through area is useful for holding prepared trays
during panel set up. Drawers directly under the counters are
convenient for storing napkins, pencils, plastic spoons and forks
and similar panel supplies.
Ventilation. Ventilation hoods with exhaust fans should be
installed over the stoves to reduce cooking odours in the
preparation area and to prevent spreading of these odours to the
panel room.
2.1.2
Panel Discussion Area
For product-oriented testing it is necessary to have a room
where the panelists can meet with the panel leader for instruction,
training and discussion. This discussion area should be completely
separate from the food preparation area so that noise and cooking
14
odours do not interfere with the panelists’ tasks. It should be
located so that there are no interruptions from other laboratory
personnel. A comfortable well lit area, with a large table and chairs
or stools to seat at least 10 people, is ideal. A large chalkboard, flip
chart, or white board should be located where it can be easily seen
by the panelists around the table. A bulletin board located close to
the entrance allows posting of notices and information about
panelists’ performance. An example of a panel discussion area is
shown in Figure 1.
2.1.3
Panel Booth Area
The booth area, like the discussion area, should be completely
separate from the food preparation area. Although it is preferable to
have a self-contained panel booth room, areas can be combined by
having the booths constructed along one wall of the group
discussion room, with no dividing wall between the booth and
discussion areas, as shown in Figure 1. However, group discussions
cannot then be held simultaneously with individual tasting
sessions. This arrangement could create a problem if several
sensory tasks are under way at one time.
The panel booth area should contain individual compartments
where panelists can assess samples without influence by other
panel members (Figure 2). This area may contain as few as 4
individual sections but 5 to 10 are most common. Each booth
should be equipped with a counter, a stool or chair, a pass-through
opening to the food preparation area and individual lighting and
electrical outlets. While sinks in panel booths may appear useful
for expectoration, they can cause odour and sanitation problems
and are not recommended.
15
Figure 1 Panel discussion area with panel booth constructed along one wall
16
Figure 2 Panel booths with individual sections for each panelist
I
17
It is useful to have the entrance to the panel area within partial
view of the food preparation facilities. The panel leader can then
see when panelists arrive and can supervise activities in both the
food preparation and panel rooms.
Panel booths. Panel booths can be constructed with permanent
dividers or can consist of a countertop with movable partitions.
Each booth should be approximately 60 cm (24”) in depth and be a
minimum of 60 cm (24") in width, but the preferred width is 76-86
cm (30-34"). The booth counter should be the same height as the
counter on the food preparation side of the pass-through to allow
sample trays to be passed from one side to the other with ease. This
may be desk height, 76 cm (30") or counter height, approximately
90 cm (36"). Counter height is usually more convenient and useful
for the food preparation area. Partitions between the booths should
be at least 90 cm (36") high and should extend approximately 30
cm (12") beyond the edge of the countertop to provide privacy for
each panelist.
Chairs or stools. Chairs must be the appropriate height so that
panelists can sit comfortably at the 76 or 90 cm (30 or 36")
counter. Adequate space must be provided from the edge of the
counter to the back wall of the booth area to allow chairs to be
moved back and forth, and panelists to enter and leave while others
are doing evaluations. A minimum distance of 90 cm (36") is
required.
Pass-throughs. Each booth should have a pass-through from
the food preparation area to allow samples and trays to be passed to
panelists directly. The pass-through opening should be
approximately 40 cm (16") wide, 30 cm (12") high, and should be
flush with the counter top. The opening can be fixed with a sliding,
18
hinged or flip-up door. Sliding doors must be well fitted or they
may stick and cause problems. Hinged or flip-up doors require a lot
of clear counter space to work properly.
Lighting and electrical outlets. Each booth should have
individual overhead lighting so that the light distribution is uniform
from one booth to another. Incandescent or fluorescent lighting
may be used. Incandescent lighting offers a range of illumination
but is more costly to install and maintain than fluorescent lighting.
Fluorescent lights can be obtained in cool white, warm white or
simulated daylight. Day light tubes are recommended for food
testing. Lights of various colours such as red and yellow should be
installed, in addition to the conventional white lights. These can be
used to mask colour differences between food samples. Flood
lights with removable plastic coloured filters provide an
economical means of controlling light colour. Each booth should
have an electrical outlet so that warming trays can be used.
Ventilation. The panel room should be adequately ventilated
and maintained at a comfortable temperature and humidity. The
ventilation system should not draw in odours from the cooking
area. If the building in which the sensory facilities arc being
installed has air conditioning, then positive air pressure may be
maintained in the panel booth area to prevent infiltration of
external odours.
f
2.1.4
Office Area
In addition to the space needed for the actual sensory testing, a
place where the panel leader can prepare ballots and reports,
analyze data and store results is required. This area should be
/
19
equipped with a desk, a filing cabinet, and either a statistical
calculator or a computer equipped with a statistical program for
data analysis.
2.1.5
Supplies for Sensory Testing
The sensory areas should be equipped with utensils for food
preparation and with equipment and small containers for serving
samples to the panelists. All utensils should be made of materials
that will not transfer odours or flavours to the foods being prepared
or sampled. Food preparation and serving equipment, utensils and
glassware for the sensory testing area should be purchased new and
used exclusively for sensory testing. Food items, sample containers
(particularly the disposable ones), rinse cups and utensils, should
be purchased in large quantities, sufficient to last throughout an
entire study.
Utensils for food preparation. An accurate balance or scale,
graduated cylinders, pipettes, volumetric flasks and glass beakers
of various sizes will be needed to make precise measurements
during food preparation and sampling. Glass (Le. Pyrex) or
glass-ceramic (i.e. Corningware) cooking pots should be selected
rather than metal cookware because glass and glass-ceramic
containers are less likely to impart flavours or odours to the foods
cooked in them. If only metal is available, then stainless steel is a
better choice than aluminum, tin or cast iron cookware.
Thermometers and standard kitchen utensils such as sieves and
strainers, can openers, knives, forks, spoons, bowls, pot holders
and covered storage containers will also be needed.
20
Sample containers. Sample containers should be chosen
according to the sample size and characteristics. The size of the
containers will vary with the type of product being tested and with
the amount of sample to be presented. Disposable paper, plastic or
styrofoam containers of 30-60 mL (1-2 oz) size with lids
(Figure 3), disposable pctri-platcs and paper plates are convenient
but may prove costly. Reusable containers such as glasses, shot
glasses, glass egg cups, small beakers, glass custard cups, bottles,
glass plates or petri-plates (Figure 4) and glass jars are suitable
alternatives. Lids or covers of some sort are necessary to protect
the food samples from drying out or changing in temperature or
appearance, and to prevent dust or dirt from contaminating the
samples. Lids arc particularly important when odours of the food
samples arc being evaluated. Lids allow the volatiles Irom the
sample to build up in the container so that the panelist receives the
full impact of the odour when bringing the sample container to the
/
nose and lifting the lid.
When purchasing sample containers it is important to check that
the containers do not have any odours oi their own which may
interfere with the evaluation of the food products. Enough
containers of one size and shape must be purchased to ensure that
identical containers can be used for all samples served during one
study.
Trays. Plastic or metal trays, to hold the samples to be served
to each panelist, should be provided. Individual electric warming
trays for each booth arc recommended for samples served warm.
Placing samples in a water bath on the warming trays may
distribute the heat more evenly than placing samples directly on the
trays. Alternatively, samples may be kept warm in a thermos or
warming oven in the preparation area until just before serving. In
all cases, sample containers that will not melt or allow water into
I
21
Figure 3 Disposable sample containers
22
/
Figure 4
Reusable sample containers
23
the samples, are required. Styrofoam containers with lids provide
an inexpensive means of keeping samples warm for short periods
of time.
Additional supplies. Plastic spoons, forks and knives, napkins,
disposable or glass cups for water and expectoration, and large jugs
or pitchers, preferably glass, for drinking water will also be needed.
A typical sample tray set-up, for presentation to a panelist, is
shown in Figure 5. Odourless dishwashing detergent is suggested
for washing equipment.
2.2
TEMPORARY SENSORY FACILITIES
When an area specifically designed for sensory testing is not
available, or when panels, such as consumer panels, are conducted
away from the permanent facility, a temporary area can be arranged
to satisfy the basic requirements for sensory testing.
2.2.1
Food Preparation Area
Temporary cooking facilities can be set up in a laboratory using
hotplates, and styrofoam containers can be used to keep food warm
for short periods. Prepared trays can be set out on carts when
counter space is limited.
24
I
Figure 5
Typical sample tray set-up for presentation to a panelist
25
2.2.2
a
Panel Area
Samples can be presented for evaluation in any separate area
where distractions, noise and odours can be kept to a minimum. A
lunch or coffee room which is not in use at the times when sensory
tests are to be carried out might serve adequately if food odours
have cleared. To provide some privacy for the panelists, and to
minimize distractions, portable partitions of light weight wood or
heavy cardboard can be constructed to sit on table tops between
panelists.
2.2.3
Desk Area
The panel leader will need space for preparing ballots, planning
sensory tests, and analyzing data, and will need access to a
calculator with statistical capabilities.
2.2.4
Supplies for Sensory Testing
The same supplies will be needed as were outlined for the
permanent facility.
26
DESIGN OF A SIMPLE SENSORY TESTING
LABORATORY
2.3
At INCAP in Guatemala City, a sensory laboratory containing
panel booths and a discussion area was built adjacent to an existing
kitchen facility (Figure 6). This food preparation area was already
well equipped with stoves, sinks, refrigerators, storage cupboards
and counter space.
In the newly designed sensory facility, panel booths are
accessible from the kitchen area via pass-throughs with horizontal
sliding doors. The five panel booths are open from the back to the
group discussion area which is equipped with a large table, and
with stools to seat 12-15 people. Each booth has individual light
fixtures and an electrical outlet. The divisions between the booths
arc hinged so that they can be folded to one side if clear counter
space is needed on some occasions.
\
Although a separate office for the panel leader was not
available, a desk placed in the food preparation area provides space
for preparing ballots and analyzing data.
The following items were acquired to equip the sensory
laboratory at INCAP:
•
•
•
•
•
1 analytical balance
glassware (graduated cylinders and beakers of various sizes)
5 electric warming trays with adjustable thermostats (1 per
panel booth)
8 - 3 L glass cooking pots with lids
10 - 300 mL plastic storage containers with lids
27
If
DISCUSSION
AREA
(
)
o ^4
o o
BOOTH
1
oo
oo
r I
-r |
o
o
AREA
■
PREPARATION
I
-r
I
-T-
oo
oo
AREA
O
I
OFFICE
AREA
oo
oo
oo
oo
0
I
2m
Scale
Figure 6 Plan of a simple sensory testing laboratory
located at INCAP, Guatemala
28
20 -15 cm diameter styrofoam containers with lids
(tortilla holders)
15 white plastic serving trays
6 large water jugs
48 - 50 mL red sample glasses with tin foil lids
disposable 75 mL plastic cups for water
and for expectoration
disposable 30 mL plastic sample containers with lids
disposable 30 mL styrofoam sample cups with lids
disposable white plastic teaspoons
paper napkins
pot holders, tea towels, spoons, forks, knives, strainers,
paper towels, detergent
statistical calculator
4- Chapter 3
Establishing Sensory Panels
The testing instrument for sensory analysis is the panel of
human judges who have been recruited and trained to carry out
specific tasks of sensory evaluation. Recruiting panelists, training
them, monitoring their performance, providing leadership and
motivation is the job of the panel leader. Thorough preparation and
efficient direction of the panel by the leader are essential if the
panel is to function effectively.
3.1
RECRUITING PANELISTS
Panelists for both trained panels and untrained in-house panels
can usually be drawn from the personnel of the institution or
organization where the research is being conducted. The majority
of the people within an organization are potential panelists. They
30
will usually be interested in participation if they feel that their
contribution is important.
To help with panelist recruitment, all potential panelists should
be asked to complete questionnaires giving their food likes and
dislikes, indicating their level of interest in the project to be carried
out, listing any food restrictions or allergies they may have and
giving times when they would be available for panels. This
information will help the panel leader to select those individuals
appropriate for the study. In a company or institution where
sensory tests arc conducted on a regular basis, it is useful to keep a
file with information on all potential panelists. Records should also
be kept on each panelist who participates in any sensory panel.
3.2
ORIENTING PANELISTS
Potential panelists should be invited to the sensory panel area, in
groups of no more than 10 at a lime, to allow the panel leader to
explain the importance of sensory testing, show the panelists the
testing facilities, and answer questions that may arise. Individuals
participating only in in-house acceptability panels (untrained
panels) do not need to be given any subsequent training. However,
it is useful to demonstrate the way in which the ballots should be
marked, using enlarged ballots shown on an overhead projector or
a blackboard. Discussing the actual food to be tested should be
avoided. Explaining the test method and procedure will reduce
confusion and make it easier for panelists to complete the task. It is
important that all panelists understand the procedures and score
cards so they may complete the test in a similar manner.
31
Panelists should be advised to avoid strong odourous materials,
such as soaps, lotions and perfumes prior to participating on panels
and to avoid eating, drinking or smoking at least 30 minutes prior
to a sensory test.
3.3
SCREENING PANEUSTS FOR TRAINED
PANELS
Panelists who agree to serve on trained panels should be
screened for "normal" sensory acuity. This can be done by asking
panelists to identify basic tastes and common odours. Instructions
for conducting taste and odour identification tests are given in
Appendices 1 and 2.
Panelists’ sensitivity, that is their ability to discriminate between
levels of a particular sensory characteristic, should also be tested.
Triangle tests, using food samples or solutions that are identical for
all but the level of one flavour or texture characteristic, are often
used to test panelists’ discrimination skills. People with a poor
sense of smell or taste, or who are insensitive to differences in
flavour or texture intensities, can be identified through these
screening processes. For those who ultimately will serve on a
trained panel, the screening process provides some preliminary
sensory experience.
After the initial screening, panelists should be tested for their
ability to discriminate using samples very similar or identical to
those to be studied. Some panelists are excellent discriminators for
one type of food product, but are poor discriminators for others.
32
Locating panelists sensitive to differences in the test food is
important.
If 20-25 people can be screened, it should be possible to select
for training, a group of 12-14 people who have demonstrated
superior performance during screening sessions. Panelists chosen
should also be interested in the project, and able to participate on a
long term basis. Panel training takes approximately 1/2 hour a day,
usually 2-4 times per week. Panel training should begin with a
larger group of people than is needed for the final trained panel.
Some panelists will almost certainly drop out due to illness or
job-related priorities. The final trained panel should include at least
8 people with good discriminatory ability for the task to be done.
3.4
TRAINING PANELISTS
The performance of individual panelists, and of the panel as a
whole, can be improved through suitable training exercises.
Training should be designed to help panelists make valid, reliable
judgements that are independent of personal preferences. A
discussion of results, directed by the panel leader, should
accompany each training exercise, so that the panelists as a group
can develop consistent methods of evaluation. Training a panel for
difference or ranking tests can usually be done in a few sessions.
Training for quantitative analysis may require ten to twelve
sessions, or even more if a large number of sensory characteristics
are to be evaluated.
Final training should be conducted with food products similar to
those that will be used during actual testing. Panelists should
33
become familiar with the range of characteristic intensities that will
be encountered during the study. During training the best
procedures for preparing and presenting the samples can be
established and the final score card or ballot can be designed.
Discussions should be held frequently, between the panelists
and panel leader, to ensure that all panelists understand the task,
ballot and terminology, and can distinguish the characteristics
being studied. By providing precise definitions and descriptions for
the evaluation of each characteristic, and by supplying food
samples to demonstrate each characteristic wherever possible,
consistent panelist response and agreement among panelists can be
developed.
Panelists who are unsuccessful at one type of sensory task may
do well on another. Their participation on subsequent panels should
be encouraged, and appreciation for their work should be expressed
by the panel leader.
3.5
MONITORING PANELISTS’
PERFORMANCE
Panelists’ performance must be monitored during training to
determine the progress of the training. Subsequent training should
concentrate on the samples and sample characteristics that panelists
have difficulty identifying and evaluating. Training is completed
when panelists are comfortable with the evaluation procedure, can
discriminate among different samples repeatedly and can produce
34
reproducible results. Superior panelists can then be identified to
continue throughout the sensory study.
The panel leader monitors performance by evaluating the ability
of the panel as a whole, and of the individual panelists, to
discriminate differences among the samples being tested and to
reproduce results consistently. For both types of evaluation, a set of
different samples, which the panel leader knows to be different,
must be evaluated by each panelist repeatedly on several occasions
to provide the necessary data. Statistical analysis (analysis of
variance - ANOVA) is used to assess the results. The panel data is
analyzed to identify significant variation among panelists and
among samples. Significant differences among panelists, although
not unexpected, may be reduced with further training. Lack of
significant differences among samples indicates the need for
further training, if the panel leader knows that differences do in fact
exist.
Individual panelist’s results can also be analyzed. Panelists who
are able to distinguish significant differences among the samples
with small error mean squares in the analysis should be retained on
the panel. If none of the panelists find significant differences
among samples for a particular characteristic, additional training
for that characteristic is indicated. Monitoring panelist performance
during training is described in more detail in Appendix 3.
Panelist performance can also be monitored during the sensory
study by comparing replicate judgements. This ensures that
panelists continue to perform in a reliable, consistent manner and
will indicate when additional re-training may be required or when
panelists need further motivation.
35
3.6
MOTIVATING PANELISTS
Panelists who are interested in sensory evaluation, the products
under evaluation and the outcome of the study will be motivated to
perform better than uninterested panelists. It is important to
maintain this interest and motivation throughout the study to ensure
and encourage optimum panelist performance.
Feedback about their performance from day to day will provide
much of the motivation for panelists, particularly during training. If
there is not sufficient time during the panel sessions to discuss the
previous day’s results, the data can be posted on a wall chart for the
panelists to see at their convenience. However, it is more beneficial
if the panel leader personally discusses the results with the
panelists, individually or as a group. Posted results can be missed
or misinterpreted by the panelists. In addition, a small treat or
refreshment (Le. candies, chocolates, cookies, fruit, nuts, juice,
cheese, crackers) at the end of each day’s panel session is
commonly used as a reward. At the end of a long series ol panels a
larger reward such as a small party, luncheon or small gift will let
each panelist know that their contribution to the study has been
appreciated.
4- Chapter 4
Conducting Sensory Tests
Sensory tests will produce reliable results only when good
experimental control is exercised at each step of the testing process.
Careful planning and thorough standardization of all procedures
should be done before the actual testing begins. Particular attention
should be given to techniques used for sampling food materials, for
preparing and presenting samples to the panel, and for using
reference and control samples. These techniques are discussed in
the following sections of the manual.
4.1
SAMPLING FOOD FOR SENSORY TESTING
All foods presented to the panelists for testing must, of course,
be safe to eat. Panelists should not be asked to taste or eat any food
that has become moldy, or has been treated in a way that might
cause microbiological or chemical contamination. If a food, or an
38
ingredient of the food, has been treated or stored in a way that may
make it unsafe to eat, then only the odour and appearance attributes
of the food can be evaluated.
When batches of food are being sampled for sensory testing
samples taken should be representative of the total batch. If the
portions ultimately served to the panelists are not representative of
the food as a whole, results will not be valid. For a commodity
such as beans, the lot to be tested should first be thoroughly mixed,
then divided into four parts and a sample from each part extracted.
These four samples should be recombined to form the test sample.
The size of the lest sample should be calculated beforehand, based
on the number of portions that will be required for the panel.
4.2
PREPARING SAMPLES FOR
SENSORY TESTING
Samples for sensory comparison should all be prepared by a
standardized method to eliminate the possibility of preparation
effects (unless, of course, preparation method is a variable of
interest). Preparation steps should be standardized during
preliminary testing and clearly documented before sensory testing
is begun, to ensure uniformity during each testing period. When
different types of beans, for example, arc to be cooked and
prepared for sensory analysis, factors that need to be controlled
include the ratio of beans to soaking and cooking water, soaking
time, size and dimensions of the cooking container, cooking rate
and time, holding time before serving, and serving temperature. If
samples require different cooking times, starting times can be
staggered so that all samples finish cooking together. If this is not
39
done, variations in holding time may influence sensory assessment.
Holding samples for an extended period of time can drastically
alter their appearance, flavour and texture.
4.3
PRESENTING SAMPLES FOR
SENSORY TESTING
Methods of sample presentation should also be standardized. It
is important that each panelist receive a representative portion of
the test sample. Tortillas, for instance, can be cut into wedges of
uniform size so that each panelist will receive part of both the edge
and the centre of a tortilla. Fluid products should be stirred during
portioning to maintain uniform consistency within the portions.
The end crusts of breads or baked goods and the outer surface of
meat samples may have to be discarded so that each panelist
receives a similar portion. If bread crusts are left on samples, each
panelist should receive a sample with a similar crust covering.
Portions should all be of the same size. When food products consist
of a number of small pieces which may differ from piece to piece,
panelists should receive a portion large enough that they can
evaluate a number of pieces for each characteristic. When beans,
for example, are being tested for firmness, panelists should test 3-4
beans before recording their score for the firmness of the sample.
In general, at least 30 grams (1 oz) of a solid food or 15 mL (0.5
oz) of a beverage should be served (ASTM STP 434, 1968).
Samples should all be presented at the same temperature, and
this should be the temperature at which the food is usually
consumed. Milk should be served at refrigerator temperature, but
bread or cake at room temperature. Some foods require heating to
40
bring out their characteristic odours or flavours. Vegetable oils are
often evaluated for odour after being equilibrated at 50 °C.
Panelists may prefer to evaluate some foods when they are
served with carriers. Crackers, for instance, may be used as carriers
for margarine or peanut butter. Use of carriers can present
problems, however, because the carrier foods have flavour and
texture characteristics of their own which can interfere with
panelists’ evaluation of the main food product.
The food samples being evaluated may be swallowed or
expectorated, however, the panel should be encouraged to develop
a consistent technique. Cups with lids should be provided tor
expectoration.
Room temperature water is often presented to panelists so that
they can rinse their mouths before and between samples. Rinse
water can be swallowed or expectorated. If room temperature water
will not clear the mouth between tastings, warm water, lemon
water, unsalted soda crackers, white bread or apple slices may be
used. Warm water is particularly helpful when fats or oily foods are
being tested. The time between evaluation of each sample may
have to be longer than usual if the products being tested have
strong flavours. It may also be necessary to restrict to two or three
the number of samples presented at one session.
When characteristics other than colour are being evaluated it
may be necessary to mask colour differences; otherwise they may
influence the panelists’ judgements of other characteristics. Red,
blue, green or yellow light, whichever masks the sample
differences most effectively, can be used.
41
4.4
USING REFERENCE SAMPLES
References are often used in sensory testing. These can be
designated reference samples, against which all other samples are
to be compared; or they can be identified samples used to mark the
points on a measurement scale; or they can be hidden references,
coded and served to panelists with the experimental samples in
order to check panelist performance.
When sensory tests arc conducted over several weeks or
months, or when the testing must be done at widely spaced
intervals as is the case when storage effects are being studied, then
it is almost essential to use a designated reference. This can be
selected from among the actual foods or samples that are to be
tested, or can be a food product of a similiar type. When
conducting a storage study the designated reference may be the
control (a sample stored under standard conditions), or may be a
fresh sample. If the purpose of the research is to produce a product
that is an improvement on a marketed product, then the product
being marketed can serve as the reference. When the testing is done
by a trained panel, this panel should evaluate the reference before
the actual testing is begun. Scores which this panel agrees are
appropriate, for each characteristic to be measured, can then be
placed on the ballot to be used during the experiment. Providing a
scored, designated reference to the panelists at each panel session
should help them score the experimental samples more
consistently.
Reference samples which are used to mark points on a scale, or
to calibrate the scale, are often called standards. These references
may be of food similar to that being tested, or may be totally
different. If a number of product characteristics are being
42
evaluated, many references (standards) may be necessary.
Examples of food references used to identify scale endpoints for
cooked bean textural characteristics of hardness, particle size, and
seed coat toughness, are given in Appendix 5.
Hidden references, or blind controls as they are sometimes
called, can be served to the panel at some or all of the panel
sessions, to check on the panelists’ performance. The hidden
reference must be sufficiently similar to the samples being tested
that it cannot be immediately identified as the control sample. It
should be coded in the same way as the experimental samples,
using a different code number each time it is presented to the panel.
If one or several panelists’ scores for the hidden reference vary
unacceptably, these panelists should be given further training or
their scores may have to be excluded from the dataset.
A reference will improve panel consistency only if the reference
itself is consistent. If the reference changes, it will not serve its
intended purpose. Ideally, enough of the product reference should
be obtained initially to serve for the entire experiment, and the
product should be stored so that its sensory qualities do not change
over the testing period. If a "new” reference is introduced part way
through a study, or if the quality of the reference changes, results of
the experiment may be impossible to interpret. If the product
reference is a food that must be freshly prepared for each panel
session, then ingredients and methods of preparation should be well
standardized before the experiment begins.
> Chapter 5
Reducing Panel Response Error
During sensory testing panelists’ responses can be influenced by
psychological factors. If the influence of psychological factors is
not taken into account when an experiment is planned and
conducted, the error introduced can lead to false results.
Psychological factors can be responsible for a number of different
types of error. Errors that result from panelists’ expectations, from
sample positions, and from stimulus and contrast effects will be
discussed in the following sections.
5.1
EXPECTATION ERRORS
Expectation errors can occur when panelists are given too much
information about the nature of the experiment or the types of
samples before tests are conducted. If the panelists expect to find
certain differences among the samples, they will try to find these
44
differences. Panelists should be given only the information that
they need to perform their task, and when the experiment is under
way, they should be discouraged from discussing their judgements
with each other. Those conducting the experiment or whose
knowledge of it leads them to expect particular results, should not
participate in the panel.
Panelists may have other expectations about the test samples.
They may expect that a sample coded as A will be ’’better" than a
sample coded as F or that a sample coded as 1 will have more of a
characteristic than a sample coded as 5. To prevent these
expectation errors, each sample should be coded with a 3-digit
random number (such as 374 or 902). Three digit codes do not
influence panelists’ judgements as do single number or letter codes.
Random number tables, such as the one shown in Appendix 7,
Table 7.1, are useful in choosing random numbers. Starting
anywhere on the table, beginning at a different place each time and
moving in a different direction, you can choose 3-digit numbers
down a column or across a row.
5.2
POSITIONAL ERRORS
The way samples are positioned or ordered for evaluation can
also influence panelists’ judgements. For example, when two
samples are presented, the first sample evaluated is often preferred
or given a higher score. Randomizing the order of sample
presentation so that the samples are presented in different positions
to each panelist can minimize positional errors.
45
5.3
STIMULUS ERRORS
Stimulus errors occur when panelists are influenced by
irrelevant sample differences such as differences in the size, shape
or colour of food samples presented. Greater colour intensity for
example, may lead panelists to score a food higher for flavour
intensity, even when these characteristics are unrelated to each
other. To minimize stimulus errors, the samples presented should
be as similar as possible in all characteristics but the one(s) being
evaluated. Colour differences can be masked by using coloured
lights in the panel booths, as mentioned earlier. Alternately, dark
glasses or blindfolds may be used if appropriate. Evaluating each
characteristic separately, for all samples, will also reduce the error
due to association of characteristics. When evaluating the colour,
texture and flavour of three pudding samples, the stimulus error is
reduced if the colour of all three samples is evaluated, then the
texture of all three samples, and finally the flavour of all three
samples, rather than evaluating the colour, texture and flavour of
the first sample, then the colour, texture and flavour of the second
sample and then the colour, texture and flavour of the third sample.
5.4
CONTRAST ERRORS
Contrast effects between samples can also bias test results.
Panelists who evaluate a sample that is very acceptable before a
less acceptable sample may score the acceptability of the second
sample lower than they would if a less acceptable sample had been
evaluated before it. Similarly, evaluating an unacceptable sample
directly before an acceptable sample may result in amplifying the
acceptability score given to the acceptable sample. When panelists
46
evaluate a mildly flavoured sample after one with an intense
flavour, their response will be influenced by the contrast between
the two samples. If all panelists receive samples in the same order,
contrast effects can have a marked influence on panel data.
Contrast effects cannot be eliminated during sensory testing, but if
each panelist receives samples in a different order, contrast effects
can be balanced for the panel as a whole. Samples can be presented
randomly to each panelist or all possible orders oi the sample set
can be presented. For example, when four samples are presented to
each panelist at one time, four samples can be arranged and
presented in 24 combinations. To ensure that a panelist evaluates
the samples in the order selected for him/her, code numbers should
be written in the appropriate order on the ballot and the panelist
instructed to evaluate samples in the order indicated on the ballot.
If possible, the coded samples on the tray should also be arranged
for presentation to each panelist in the appropriate order, so that the
evaluation can be done from left to right.
4
Chapter 6
Collecting And Analyzing Sensory Data
Sensory data can be in the form of frequencies, rankings, or
quantitative numerical data. The form of the data depends on the
type of measurement scale used for sensory testing. To analyze the
data statistically, methods appropriate for frequency, ranked or
quantitative data must be applied. Types of scales and the statistical
methods appropriate for analysis of the data obtained will be
described briefly in the following section.
6.1
MEASUREMENT SCALES
Measurement scales are used to quantify sensory information.
Scales can be classified according to their type as nominal, ordinal,
interval or ratio scales. Because the type of scale chosen will affect
the type of statistical analysis done, the measurement scale should
be chosen only after careful consideration of the objectives of the
study.
48
6.1.1
Nominal Scales
Nominal scales are the simplest of all scales. In these types of
scales, numbers represent labels or category names and have no
real numerical value. For example, panelists can use a nominal
scale to identify odour characteristics of tomato sauces where 1 =
fruity, 2 = sweet, 3 = spicy, and 4 = pungent. Panelists record the
number of each odour characteristic present in each of the samples
and the panel leader tabulates the frequency of the appearance of
each characteristic for each sample. The products are then
compared by observing the frequency of each odour characteristic
in each sample.
Names only, rather than numbers representing names, can be
used in a nominal scale. Classifications or categories can be given
names and the frequencies within each classification tabulated and
compared. Food samples could be classified as acceptable or
unacceptable and the number of panelists placing a sample in the
unacceptable category compared to the number of panelists
considering it acceptable.
6.1.2
Ordinal Scales
In ordinal scales the numbers represent ranks. Samples are
ranked in order of magnitude. Ranks do not indicate the size of the
difference between samples. Ranking is used for both
consumer-oriented and product-oriented testing. In consumer
panels, samples arc ranked on the basis of preference or
acceptability. Biscuits made from three different formulations
could be ranked for preference with the sample ranked 1 as the
most preferred and the sample ranked 3 as the least preferred. In
49
product-oriented testing the intensities of a particular product
characteristic are ranked. A series of five chicken soup samples
could be ranked for saltiness, with the sample ranked 1 the most
salty soup and the sample ranked 5 the least salty soup.
6.13
Interval Scales
Interval scales allow samples to be ordered according to the
magnitude of a single product characteristic or according to
acceptability or preference. The degree of difference between
samples is indicated when interval scales are used. If chicken soups
were evaluated using an interval scale, not only would the most
salty sample be identified, but the number of intervals separating
the most salty soup from the least salty soup would be known. To
provide a measurement of the degree of difference between
samples the length of the intervals on the scale must be equal.
Category scales and line scales (Figure 7) are two types of
sensory scales, commonly treated as interval scales. A category
scale is one that is divided into intervals or categories of equal size.
The categories are labelled with descriptive terms and/or numbers.
All the categories may be labelled or only a few, such as the
endpoints and/or midpoint of the scale. The total number of
categories used varies, however, 5-9 categories are common.
Pictures or diagrams illustrating the categories on the scale are
particularly useful if panelists have trouble reading or
understanding the language of the scale (Figure 7). Line scales,
with endpoints and/or midpoint of the scale labelled, are commonly
used to quantify characteristics. The length of the line scale can
vary, but 15 cm is often used. Panelists may not always use
category and line scales as equal interval scales. This is particularly
50
a) 5-Point Category Scale for the Intensity of a
Characteristic
CODE
trace
slightly intense
moderately intense
very intense
extremely intense
b) Line Scale for the Intensity of a Characteristic
J
strong
L
weak
c) Smiley Scale for Degree of Liking
Dislike
a lot
Dislike
a little
o o
0 0
o
0 0
o
Neither like
nor dislike
Like
a little
Like
a lot
□
Figure 7
Examples of.Commonly Used Sensory Scales
1
51
true of untrained consumer panelists. When in doubt about the
equality of scale intervals, the panelists’ scores should be converted
to ranks and the category or line scales treated as ordinal scales.
Examples in this manual will, however, be based on the assumption
that equal interval sizes do exist between categories and along the
line scales, and both scales will be considered and analyzed as
interval scales.
Interval scales are used in both consumer-oriented and
product-oriented tests. The degree of liking, the level of preference
or the acceptability of the products are scored in consumer tests.
The intensity of product attributes are scored in product-oriented
testing.
6.1.4
Ratio Scales
Ratio scales are similar to interval scales, except that a true zero
exists. On an interval scale the zero end point is chosen arbitrarily
and docs not necessarily indicate the absence of the characteristic
being measured. On a ratio scale the zero point indicates the
complete absence of the characteristic. If a ratio scale were used to
measure the five chicken soup samples, the number of saltiness
intervals separating the samples would indicate how many times
more salty one sample was than another. If two of the samples, A
and B, were given scores of 3 and 6 respectively for salty flavour
intensity, on a ratio scale, sample B would be twice as salty as
sample A. Ratio scales are seldom used for consumer-oriented
testing, because training is required to use ratios successfully.
a"a
NO T
10 O
52
6.2
STATISTICAL ANALYSIS
Sensory results are analyzed statistically in order to allow the
experimenter to make inferences or draw conclusions about
populations of people or food products on the basis of a sample
drawn from these populations. Prior to conducting the experiment,
assumptions or informed guesses, can be made about the
populations and about the expected results of the experiment.
These assumptions are called hypotheses, and can be stated in two
ways. The assumption that no difference exists between two
samples, or among several samples, is termed the null hypothesis.
This is also the statistical hypothesis which, based on statistical
analysis of experimental results, is accepted or rejected. The other
assumption that can be made is that differences do exist between or
among samples. This is the alternate hypothesis, or what is often
termed the research hypothesis. For example, in an experiment to
determine whether adding salt to cooking water produces softer
beans, the null hypothesis would be that there is no difference in
softness between beans cooked with or without added salt. The
alternate (or research) hypothesis might state that the beans cooked
with salt are softer than those without salt. Using the appropriate
statistical test, it is possible to determine whether the null
hypothesis should be accepted or rejected. If it is accepted, then the
conclusion is that there is no difference in softness between beans
cooked by the two methods according to this test. If it is rejected,
then the conclusion is that the beans cooked with salt are probably
softer.
Results of statistical tests are expressed by giving the probablity
that the outcome could be due to a chance occurrence rather than
being a real difference. If a result occurs 5 times out of 100, due to
chance, then the probability is said to be 0.05. A statistical result is
usually considered significant only if its probability is 0.05 or less.
53
At this level of probability, the null hypothesis will be rejected 5
times out of 100 when in fact it should be accepted. When it is
stated that a difference is significant at the 5 percent level (a
probability of 0.05), it means that 95 times out of 100 the
difference is a real one.
The level of significance to be used in a sensory test should be
decided on before the test is carried out. This is done so that the
results of the test do not influence the decision. Usually levels of
0.05 or 0.01 are employed. Using a level of significance of 0.05
rather than 0.01 makes it more likely that a difference will be found
if one exists (Le. the null hypothesis is more likely to be rejected).
However, it also means that there is a greater probability that the
difference identified is due to chance.
In consumer-oriented testing inferences can be made concerning
a group, such as the intended users of a food product, if the group
or population has been sampled randomly to form the consumer
panel. In product-oriented testing, panelists are not selected
randomly and no inferences can be made about a particular
population of consumers. Inferences can be made, however, about
the characteristics of the population of foods being tested. In both
types of testing the food samples should be chosen at random from
the product lots of interest, if results are to be inferred for the
product as a whole. When sampling cannot be done on a random
basis, care must be taken in generalizing the conclusion of the test
to the wider group or population.
Random sampling of a population requires that every unit of the
population has an equal chance of being selected. Obtaining a truly
random sample of a food product is seldom possible. However, it is
important that the samples to be tested are as representative of the
original batch of food as possible.
54
A sample lot large enough to provide for all components of the
study should be gathered initially. Subsamples from this lot should
be assigned randomly to each experimental treatment or replication
or block. At each stage of the process, subsamples or portions are
randomly chosen.
6.3
STATISTICAL TESTS
Statistical tests are used to analyze data resulting from sensory
studies. Statistical analyses arc used for the following purposes:
1) to test hypotheses.
2) to find out if significant differences exist among samples,
treatments or populations and if these differences are
conditional upon other variables or parameters.
3) to monitor the consistency of trained panelists both during
training and during the actual study.
6.3.1
Statistical Tests for Scalar Data
Data from nominal and ordinal scales are analyzed using
non-paramctric statistical tests while data from interval and ratio
scales are analyzed using parametric statistical tests.
Non-parametric methods are less discriminating than parametric
tests but do not require data to be normally and independently
55
distributed as do parametric tests. Parametric tests also require
interval scales to have intervals or categories that are equal both
psychologically and in size. If this is not true, then the categories
should be treated as nominal data and analyzed by non-parametric
methods. The use of parametric tests versus non-parametric tests
for the analysis of category scale data has been discussed in
numerous books and articles (O’Mahony, 1986, 1982; McPherson
and Randall, 1985; Powers, 1984; Gacula and Singh, 1984; Daget,
1977).
Nominal sensory data is usually analyzed by binomial or
chi-square tests. Ordinal or ranked sensory data is most frequently
analyzed by the Kramer test or the Friedman test. The Kramer test,
however, has recently been found to be inappropriate (Basker,
1988; Joanes, 1985) and is not recommended. The most common
parametric test for interval or ratio scale sensory data is the
Analysis of Variance (ANOVA).
Multiple comparison of means tests are utilized to identify
samples that differ from each other, once the presence of statistical
differences has been confirmed using Analysis of Variance. Many
multiple comparison tests, such as Duncan’s New Multiple Range
Test, Tukey’s Test, the Least Significant Difference (LSD) Test
and Scheffe’s Test, are available. Of these, the LSD test is the most
powerful and most liberal test, followed by Duncan’s, Tukey’s and
Scheffe’s test. Thus, using the LSD test will make it more likely to
find significant differences between two samples. However, it may
also identify differences when none really exist. Scheffe’s test, on
the other hand, is very cautious or conservative and may miss
finding differences when they do exist. Duncan’s and Tukey’s test
are frequently used for sensory data as they are considered neither
too liberal nor too conservative.
56
Multivariate analysis techniques can be used when relation
ships among a number of different measurements or tests are being
investigated. Correlation and Regression Analysis, Discriminant
Analysis, Factor Analysis and Principal Component Analysis are
types of multivariate analysis frequently used in sensory studies.
These analyses require sophisticated statistical treatment, and will
not be discussed in this manual. For further information on the use
of multivariate techniques for sensory analysis data see O’Mahony
(1986), Gacula and Singh (1984), Piggott (1984), Powers (1984,
1981), Moskowitz (1983), Ennis et al. (1982) and Stungis (1976).
Programmable calculators can be used to analyze small data sets
using the statistical tests illustrated in this manual. Computerized
statistical programs or packages are needed to carry out more
complicated statistical analyses.
6.4
EXPERIMENTAL DESIGN
Experimental designs are plans, arrangements, or a sequence of
steps for setting up, carrying out and analyzing the results of an
experiment. An appropriate and efficient experimental design must
be chosen to ensure the reliability of data and test results. The
design is selected based on the objectives of the study, the type of
product under study, the testing procedures and conditions, the
resources available and the type of statistical test to be conducted.
There are many types of experimental designs from simple,
completely randomized designs to more complicated, fractional
factorial designs. Good statistical textbooks and a statistician
57
should be consulted to recommend the simplest, most efficient
design to meet the specific objectives of the study.
Common features of good experimental designs are
randomization, blocking and replication. These concepts are
discussed in the following sections.
6.4.1
Randomization
Randomization is introduced into an experimental design to
minimize the effects of uncontrollable sources of variation or error
and to eliminate bias. Randomization is a procedure for ordering
units or samples such that each unit has an equal chance of being
chosen at each stage of the ordering process. For example, to
randomize the assignment of different cooking treatments to food
samples, one sample is chosen to be cooked by method 1, but all of
the other samples have an equal chance of being cooked by that
same method. Random number tables (Appendix 7, Table 7.1) are
used for randomization in the same manner as was described for
choosing 3-digit random numbers (Section 5.1).
6.4.2
Blocking
Blocking is included in many experimental designs to control
for known sources of variation and to improve efficiency. Blocks
may be growing plots, day effects, panelists, replications or sample
presentation orders; anything that is a known source of error in the
experiment. Experimental units are grouped into blocks. Variation
among the units within a block is likely to be less than the variation
among blocks. Blocking provides a truer measure of pure or
58
experimental error by accounting for the variance due to the
blocked factors and separating it out from the uncontrollable
sources of experimental error. For instance, sensory panelists,
being human, are often a known source of variability in sensory
experiments. By blocking panelists in the experimental design and
data analysis, the variation due to panelists can be removed from
the experimental error and separated out as a panelist effect. Then
the error term used to determine whether there are significant
differences among the samples, will be more indicative of pure
error.
6.4.3
<!
Replication
Replication of an experiment involves repeating the entire
experiment under identical conditions. Replication provides an
estimate of experimental error and improves the reliability and
validity of the test results. Through replication, the consistency of
both the panel and individual panelists can be determined. The
number of replications of an experiment varies and often is
determined by considering time, cost and sample restraints,
however, usually the more replications that are done, the belter the
estimate of experimental error and the more reliable the test results.
> Chapter 7
Sensory Tests: Descriptions and
Applications
Sensory tests can be described or classified in several ways.
Statisticians classify tests as parametric or non-parametric
according to the type of data obtained from the test. Sensory
specialists and food scientists classify tests as consumer-oriented
(affective) or product-oriented (analytical), basing this
classification on the purpose of the test. Tests used to evaluate the
preference for, acceptance of, or degree of liking for food products
are termed consumer-oriented. Tests used to determine differences
among products or to measure sensory characteristics are termed
product-oriented.
60
CONSUMER-ORIENTED TESTS
7.1
Preference, acceptance and hedonic (degree of liking) tests are
consumer-oriented tests. These tests are considered to be consumer
tests since they should be conducted using untrained consumer
panels. Although panelists can be asked to indicate their degree of
liking, preference or acceptance of a product directly, hedonic tests
are often used to measure preference or acceptance indirectly. In
this section preference, acceptance and hedonic tests will be
described using a paired-preference test, an acceptance ranking
scale, and a 9-point hedonic scale as examples.
7.1.1
Preference Tests
Preference tests allow consumers to express a choice between
samples; one sample is preferred and chosen over another or there
is no preference. The paired-preference test is the simplest
preference test but category scales and ranking tests are also often
used to determine preference.
General Instructions for Conducting a
Paired-Preference Test.
Description ofpanelists' task: Panelists are asked which ol two
coded samples they prefer. Panelists are instructed to choose one,
even if both samples seem equal. The option of including a "no
preference" choice or a "dislike both equally" is discussed in Slone
and Sidcl (1985), but is not recommended for panels with less than
50 panelists as it reduces the statistical power of the test (a larger
61
difference in preference is needed in order to obtain statistical
significance).
Presentation of samples: The two samples (A and B) are
presented in identical sample containers coded with 3-digit random
numbers. There are two possible orders of presentation of the
samples; A first, then B (AB) or B first, then A (BA). Each order
should be presented an equal number of times. If the panel included
20 panelists, ten would receive A first and ten, B first. When the
panel is large, the order for each panelist can be selected at random.
Since there is a 50% chance of each panelist receiving either the A
or B sample first, both orders should be presented to approximately
the same number of panelists.
The samples are presented simultaneously in the order selected
for each panelist, so that the panelists can evaluate the samples
from left to right. Retasting of the samples is allowed. An example
of a ballot for the paired-preference test is given in Figure 8. The
order in which the panelists are to evaluate the samples should be
indicated on the ballot.
Analysis of data:
Results are analyzed using a 2-tailed
binomial test. The 2-tailed test is appropriate since either sample
could be preferred; and the direction of the preference cannot be
determined in advance. The number of judges preferring each
sample is totalled and the totals tested for significance using Table
7.2 (Appendix 7). In this table X represents the number of panelists
preferring a sample and n represents the total number of panelists
participating in the test. The table contains 3 decimal probabilities
for certain combinations of X and n. In the table, the decimal point
has been omitted to save space, therefore 625 should be read as
0.625. For example, if 17 out of 25 panelists prefer sample A, the
probability from Table 7.2 (X = 17, n = 25) would be 0.108. Since
62
a probability of 0.05 or less is usually required for the result to be
considered significant, it would be concluded that sample A was
not significantly preferred over sample B. Had 19 of the 25 judges
chosen sample A as being the preferred sample, the probability
would have been 0.015 and a significant preference for sample A
would have been shown.
No knowledge of the degree of preference for the preferred
sample or of the degree of difference in preference between the
samples results from the paired-preference test.
Example of a paired-preference test used by an
in-house consumer panel to determine preference
for pureed beans
Bean purees were prepared from two varieties of black beans, A
(631) and B(228). A paired-preference test was used to determine if
one bean puree was preferred over the other.
Forty untrained panelists were recruited from within the institute
(in-house panel). The two samples were presented to each panelist
simultaneously. Each panelist evaluated the two samples only once.
Twenty panelists received sample A (631) first, twenty panelists
received sample B (228) first. The ballot used when sample A was
presented first is shown in Figure 8.
The number of panelists who preferred each sample was
totalled. Thirty of the forty panelists preferred sample B. In Table
7.2 (Appendix 7) for X = 30 and n = 40, the probability is 0.002.
This result was therefore statistically significant, and it was
63
concluded that the in-house panel preferred bean puree B over bean
puree A.
Name:
Date:
Taste the two bean puree samples in front of you, starting with the
sample on the left. Circle the number of the sample that you prefer. You
must choose a sample. Guess if you are unsure.
631
228
Figure 8 Ballot for bean puree paired-preference test
7.1.2
Acceptance Tests
Acceptance tests are used to determine the degree of consumer
acceptance for a product. Category scales, ranking tests and the
paired-comparison test can all be used to assess product
acceptance. Acceptance of a food product usually indicates actual
use of the product (purchase and eating).
General Instructions for Conducting an
Acceptance Test Using Ranking.
Description of panelists' task:
Panelists are asked to rank
coded samples for acceptance in order from the least acceptable to
64
the most acceptable. Ties, where the samples are given equal
acceptance ranks, are not usually allowed.
Presentation of samples: Three or more samples are presented
in identical sample containers, coded with 3-digit random numbers.
Each sample is given a different number. All the samples are
simultaneously presented to each panelist in a balanced or random
order and retasting of the samples is allowed. An example of a
ballot for ranking of acceptance is given in Figure 9.
Name:
Date:.
Please taste each of the samples of black beans in the order listed
below. Assign the sample with the most acceptable texture a rank value
of 1, the sample with the next most acceptable texture a rank value of 2,
and the sample with the least acceptable texture a rank value of 3. Do not
give the same rank to two samples.
Code
Figure 9
Rank assigned
Ballot for bean texture acceptability ranking test
65
Analysis of data: For data analysis, the ranks assigned to each
sample are totalled. The samples are then tested for significant
differences by comparing the rank totals between all possible pairs
of samples using the Friedman Test. Tables 7.3 and 7.4 (Appendix
7) present expanded tables for this test, for 3-100 panelists and
3-12 samples (Newell and MacFarlane, 1987). The differences
between all possible rank total pairs are compared to the tabulated
critical value, based on a specific significance level (5% in Table
7.3; 1% in Table 7.4) and the number of panelists and samples
involved in the test. If the difference beween pairs of rank totals is
larger than the tabulated critical value, the pair of samples are
significantly different at the chosen significance level.
Example of a ranking test used by an in-house
consumer panel to determine acceptability of
bean texture.
Cooked bean samples were prepared from three varieties of
black beans. A ranking test was used to obtain an indication of the
most acceptable black bean texture.
Thirty untrained panelists were recruited from within the
institution (in-house panel). All treatments were simultaneously
presented to each panelist. Each panelist evaluated the samples
only once. The three samples could be served in six possible
orders, as shown in Table 4 (Section 7.2.1). Since there were thirty
panelists, the order of sample presentation was balanced such that
five panelists received samples in each of the six possible orders.
The ballot used for ranking acceptability is shown in Figure 9.
Panelists were instructed to rank the texture of the samples for
acceptability without ties, giving each sample a different rank even
if they seemed to be similar. The sample which was ranked as
66
having the most acceptable texture was assigned a rank of 1, the
sample with the next most acceptable texture was assigned a rank
of 2 and the sample with the least acceptable texture was assigned a
rank of 3. The ranked values given to each sample by all 30
panelists were tabulated as shown in Table 1.
The differences between rank total pairs were:
C - A = 76-33 = 43
C - B = 76-71 = 5
B-A = 71-33 = 38
The tabulated critical value at p=0.05, for 30 panelists and three
samples, from Table 7.3, is 19. Thus, the cooked texture of bean
varieties A and C were significantly different and the cooked
texture of bean varieties A and B were significantly different.
The in-house panel found the cooked texture of black bean
varieties B and C less acceptable than the cooked texture of bean
variety A. There was no difference in texture acceptability between
varieties B and C.
7.1.3
Hedonic Tests
Hedonic tests are designed to measure degree of liking for a
product. Category scales ranging from like extremely, through
neither like nor dislike, to dislike extremely, with varying numbers
of categories, are used. Panelists indicate their degree of liking for
each sample by choosing the appropriate category.
67
Table 1
Tabulated Ranking1 for Acceptance Test Data
Panelist
A
Black Bean Varieties
B
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
1
1
1
1
1
1
1
1
1
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
1
1
1
2
2
3
2
2
3
2
2
3
2
1
3
3
3
2
3
2
3
2
2
3
3
2
2
3
3
1
2
3
3
1
3
2
3
3
2
3
3
2
3
3
2
2
2
3
2
3
2
3
3
2
2
3
3
2
2
3
3
2
2
3
Rank Total
33
71
76
1 Highest rank = 1 = most acceptable texture, 3 = least acceptable texture
C
68
General Instructions for Conducting a Hedonic
Test Using a 9-point Scale.
Description of panelists' task: Panelists are asked to evaluate
coded samples of several products for degree of liking, on a 9-point
scale. They do this by checking a category on the scale which
ranges from like extremely to dislike extremely. More than one
sample may fall within the same category.
Presentation of samples:
The samples are presented in
identical sample containers, coded with 3-digit random numbers.
Each sample must have a different number. The sample order can
be randomized for each panelist or, if possible, balanced. In a
balanced serving order each sample is served in each position (f.e.
first, second, third, etc.) an equal number of times. A good
discussion of serving orders with examples of 3, 4, 5 and 12 sample
balanced designs is given in Stone and Sidel (1985). A balanced
serving order for three samples is given in Table 4 (Section 7.2.1).
Samples may be presented all at once or one at a time. Presenting
the samples simultaneously is preferred as it is easier to administer
and allows panelists to re-evaluate the samples if desired and make
comparisons between the samples. An example of a ballot for the
hedonic test is given in Figure 10.
Analysis of Data: For data analysis, the categories are
converted to numerical scores ranging from 1 to 9, where 1
represents dislike extremely and 9 represents like extremely. The
numerical scores for each sample are tabulated and analyzed by
analysis of variance (ANOVA) to determine whether significant
differences in mean degree of liking scores exist among the
samples. In the ANOVA, total variance is partitioned into variance
assigned to particular sources. The among-sample means variance
is compared to the within-sample variance (also called the random
69
experimental error)1. If the samples are not different, the
among-sample means variance will be similiar to the experimental
error. The variance due to panelists or other blocking effects can
also be tested against the random experimental error.
The measure of the total variance for the test is the total sum of
squares or 55(T). The measured variance among the sample means
is the treatment sum of squares or S5(Tr). The measure of the
variance among panelists’ means is the panelist sum of squares or
SS(P). Error sum of square, ^(E), is the measure of the variance
due to experimental or random error. Mean Squares (MS) for
treatment, panelist and error are calculated by dividing each SS by
its respective degrees of freedom. The ratios of the AfS(Tr) to the
MS(E) and the ratio of the AfS(P) to the AfS(E) are then calculated.
These ratios are termed F ratios or F statistics. Calculated F ratios
are compared to tabulated F ratios (Tables 7.5 and 7.6, Appendix 7)
to determine whether there are any significant differences among
the treament or panelists’ means. If the calculated F ratio exceeds
the tabulated F ratio for the same number of degrees of freedom,
then there is evidence of significant differences. Tabulated F ratios
are given for 0.05 and 0.01 levels of significance in Tables 7.5 and
7.6, respectively.
Once a significant difference has been found, multiple
comparison tests can be carried out to determine which treatment
or population means differs from each other. Details of the
ANOVA calculations are given in the following example.
1 Since the total variance within-samples comes from pooling individual
variances within-samples, a necessary assumption is that the true within-sample
variances are equal. There are formal tests that can be done to test for equality of
within-sample variances (Homogeneity of Variance).
70
Example of a hedonic test used by an in-house
consumer panel to determine degree of liking for
bean varieties.
A hedonic test was conducted to determine consumers’ degree
of liking for five varieties (treatments) of cooked black beans using
the 9-point category scale shown in Figure 10.
The beans were cooked, staggering the cooking times, so that all
five samples were done ten minutes before the panel began.
Twenty-eight untrained in-house consumer panelists evaluated the
five samples once. Ten gram samples of the five varieties of beans
were presented simultaneously, in styrofoam sample cups with lids,
to each panelist. For five samples, 120 serving orders were
possible, however, with only 28 panelists this large number of
serving orders was impossible to balance. Therefore, the serving
order was randomized for each panelist.
After each panelist had evaluated the five samples, the
descriptive categories were converted to numerical scores. The
scores were tabulated and analyzed by analysis of variance. The
tabulated scores for the first seven panelists are shown in Table 2.
The analysis of variance shown was carried out using the scores for
the seven panelists only.
71
Name:
Date:
Please look at and taste each sample of black beans in order from left
to right as shown on the ballot. Indicate how much you like or dislike
each sample by checking the appropriate phrase under the sample code
number.
Code
Code
Code
Code
Code
Like
Extremely
Like
Extremely
Like
Extremely
Like
Extremely
Like
Extremely
Like
Very Much
Like
Very much
Like
Very Much
Like
Very Much
Like
Very Much
Like
Moderately
Like
Moderately
Like
Moderately
Like
Moderately
Like
Moderately
Like
Slightly
Like
Slightly
Like
Slightly
Like
Slightly
Like
Slightly
Neither Like
Nor Dislike
Neither Like
Nor Dislike
Neither Like
Nor Dislike
Neither Like
Nor Dislike
Neither Like
Nor Dislike
Dislike
Slightly
Dislike
Slightly
Dislike
Slightly
Dislike
Slightly
Dislike
Slightly
Dislike
Moderately
Dislike
Moderately
Dislike
Moderately
Dislike
Moderately
Dislike
Moderately
Dislike
Very Much
Dislike
Very Much
Dislike
Very Much
Dislike
Very Much
Dislike
Very Much
Dislike
Extremely
Dislike
Extremely
Dislike
Extremely
Dislike
Extremely
Dislike
Extremely
Comments:
Comments:
Comments:
Comments:
Comments:
Figure 10 Ballot for bean varieties hedonic test using a 9-point scale
72
For the analysis of variance (ANOVA), the following
calculations were carried out, (where N = the total number of
individual responses, s = sum of):
Correction Factor:
CF
= Grand Total2
N
1632
“35“
= 759.1
Total Sum of Squares:
5S(T)
= S (each individual response2) - CF
= (22+l2+ 12+... + 22 + 32)- 759.1
= 917-759.1
157.9
Treatment Sum of Squares:
SS (Tr)
2
E (each treatment total )
number of responses per treatment
152 + 432 + 522 + 312 + 222
7
6223 - 759.1
= 889 - 759.1
129.9
- 759.1
-CF
73
Panelist Sum of Squares:
2
E (each panelist total )
number of responses per panelist
SS(P)
- CF
262 + 252 + 182 + 232 + 232 + 242 + 242
5
3835
- 759.1
- 759.1 = 767-759.1
= 7.9
Error Sum of Squares:
SS(T) - SS(Tr) - S5(P)
S5(E)
157.9 - 129.9 - 7.9
= 20.1
Table 2
Tabulated Category Scores1 for the Hedonic Test
Black Bean Varieties (Treatments)
Panelist2
A
B
C
D
E
1
2
3
4
5
2
1
1
2
2
4
3
6
8
9
6
4
6
6
7
5
8
7
8
3
5
4
4
5
4
4
2
4
3
2
3
15
43
52
31
22
6
7
TREATMENT
TOTAL
7
6
6
6
Panelist Panelist
Total
Mean
163
GRAND TOTAL
TREATMENT
2.1
MEAN
6.1
7.4
4.4
26
25
18
23
23
24
24
3.1
highest score = 9 = like extremely Lowest score = 1 = dislike extremely
2the responses of only 7 of the 28 panelists are given and analyzed
5.2
5.0
3.6
4.6
4.6
4.8
4.8
74
The mean square (MS) values were calculated by dividing the
SS values by their respective degrees of freedom as follows:
Total Degrees of Freedom, df(T)
=
=
=
=
The total number of responses -1
N-l
35-1
34
Treatment Degrees of Freedom, df(Tr) = The number of treatments -1
= 5-1
= 4
Panelist Degrees of Freedom, df(P)
= The number of panelists - 1
= 7-1
= 6
Error Degrees of Freedom, df(E)
= df(T) - df(Tr) - df(P)
= 34-4-6
= 24
Treatment Mean Square, MS(Tr)
S5(Tr) / df(Tr)
129.9
4
Panelist Mean Square, MS(P)
= S5(P)/df(P)
Z9
6
Error Mean Square, AfS(E)
= 32.48
= 1.32
= S5(E)/df(E)
=2^- = 0.84
24
75
The F ratios, for treatments and panelists were calculated by
dividing their respective MS values by the MS for error. The tabular
F ratios were obtained from statistical tables of the F distribution
(Appendix 7, Table 7.5). For example, the tabulated F ratio for
treatments with 4 degrees of freedom (df) in the numerator and 24
df in the denominator, at p ^.05, is 2.78. The F ratio for panelists
with 6 df in the numerator and 24 df in the denominator at p ^.05 is
2.51. The calculated F ratios must exceed these tabular F values in
order to be considered significant at the 5% level.
The sums of squares, mean squares, degrees of freedom and F
ratios are summarized in the ANOVA table shown in Table 3.
Table 3
ANOVA Table for the Hedonic Test
______ F ratio_____
Calculated Tabular
(p^.()5)
Source of
Variation
df
SS
Total (T)
34
157.9
Treatment (Tr)
4
129.9
32.48
38.67
2.78
Panelists (P)
6
7.9
1.32
1.57
2.51
Error (E)
24
20.1
0.84
MS
Since the calculated treatment F ratio of 38.67 exceeded the
tabulated F ratio of 2.78, it was concluded that there was a
significant (p .05) difference among the mean hedonic scores for
the five bean varieties. The calculated panelist F ratio of 1.57,
however, did not exceed the tabular F ratio of 2.51. Thus, no
significant panelist effect was present.
76
The ANOVA indicated that there were significant differences
among the five bean varieties. To determine which bean samples
differed significantly from each other, a multiple comparison test,
Duncan’s New Multiple Range Test and Tables 7.7 and 7.8,
Appendix 7, were used. This test compares the differences between
all pairs of means to calculated range values for each pair. If the
difference between pairs of means is larger than the calculated
range value, the means are significantly different at the specified
level of significance. Range values are computed based on the
number of means that lie between the two means being tested,
when the means are arranged in order of size.
To carry out the Duncan’s Test, treatment means were arranged
in order of magnitude as shown.
C
7.4
Black Bean Varieties
Treatment Means
B
6.1
D
4.4
E
3.1
A
2.1
To compare the 5 means in this example, range values lor a
range of 5, 4, 3 and 2 means were calculated from the following
equation:
Range = Q / AfS(E)
J t
The MS(E), taken from the ANOVA table (Table 3) was
0.84. The t is the number of individual responses used to calculate
each mean; in this example t = 7.
Range = Q / 0.84 = Q (0.346)
J
7
II
77
Q values were obtained from Table 7.7 (Appendix 7) at the
same level of significance used in the ANOVA, p^.05. The df(E),
or 24 df, are also needed to determine Q values. From Table 7.7, Q
values for 24 df are:
Q value for 5 means
Q value for 4 means
Q value for 3 means
Q value for 2 means
=
=
=
=
3.226
3.160
3.066
2.919
Range values were then calculated.
Range = Q (0.346)
Range for 5 means
Range for 4 means
Range for 3 means
Range for 2 means
= 3.226 (0.346)
= 3.160 (0.346)
= 3.066 (0.346)
= 2.919 (0.346)
= 1.12
= 1.09
= 1.06
= 1.01
The 5 mean range value was applied to the means with the
greatest difference between them, 7.4 and 2.1, since these values
covered the range over 5 means. The difference, 5.3, was greater
than 1.12. These two means, therefore, were significantly different.
The next comparison was between the means 7.4 and 3.1, using
the 4 mean range value (1.09). Since the difference between the
means (4.3) was greater than 1.09, these means were also
significantly different.
The three mean comparison was between means 7.4 and 4.4.
78
7.4 - 4.4 = 3.0 > 1.06
One two mean comparison was between 7.4 and 6.1.
7.4-6.1 = 1.3 > 1.01
The next highest mean was then compared with the lowest mean
and the difference was compared to the range value for 4 means.
6.1 -2.1 = 4.0 > 1.09
This procedure was carried out as shown, until all mean
comparisons had been made.
6.1 -3.1
6.1 -4.4
4.4- 2.1
4.4- 3.1
3.1 -2.1
=
=
=
=
=
3.0
1.7
2.3
1.3
1.0
>
>
>
>
<
1.06
1.01
1.06
1.01
1.01
The significant differences among the means were presented by
using letters. Means followed by different letters were significantly
different at the 5% level of probability.
Black Bean Varieties
Treatment Means
C
7.4a
B
6.1b
D
4.4c
E
3. Id
A
2.1d
79
Bean variety C was liked significantly more than all other
samples; variety B was significantly more liked than varieties D, E
and A; variety D was liked more than varieties E and A; variety E
and A were equally liked.
7.2
PRODUCT-ORIENTED TESTS
Product-oriented tests commonly used in food testing
laboratories include difference, ranking for intensity, scoring for
intensity, and descriptive analysis tests. These tests are always
conducted using trained laboratory panels. Examples of
product-oriented tests which have been included in this manual are:
a triangle test for difference, a ranking test for intensity and a
scoring test for intensity.
7.2.1
Difference Tests
Tests for difference are designed to determine whether two
samples can be distinguished from each other by sensory analysis.
Difference tests can be used to determine whether a noticeable
change has occurred in a food’s appearance, flavour, or texture as a
result of storage, of a change in processing methods, or of
alteration of an ingredient.
The triangle test is a form of difference test that is commonly
used to determine whether there are perceptible differences
between two samples. The size and direction of difference between
two samples, however, is not specified in this test. The test may
also be used to determine panelists’ ability to discriminate
80
differences in appearance, odour, flavour or texture of foods. To
test for discrimination of differences related to one characteristic,
the samples being compared must be identical in all other
characteristics. Other tests, such as the paired-comparison or
duo-trio test can be used for similar purposes.
The paired-comparison test is similar to the paired-preference
test described in Section 7.1.1, except that panelists are asked
which of the two samples has the greater intensity of a specific
characteristic. For example, panelists may be asked "which sample
is sweeter?" or "which sample is most tender?". Using this test the
sweeter or more tender sample can be identified, but the extent of
the difference is not measured.
In the duo-trio test panelists are presented with three samples.
One sample is labelled R for reference and the other two samples
are coded with 3-digit random numbers. One of the coded samples
is identical to the reference (R) and the other is not. Panelists are
asked to taste R first, then the coded samples and identify which of
the two coded samples is the same as R (or different from R). The
duo-trio test indicates difference, but not the direction or magnitude
of the difference between samples.
The paired-comparison and duo-trio difference tests will not be
described in detail in this manual. Procedures for conducting these
tests and analyzing the data arc described by O’Mahony (1986),
Stone and Sidel (1985), Gacula and Singh (1984), Larmond (1977)
and ASTM Committee E-18 (1968).
81
General Instructions for Identifying a Difference
Using a Triangle Test
Description of panelists' task: Panelists are presented with
three coded samples, one different and two the same, and asked to
select the different sample. Panelists are required to select the
different sample even if they cannot discern any differences among
the samples (fe. panelists must guess when in doubt).
Presentation of samples: The two different samples (A and B)
are presented to the panelists in sets of three. Panelists receive
either two A’s and one B, or two B’s and one A. The three samples
are presented in identical sample containers coded with 3-digit
random numbers. All three code numbers on the samples presented
to each panelist must be different, even though two of the samples
are identical.
There are six possible serving orders for the triangle test and
these are shown in Table 4. Each order should be presented an
equal number of times, for a balanced serving order. This is
possible, however, only if there are six, or some multiple of six,
panelists. Alternately the order can be randomized so that each
panelist has an equal chance of receiving any of the six possible
serving orders.
The samples are presented simultaneously in the order selected
for each panelist so that the panelists can evaluate the samples from
left to right. Retasting of the samples is allowed. An example of a
ballot for the triangle test is given in Figure 11. The order in which
the panelists are to evaluate the samples should be indicated on the
ballot.
82
Table 4
Six Possible Serving Orders for a Triangle Test
Panelist Number
First
1
2
3
4
5
6
Order of Sample Presentation
Second
Third
256(A)
256(A)
670(B)
349(B)
349(B)
831(A)
831(A)
349(B)
256(A)
670(B)
256(A)
349(B)
349(B)
831(A)
831(A)
256(A)
670(B)
670(B)
Analysis of Data: Results are analyzed using a one-tailed
binomial test for significance. The one-tailed test is appropriate
since one sample is known to be different and there is therefore
only one ’’correct’' answer. The 2-tailcd binomial test was used to
analyze the paired-preference data of Section 7.1.1. For that lest
either of the two samples could have been preferred; that is, two
"correct” answers were possible, and so a 2-tailed test of
significance was used. The triangle test also differs from the paired
test in that the probability of picking the correct sample by chance
is 1/3. In the paired test the probability of picking the correct
sample by chance is 1/2. Thus the table used for the triangle test
(Table 7.9) is not the same as the one used for the paired test (Table
7.2).
In the triangle test the number of panelists correctly identifying
the different sample is totalled and the total tested for significance
using Table 7.9 (Appendix 7). In this table X represents the number
of panelists choosing the different sample correctly and n
represents the total number of panelists participating in the test.
The table contains 3 decimal probabilities for certain combinations
83
of X and n. In Table 7.9 the initial decimal point has been omitted
to save space, therefore 868 should be read as 0.868. For example,
if 9 out of 17 panelists correctly choose the different sample, the
probability from Table 7.9 (X=9, n=17) would be 0.075. Since a
probability of 0.05 or less is usually required for significance, it
would be concluded that there was no significant difference
between the samples. In this type of difference test, both reliability
and sensitivity improve when more panelists are used.
Example of a triangle test used by a trained panel to
detect difference between treated and untreated
samples.
A triangle test was conducted to determine whether black beans
that had received a prestorage heat treatment were noticeably
different from untreated beans, after both had been stored under the
same conditions for six months. Each bean sample was cooked to
its optimum doneness following a standard procedure.
An in-house panel of 36 panelists evaluated the cooked bean
samples. Three samples were presented simultaneously to each
panelist. Six panelists received each of the six serving orders
shown in Table 4. The appropriately coded samples were selected
for each panelist and presented accompanied by a ballot on which
code numbers were listed in the order for tasting. The ballot used is
shown in Figure 11.
When all members of the panel had completed the test, their
ballots were marked either correct (+) when the odd sample was
correctly identified, or incorrect (-). Results were tabulated as
shown in Table 5. Using Statistical Table 7.9 (Appendix 7) the
84
Table 5
Tabulated Triangle Test Data
Panelist
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
Total Correct (+) =
Result
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
20
85
Name
Date:
You have been given three samples of beans. Two of these samples
are identical and one is different.
Taste the samples listed and place a check beside the code number of
the sample that is different.
Code
The Different
Sample is:
Figure 11 Ballot for bean storage pretreatment triangle test
total number of panelists with correct answers (X) was compared to
the total number of panelists (n) and the significance level
determined.
From Table 7.9 (Appendix 7), it was determined that for 36
panelists and 20 correct responses, the significance level was
0.005.
It was concluded that the samples were significantly different at
the 0.005 level of probability, since 20 of the 36 panelists correctly
chose the different sample. The beans that had been pretreated were
86
therefore different from the untreated beans after 6 months of
storage. The size and type of difference, however, was not known.
7.2.2
Ranking for Intensity Tests
Intensity ranking tests require panelists to order samples
according to the perceived intensity of a sensory characteristic.
This type of test can be used to obtain preliminary information on
product differences, or to screen panelists for their ability to
discriminate among samples with known differences. Ranking tests
can show where there are perceptible differences in intensity of an
attribute among samples, but ranking does not give information
about the size of the difference between two samples. Samples
ranked one and two, for instance, could have a small but readily
perceived difference in intensity, while samples ranked two and
three could have a large difference in intensity of the attribute. This
would not be evident from the rankings.
General Instructions for Conducting a Ranking
Test for Intensity.
Description of panelists ’ task: Trained panelists are asked to
rank coded samples for the intensity of a specific characteristic, by
ordering the samples from the most intense to the least intense.
Ties arc not usually allowed.
Presentation of samples: Three or more samples are presented
in identical sample containers, coded with 3-digit random numbers.
Each sample is given a different code number. All the samples are
simultaneously presented to each panelist in a balanced or random
87
order. The panelists are allowed to re-evaluate the samples as
necessary to make the required comparisons among them. An
example of a ballot for ranking for intensity is given in Figure 12.
Analysis of data: When all panelists have ranked the samples,
the ranks assigned to each sample are totalled. The samples are
then tested for significant differences by comparing the rank totals
between all possible pairs of samples using the Friedman Test and
Statistical Tables 7.3 and 7.4 (Appendix 7). This method of data
analysis was used for the ranking for acceptability data, Section
7.1.2. That example should be reviewed for details of the test.
Example of a ranking test used by trained panel to
compare bean seedcoat toughness.
A ranking test was conducted to compare the seedcoat
toughness of beans which had been stored under four different
temperature and humidity conditions for three months. Ten
panelists, trained to evaluate seedcoat toughness (Appendix 4),
participated in the test. All four coded bean samples were
simultaneously presented to each panelist. Each panelist evaluated
the samples only once. Panelists were instructed to rank the
samples for seedcoat toughness without ties, giving each sample a
different rank even if the products seemed to be similar. A rank of
1 was given to the sample with the toughest seedcoat, and a rank of
4 to the sample with the least tough seedcoat. The ballot used is
shown in Figure 12.
The ranked values given to each sample were tabulated and
totalled as shown in Table 6. The differences between rank total
pairs were:
88
D-A
D-B
D-C
C-A
B-C
B-A
36-18
36-26
36-20
20-18
26-20
26-18
= 18
= 10
= 16
= 2
= 6
= 8
Name:
Date:
Please evaluate each cooked bean sample for seedcoat toughness.
Separate the seedcoat from the cotyledon by biting the beans (2 beans)
between the molar teeth and rubbing the cotyledon out between the
tongue and palate. Then evaluate the force required to bite through the
seedcoat with the front teeth.
Evaluate the samples in the order listed below, from top to bottom,
then arrange the samples in order of their seedcoat toughness. Assign the
sample with the toughest seedcoat a rank value of 1; the samples with the
next toughest seedcoats rank values of 2 and 3 and the sample with the
least tough seedcoat a rank of 4.
Code
Figure 12
Rank assigned
Ballot for seedcoat toughness ranking test
89
Table 6
Tabulated Ranking1 for Intensity Test Data
Panelist
A
1
2
3
4
Storage Treatment
B
C
3
2
2
3
3
1
2
2
2
1
1
1
2
3
1
5
1
3
6
3
7
8
9
10
2
1
3
1
4
2
4
1
Rank Total
18
D
4
4
4
4
4
4
3
3
2
2
4
1
4
26
20
36
3
’Lowest rank = 1 = toughest seedcoat
Tabulated critical values for psO.05, 10 panelists and 4 samples
from Table 7.3 is 15. Only the differences between D and A, and
D and C, were significant (z>. larger than 15).
Therefore the seedeoats of bean samples stored under D
conditions were not as tough as the seedcoats of samples stored
under A and C conditions.
90
7.2.3
Scoring for Intensity Tests
Scoring tests for intensity measurements require panelists to
score samples, on line scales or category scales, for the perceived
intensity of a sensory characteristic. Scoring tests measure the
amount of difference between samples, and allow samples to be
ordered by increasing or decreasing intensity of a characteristic.
General Instructions for Conducting a Scoring Test
for Intensity.
Description of panelists' task: Panelists score the perceived
intensity of the specific characteristic in each coded sample on an
interval scale from low intensity to high (or strong) intensity.
Presentation of samples: Samples are presented in identical
sample containers, coded with 3-digit random numbers. Each
sample is given a different number. All samples are simultaneously
presented to each panelist in a balanced or random order. Panelists
are instructed to evaluate each sample independently. In order to
minimize intercomparison of samples, the experimenter may
present samples one at a time to each panelist, removing each
sample after testing and before presenting the next sample. In cither
case, the panelists are instructed to evaluate each sample, and
indicate the intensity of the specified characteristic by checking an
appropriate category or by making a vertical mark on a line scale.
An example of a category scale used for scoring intensity is shown
in Figure 7, Section 6.1.3.
Analysis of data: For analysis of category scale data, the
categories are converted to numerical scores by assigning
91
successive numbers to each category; usually the number 1 is given
to the category of lowest intensity. For analysis of line scale results,
panelists’ marks are converted to numerical scores by measuring
the distance in cm from the left or lowest intensity point on the
scale to the panelists’ marks, converting the scores using 0.5 cm =
1 unit score. The numerical scores for each sample are tabulated
and analyzed by analysis of variance (ANOVA) to determine if
significant differences exist among the samples.
Multiple
comparison tests can then be used to determine which samples
differ from each other.
The entire scoring test is usually repeated on several occasions
to obtain several replications of the data. This allows for an
accurate measure of experimental error. The use of replicated data
also allows the experimenter to assess the performance of the panel
by examining the panel results from each replication to see whether
significant differences exist among means for each replication.
Example of line scale scoring used by a trained panel
to determine bean hardness.
A scoring test was used to compare the hardness of beans
cooked for five different cooking times. Samples of one variety of
black bean were cooked for 30, 50, 70, 90 and 110 minutes.
Starting times were staggered such that all samples finished
cooking at the same time.
Seven panelists, who had been trained for texture evaluation of
beans, served on the panel. The five bean samples were
simultaneously presented to each panelist at each session, in a
randomized complete block design. This was repeated two more
92
times, using different code numbers on each occasion, to give three
replications. The ballot for scoring, which used a 15 cm line scale,
is shown in Figure 13. Panelists scored the samples by placing a
vertical line at the appropriate point on each line scale.
Name:
Date:
Evaluate the 5 bean samples for hardness in the order shown on the
ballot, from top to bottom. Bite down once with the molar teeth on the
sample of beans (2 beans). Hardness is the force required to penetrate the
sample. Place a vertical line on the horizontal line scale at the position
that indicates the hardness of the bean sample.
CODE
1
not hard
I
not hard
I
not hard
I
not hard
I
not hard
Figure 13 Ballot for bean hardness scoring test using a line scale
[Actual line scale should be 15 cm in length.]
J___
hard
j____
hard
j____
hard
J___
hard
J___
hard
93
Table 7
Tabulated Scoring for Hardness Test Data1
1
2
3
Cooking Time Treatments (min)
D
C
B
(90)
(50)
(70)
Replication
Replication
Replication
1
2
3
1
2
3
1
2
3
1
20
24
21
15
19
15
12
18
17
8
12
11
18
16
13
239
2
18
21
22
12
14
8
6
6
8
9
6
9
3
4
6
152
3
13
19
16
6
10
7
5
5
4
5
3
2
5
3
4
107
4
19
10
15
13
6
10
8
5
3
6
4
7
5
3
4
118
5
19
18
24
4
13
9
2
10
7
7
2
7
4
6
12
144
6
20
23
19
17
20
18
8
8
15
5
18
10
14
7
7
209
7
20
16
19
11
13
6
8
6
4
6
6
5
4
4
9
137
Treatment by Replication Total
129 131 136 78 95 73
49
58
58
46
51
51
53 43
Grand Total
55
p
A
N
E
L
I
S
A
(30)
Replication
Treatment Total
A=396
(Mean)
(18.9)
Replication Total2
(Mean)
B=246
(11.7)
Rep 1 =355
(10.1)
C=165
(7-9)
Rep 2 = 378
(10-8)
1 Highest score = 30 = hard
Replication totals are for each replication over all treatments
D=148
(7.0)
E
(110)
Replication Totals
1
2
3
1106
E=151
(7.2)
Rep 3 = 373
(10.7)
94
Numerical scores were determined by measuring the distance
from the left hand end of the scale to the mark, in 0.5 cm units. A
measured distance of 10 cm was therefore equal to a score of 20.
Data were tabulated as shown in Table 7 and analyzed by a
two-way analysis of variance. The effects of treatments (samples),
panelists, replications and interactions were partitioned out. The
analysis was similar to that used for the hedonic test. That example
should be reviewed for details of the ANOVA (Section 7.1.3).
For the analysis of variance the following calculations were
made:
Correction Factor:
11062
105
CF
11649.87
Total Sum of Squares:
SS(T)
= (202 + 242 + ... + 42 + Q2) - 11649.87
15522 - 11649.87
3872.13
Treatment Sum of Squares:
2
5S(Tr)
=
2
2
2
396 + 246 + 165 + 148 + 151
21
13774.38 -11649.87
= 2124.51
2
- 11649.87
95
Panelist Sum of Squares:
SS(P)
2392 + 1522 + ... + 1372
15
11649.87
12585.60-11649.87
= 935.73
Replication Sum of Squares:
2
SS(R)
S (each replication total )
number of responses in each replication total
- CF
3552 + 3782 + 373‘
L- 11649.87
35
= 11658.23- 11649.87
= 8.36
Error Sum of Squares:
S5(E)
= SS(T) - 55(Tr) - 5S(P) - 5S(R)
= 3872.13 - 2124.51 - 935.73 - 8.36
= 803.53
The mean square (AfS) values were calculated by dividing the
SS values by their respective degrees of freedom. Degrees of
freedom (df) were as follows:
96
Total Degrees of Freedom, df(T)
105-1 = 104
Treatment Degrees of Freedom, df(Tr)
5-1
= 4
Panelist Degrees of Freedom, df(P)
7-1
= 6
Replication Degrees of Freedom, df(R)
3-1
= 2
Error Degrees of Freedom, df(E)
df(T) - df(Tr) - df(P) - df(R)
104-4-6-2
92
Mean Squares were then calculated as shown:
MS (Tr) =
2124.51
4
= 531.13
MS(P) =
935.73
~~6~
= 155.96
MS(R) =
8.36
~2~
= 4.18
MS(E) =
803.53
—92“
= 8.73
The F ratios were calculated by dividing the MS for panelists by
the MS for error, the MS for treatments by the MS for error, and the
MS for replications by the MS for error. The tabular F ratios were
97
obtained from Statistical Tables 7.5 and 7.6 (Appendix 7) of the F
distribution. Since actual error degrees of freedom (92) are not
listed in the table, F values for 92 df were extrapolated from those
given. In this example F ratios were compared to the tabular values
of F for a 1% level of significance (p .01), Table 7.6. F ratios for
the main effects of panelists, treatments and replications are shown
in Table 8. Calculated F ratios for treatments and panelists were
much greater than the tabulated F ratios, indicating a highly
significant effect of both treatments and panelists. The replication
main effect was not significant.
The significant panelist effect could mean that the panelists
scored the samples in the same order, but that some panelists used
different parts of the scale. Therefore, the actual scores given to
the samples differed. Since there were large significant differences
due to both panelists and treatments, it is possible that some of
these differences were due to an interaction. A significant
interaction would indicate that the panelists were not all scoring the
samples in the same order. For examination of this interaction it
was necessary to calculate the sum of squares for the interaction
between panelists and treatments. Data from the original tabulated
data (Table 7) were totalled to obtain a treatment total for each
panelist combined over all three replications. These data are shown
in Table 9.
To calculate the treatment by panelist interaction the following
calculations were needed:
98
Table 8
ANOVA Table I Scoring for Hardness Test
Source of
Variation
df
ss
MS
Total
Treatments
Panelists
Replications
Error
104
4
6
2
92
3872.13
2124.51
935.73
8.36
803.53
531.13
155.96
4.18
8.73
________ F_______
Calculated Tabular
(P*.O1)
60.84
17.86
0.48
3.56
3.03
4.88
Table 9
Data Matrix of Treatment Totals for Each Panelist
Panelists
A
(30)
1
2
3
4
5
6
7
65
61
48
44
61
62
55
Cooking Time Treatment (min)
BCD
(50)
(70)
(90)
49
34
23
29
26
55
30
47
20
14
16
19
31
18
31
24
10
17
16
33
17
E
(HO)
47
13
12
12
22
28
17
99
Treatment x Panelist Matrix, Total Sum of Squares:
2
SSTCTrxP)
Z (treatment total for each panelist ) - CF
number of replications
652 + 612 + ... 4-282+ 17',2
—11649.87
3
14931.33 - 11649.87
3281.46
Interaction Sum of Squares:
SS(TrxP)
SSf (TrxP) - 5S(P) - SS(Tr)
3281.46 - 935.73 - 2124.51
221.22
Interaction Degrees of Freedom:
df(TrxP)
df(treatments) x df(panelists)
4x6
24
The degrees of freedom and the sum of squares for the
interaction between panelists and treatments were then added to the
ANOVA table and the mean square calculated (Table 10). The
main effects of treatments and panelists and the interaction effect of
panelists with treatments were tested with a new error mean square.
This new error mean square was calculated by subtracting the
Treatment SS, Panelist SS, Replication SS and the Interaction
(TrxP) SS from the Total SS. The new degrees of freedom for error
were calculated by subtracting from the total df, the df for
Treatments, Panelists, Replication and Interaction (TrxP). These
values were placed in the second ANOVA Table (Table 10).
100
Table 10
ANOVA Table II Scoring for Hardness Test
Source of
Variation
df
ss
MS
Total
Treatments
Panelists
Replications
TrxP
Error
104
4
6
2
24
68
3872.13
2124.51
935.73
8.36
221.22
582.31
531.13
155.96
4.18
9.22
8.56
_________ F
Tabular
Calculated
(P *-Qi)
62.05
18.22
0.49
1.08
3.63
3.10
4.96
2.10
The treatment by panelist interaction was not significant,
therefore, the significant panelist effect indicated that the panelists
scored the treatments in the same order. Some panelists may,
however, have scored the samples using different parts of the scale.
For example, one panelist may have scored all the samples using
the upper end of the scale only while others may have used the
central portion of the scale, resulting in samples which were scored
in the same order but with different numerical scores. A multiple
comparison test of the panelists’ mean scores could be used to
discover where the differences among panelists exist. This would
be useful during panel training to determine which panelists were
scoring samples, or using the scale differently, from others. During
studies, however, it is usually the specific treatment differences
which are of interest. The absence of a replication effect and of a
significant treatment x panelist interaction confirmed the
consistency of the panel performance in the example.
■ I. '
'
• V./ ■-
•
■ '• • -.. I
/’
.•
■
101
The ANOVA indicated that there was a significant difference in
the hardness of the four bean samples. To determine which
treatments differed significantly from the others, Tukey’s multiple
comparison test and Tables 7.10 and 7.11 (Appendix 7) were used.
Tukey’s test is similar to Duncan’s test (Section 7.1.3). Pairwise
comparisons between all of the means are tested against a
calculated range value. If the difference between pairs of means is
larger than the range value, the means are significantly different.
However, whereas Duncan’s test involves the calculation of a
number of range values, only a single range value is computed for
Tukey’s test. Any two means with a difference greater than the
range value are significantly different. To carry out this test,
treatment means were arranged in order of size as shown:
Cooking Treatments
Hardness Means
A
18.9
B
11.7
C
7.9
E
7.2
D
7.0
The standard error of the sample (treatment) means was
estimated by:
Standard Error =
(SE)
MS(E)
n
where MS(E) is taken from the final ANOVA table (Table 10) and
n is the number of responses per treatment.
A/^CPHEV/S
_
w (CLiO
102
SE
MS(E)
n
8.56
21
0.41
0.64
The range value was calculated from the following equation:
Range value
= Q(SE)
= Q(0.64)
The Q value was obtained from Table 7.11 (Appendix 7) with
68 df(E), 5 treatments and the same level of significance as the
ANOVA (p .01). Thus, Q = 4.80 (extrapolated from the table).
Range value = 4.80 (0.64)
= 3.07
Any two sample means which differed by a value greater than
the range value, 3.07, were significantly different at the 1% level.
All sample means were compared as follows:
103
A-D
18.9 - 7.0
11.9 >3.07
A-E
18.9 - 7.2
11.7 >3.07
A-C
18.9 - 7.9
11.0 >3.07
A-B
18.9-11.7
7.2 >3.07
B-D
11.7-7.0
4.7 >3.07
B-E
11.7-7.2
4.5 >3.07
B-C
11.7-7.9
3.8 >3.07
C-D
7.9 - 7.0
0.9 <3.07
C-E
7.9 - 7.2
0.7 <3.07
E-D
7.2 - 7.0
0.2 <3.07
Samples A and B were significantly different from each other
and all other samples. Samples C, E and D were not significantly
different from each other. The significant differences among the
means were shown by underlining together those means which
were not significantly different at the 1% level of probability.
Cooking Treatments
Treatment Means
A
18.9
B
11.7
C
7.9
E
7.2
_D_
7.0
Black beans cooked for 30 min (A) were harder than beans
cooked for 50 min (B), 70 min (C), 90 min (D) or 110 min (E).
Black beans cooked for 50 min (B) were significantly harder than
beans cooked for 70 min (C), 90 min (D) or 110 min (E).
However, beans cooked for 70 (C), 90 (D) or 110 (E) minutes did
not differ in hardness.
104
7.2.4
Descriptive Tests
Descriptive tests are similar to scoring for intensity tests except
that panelists score the intensity of a number of sample
characteristics rather than just one characteristic. Trained panelists
provide a total sensory description of the sample, including
appearance, odour, flavour, texture and aftertaste. There are many
types of descriptive tests including the Flavour Profile (Pangborn,
1986; Stone and Sidel, 1985; Powers, 1984; Moskowitz, 1983;
IFT, 1981; ASTM, 1968; Amerine et al., 1965; Caul, 1957;
Cairncross and Sjostrom, 1950), the Texture Profile (Pangborn,
1986; Stone and Sidel, 1985; Moskowitz, 1983; IFT, 1981; Civille
and Szczesniak, 1973; Brandt et al., 1963; Szczesniak, 1963;
Szczesniak et al., 1963) and Quantitative Descriptive Analysis
(Pangborn, 1986; Moskowitz, 1983; IFT, 1981; Zook and
Wessman, 1977; Stone et al., 1980, 1974). These methods will not
be described in this manual but the references listed provide
discussions and explanations of the techniques.
4-
Chapter 8
Planning A Sensory Experiment
In planning a sensory experiment, all of the factors discussed in
the previous sections of this manual should be considered carefully.
With these considerations in mind, specific tests and appropriate
methods of statistical analysis can be chosen. To facilitate planning
and conducting of sensory experiments, especially by researchers
who are new to this area, a number of tests have been described in
detail. Following the step-by-step descriptions given for each test
should assist with planning for similar types of testing.
Planning for a sensory experiment should include the steps
outlined below:
1) Define the specific objectives of the experiment. Clarify
questions to be answered (hypotheses to be tested) and state
them clearly.
2) Identify the constraints on the experiment: cost limits,
availability of materials, equipment, panelists and time.
106
3) Choose the type of test and panel to be used. Design the
ballot.
4) Design the experimental procedures so that, wherever
possible, variables not being tested will be controlled, and
panel results will not be biased. Randomization of
experimental factors that could bias results, such as the order
of sample preparation and presentation, should be planned.
5) Decide on the statistical methods to be used, keeping in mind
the objectives of the project, the type of test and type of
panel.
6) Prepare the forms to be used for recording sensory data.
Data should be recorded in a way that makes it convenient to
do the statistical analyses.
7) Plan for recruiting and orienting panelists; also, screening
and training of panelists, if required.
8) Do a trial run before proceeding with the experiment, to
check the appropriateness of sample preparation and
presentation procedures and the ballot.
Appendices
109
APPENDIX 1
Basic Taste Recognition Test
The following concentrations of the four basic tastes of sweet,
sour, salty, and bitter can be used for recognition tests.
Basic Taste
Substance
Concentration
Sweet
Salty
Sour
Bitter
sucrose
sodium chloride
citric acid
caffeine
or
quinine sulfate
1.0% w/v (2.5 g/250 mb)
0.2% w/v (0.5 g/250 mb)
0.04% w/v (0.1 g/250 mb)
0.05% w/v (0.125 g/250 mb)
0.00125% w/v (0.003 g/250 mL)
These solutions are prepared with distilled water and should be
prepared the day before and allowed to equilibrate overnight.
Approximately 25-30 mb of solution is needed per panelist. The solutions
are portioned into individual coded sample cups for tasting. 1-2 water
blanks are prepared and randomly placed among the 4 basic taste solutions.
The coded samples should be presented in a different random order to each
panelist. Panelists should be instructed to rinse the mouth with water
between samples and clear the mouth with crackers if necessary. An
example of a ballot is shown on the next page.
Panelists should be informed about their performance immediately
following the test. Poor performers could be allowed to repeat the test on
another day following some initial discussion about the basic taste
sensations and how they are perceived in the tongue and mouth. Panelists
who are unable to identify any of the basic taste solutions may be ageusic
(lack of taste sensitivity) and would not be good candidates for taste
panels.
110
APPENDIX 1 (cont.)
Ballot for Basic Taste Recognition Test
Name:
Date:
Basic Taste Recognition
Please taste each of the solutions in the order indicated on the
ballot, from top to bottom. The solutions may taste sweet, sour, salty or
bitter. There may be one or more samples of only water among the basic
taste solutions. Identify the taste solution in each coded cup. Rinse your
mouth with water before you begin tasting and also between each sample.
Crackers are also provided to clear the mouth between samples.
Cotie
Taste
111
APPENDIX 2
Basic Odour Recognition Test
Common household substances can be used for odour recognition
testing. The odourous substances (10-15) should be placed in dark
coloured (clear vials may be wrapped in aluminum foil) glass vials or test
tubes to mask any visual cues, and tightly capped. Liquids may be poured
onto a cotton ball in the tube, while solids can be placed directly into the
tube and covered with a cotton ball or square of cheesecloth. Vials or
tubes should be filled 1/4 -1/2 full in order to leave a headspace above the
sample for volatiles to concentrate.
Panelists are instructed to bring the vial to their nose, remove the lid
and take 3 short sniffs. Then, they should record the name of the odour, or
a related odour if they cannot identify the exact name, beside the sample
code on the ballot. For example, spicy if they cannot name the exact spice.
When interpreting results, the panel leader can give a full score to a correct
name, and a half score to a related name. An example of a ballot is shown
on the next page.
Panelists should be informed about their performance immediately
following the test. Those who have difficulty identifying the substances
may just need more practice and could be allowed to repeat the test on
another day. Odour and flavour language, as any other language, will
improve with practice. Panelists who are unable to smell many of the
substances arc likely anosmic or may have nasal or sinus congestion and
will likely not be good candidates for odour or flavour panels.
Example of substances which have been used are listed below:
112
Possible related odours
Substance
Odour
vinegar
sour, acetic acid
pickles
coffee
coffee
roasted
onion
onion
sulfury
cloves
cloves, eugenol
spicy, cinnamon
aniseed
anethol, anise
liquorice
cinnamon
cinnamon, eugenol
spicy, cloves
vanilla
vanilla
sweet
black pepper
pepper
spicy
prepared mustard
mustard
pickles
acetone
acetone
nail polish remover
alcohol
alcohol, ethanol
vodka
almond extract
almond
sweet
garlic
garlic, allicin
sulfury
lemon
lemon, sour, acid
citrus
honey
honey
sweet
113
APPENDIX 2 (cont)
Ballot for Basic Odour Recognition Test
Name:.
Date:
Basic Odour Recognition
The covered vials contain odourous substances commonly found in the
home or workplace. Bring the vial to your nose, remove the cap, take 3
short sniffs and try to identify the odour. If you cannot think of the exact
name of the substance, try to describe something which this odour reminds
you of.
Code
Odour
114
APPENDIX 3
Training and Monitoring a Bean Texture Panel
A sensory panel was trained specifically for texture evaluation of
cooked beans at INCAP. The textural characteristics to be evaluated and
the techniques for evaluation of the beans (Appendix 4) were first
developed by a trained sensory panel in the Department of Foods and
Nutrition at the University of Manitoba. Bean samples were evaluated
using a line scale. Food references were also selected (Appendix 5) to
anchor the endpoints of the line scale for each textural characteristic,
except chewiness. The INCAP panelists were trained using the techniques,
ballot and the line scale food references developed in Manitoba for each
characteristic. The ballot is shown in Appendix 6.
Training began by presenting panelists with the end point references
for each of the texture characteristics to be examined (Appendix 5). Each
panelist was then presented with a tray containing a ballot, directions for
evaluating the specific textural characteristics and the reference samples.
Definitions of the textural characteristics were reviewed by the panel
leader and the techniques to be used in the evaluation process were
illustrated. Each panelist practised the techniques using the reference
samples. After discussion to ensure that the panelists understood the
procedures, cooked bean samples that varied greatly in the textural
parameters being examined were presented for evaluation and scored on
the line scale a number of times. Thus, panelists received experience both
in evaluating the intensity of the specific textural characteristics in beans
and in using a line scale for scoring the samples.
After the bean samples had been evaluated, marks on the line scales
were converted to numerical scores by the panelists or panel leader,
measuring the distance in cm and converting the scores using 0.5cm = 1
unit score. Scores for each panelist were listed on a blackboard for
discussion and comparison. Although actual scores varied from one
panelist to another, most of the panelists achieved a consistent ordering of
the bean samples. It is more important for the relationship between the
products to be consistent (ie. sample A is always scored as being more soft
than sample B) and for individual panelists to be consistent over replicate
tests, than it is for all the panelists to give the samples identical scores.
However, training should, ideally, bring the panelists’ scores closer
together. For those who had scored products in the wrong order,
definitions and evaluation techniques were reviewed by the panel leader
115
and the panelists evaluated the samples again. The same training
procedure was repeated, using comparable bean samples, for several
sessions (days) until the panelists were comfortable with the techniques
and the repeatability of their scores was improved.
The next step in the training of the texture panel at INCAP was to have
the panel evaluate a variety of cooked black bean samples which had less
obvious textural differences, along with samples with large differences in
texture. For example, for hardness evaluation, samples which were
obviously under-cooked (hard), optimally cooked and over-cooked (soft)
were prepared and served along with samples that had varying degrees of
hardness.
To monitor the panelists’ performance, the same six bean samples were
evaluated on four different occasions. The samples were evaluated for
hardness, particle size, seedcoat toughness and chewiness, and were
prepared to have a wide range of differences in each of these attributes. An
analysis of variance with 6 treatments and 4 replications was used to
evaluate the data for each panelist individually, for each characteristic
measured. Treatment F values were calculated for each panelist’s scores,
and used as a measure of the panelist’s ability to discriminate among the
bean samples and to replicate his/her judgements for each attribute.
Characteristics that required more training (ie. many panelists scored them
inconsistently) were also identified. The results of these analyses were
discussed with the panel to provide an incentive for panelists to improve or
maintain their performance. At a later time a second panel evaluation was
conducted. Panel training was complete when the majority of the panelists
could discriminate between samples without difficulty and could
reproduce their scores consistently. Panelists who were having problems
and could not replicate their judgements were released from the panel.
116
APPENDIX 4
Techniques for Evaluating Textural Characteristics
of Cooked Beans
HARDNESS: Bite down once with the molar teeth on the sample of
beans (2 beans) and evaluate the force required to penetrate the sample.
PARTICLE SIZE: Chew the sample (2 beans) for only 2-3 chews
between the molar teeth, and then rub the cotyledon between the tongue
and palate and assess the size of the particles which are most apparent.
SEEDCOAT TOUGHNESS:
Separate the seedcoat from the
cotyledon by biting the beans (2 beans) between the molar teeth and
rubbing the cotyledon out between the tongue and palate. Then evaluate
the force required to bite through the seedcoat with the front teeth.
CHEWINESS:
Place a sample of beans (2 beans) in your mouth
and chew at a constant rate (1 chew per second), counting the number of
chews until the sample is ready for swallowing.
117
APPENDIX 5
Food References Used for Bean Texture Panels
Textural
Characteristic1
End Points
Reference
Hardness
soft
cream cheese -1 cm cube
hard
parmesan cheese -1 cm cube
Particle2
smooth
butter -1 cm cube
Size
chunky
coarsely chopped peanuts
Seedcoat
tender seedcoat
black-eyed beans (cooked 2 hr)
Toughness
tough seedcoat
navy beans ("Chapin brand")
a
(cooked 1 hr. 50 min)
'References for chewiness are not included as a chew count was used as a measure of
chewiness
Additional references of starchy (5% w/v slurry of cornstarch in water) and grainy
(cooked semolina - cream of wheat) were used during training.
3Taken directly from refrigerated storage and served,
temperature.
All others served at room
a
118
APPENDIX 6
Line Scale Ballot Used for Bean Texture Panels
Using the techniques provided for evaluating texture, evaluate the
samples according to the following parameters. First, evaluate the
reference samples to establish reference points, and then evaluate the
coded samples. Mark the relative intensity of the coded bean samples on
each scale, placing the code number of the sample above the mark.
INITIAL BITE
Hardness
j____
___ L
soft
hard
MASTICATORY PHASE
Particle Size
I___
chunky
I
smooth
Seedcoat
Toughness
I__________________
tender
(barely distinguishable
from cotyledons)
CHEWINESS
Code
Number of chews
I
tough
(leathery)
119
APPENDIX 7
Statistical tables
120
The authors wish to thank those who have granted permission for the
use of the following tables:
Tables 7.2 & 7.9
— Reproduced from E.B. Roessler, R.M.
Pangborn, J.L. Sidel and H. Stone, "Expanded
Statistical Tables for Estimating Significance
in Paired-Preference, Duo-Trio and Triangle
Tests".
Journal
of Food
Science,
43:940-943,947,1978.
Tables 7.3 & 7.4
— Reproduced from G.J. Newell and J.D.
MacFarlane, "Expanded Tables for Multiple
Comparison Procedures in the Analysis of
Ranked Data". Journal of Food Science,
52:1721-1725,1987.
Tables 7.5 & 7.6
Reproduced from M. Merrington, C.M.
Thompson and E.S. Pearson, "Tables of
Percentage Points of the Inverted Beta (F)
Distribution". Biometrica 33:73-88, 1943,
with permission from the Biometrika
Trustees.
Tables 7.7 & 7.8
Reproduced from H.L. Harter, "Critical
Values for Duncan’s New Multiple Range
Test". Biometrics 16:671-685, 1960, with
permission from The Biometrics Society.
Tables 7.10 & 7.11 — Reproduced from E.S. Pearson and N.D.
Hartley (Ed.). Table 29 in "Biometrika Tables
for Statisticians", Vol. 1, Third Edition
(1966), with permission from the Biometrika
Trustees.
121
TABLE 7.1
Random Numbers Table
92 73 35 54 98
16 51 87 38 01
33 17 94 03 07
27 57 83 36 77
61 29 94 65 15
26 56 39 28 82
90 16 71 58 81
27 41 40 81 74
07 53 58 09 94
91 54 01 44 49
91 43 06 93 24
97 58 00 77 86
55 96 82 24 83
24 00 21 76 21
97 49 97 99 48
72
36
90
58
94
00 82 80 75
00 66 83 36
41 63 36 50
55 77 99 65
72 47 63 35
85
01
48
52
36
19 70 64 43
19 53 58 68
18 86 67 17
38 17 40 90
06 68 95 71
83
19
32
19
75
81 58 29 20
73 59 65 95
47 42 59 60
44 93 63 76
87 08 73 42
93
16
96
45
32
72 49 83 27
27 57 65 41
19 56 32 02
72 47 25 60
58 61 49 91
06 73 46 53 80
36 49 07 54 07
16 03 06 41 98
18 69 63 00 95
95 40 38 76 23
05 74 62 18 31
43 91 74 14 40
79 75 15 66 64
80 72 06 98 19
84 49 63 08 97
95
95
63
73
68
28 64 99 86
28 57 76 51
29 50 27 92
61 99 74 05
61 99 05 55
09
40
53
48
91
88 60 27 23
69 87 66 60
54 06 47 69
52 50 77 53
52 19 84 90
44 53 22 40 86
64 95 99 77 03
72 03 60 45 24
33 50 89 98 24
77 32 15 76 35
35 87 80 47 11
79 67 71 05 99
21 42 53 79 70
19 74 34 26 41
44 71 26 06 01
96
00
87
12
91
23 64
48 94
15 89
11 50
57 51
69 33
87 42
22 45
40 11
20 03
80
18
71
58
84
49 89
98 77
80 10
08 97
44 32
56
68
41
11
78
32 08 70 52
54 50 25 19
46 28 06 13
91 09 05 33
96 49 50 26
62 85 85 53 60
38 80 73 89 22
95 62 19 35 63
02 68 19 97 21
57 35 48 61 03
00 26 26 76 80
63 34 31 24 12
90 94 04 59 81
67 79 26 16 91
38 80 07 08 00
43
88
16
54
83
56 95 78 65
25 99 34 44
57 45 02 98
10 56 58 61
09 42 96 63
24 01
33 81
29 10
80 25
90 30
20 81 11 25 21
19 08 20 74 51
97 35 35 17 44
31 24 22 34 95
45 24 01 96 21
39 00 27 47 60
58 08 80 92 56
56 81 87 37 10
36 35 32 43 44
51 93 66 36 87
83
85
56
69
42
45 25 28 77
62 98 67 67
34 49 22 78
88 75 56 07
90 04 20 32
57 99 02 56 59
95 03 17 42 26
50 96 35 45 40
86 01 84 12 25
09 36 63 34 92
98 38 25 89 65
96 44 19 06 74
21 51 98 10 18
39 71 66 87 17
02 34 96 00 65
07
31
02
89
37
91 84 67 81
39 97 94 27
06 48 96 58
23 53 07 31
61 22 15 69
68 28 29 88 56
73 21 85 37 49
02 50 08 84 77
28 49 35 23 70
84 95 64 21 30
53 00 66 27 29
94 48 60 83 76
23 90 50 36 16
84 43 13 05 94
40 87 75 49 77
08 05 73 10 47
34 69 65 58 41
69 81 53 97 43
47 13 65 25 13
07 51 00 99 20
05
14
48
95
55
01
87
81
50
80
01 48 45 39
69 97 80 92
00 48 13 19
12 61 20 06
37 92 91 91
67 92 67 17
01 68 34 92
72 90 17 09
30 42 62 43
89 79 56 56
03 92 42 50 75
17 99 59 73 84
02 64 44 68 72
21 12 23 11 00
52 17 02 58 37
01
82
65
08
33
61
25
20
72
79
21 45 98 77
79 53 32 88
06 85 37 06
29 93 65 45
96 12 18 61
98 45 10 05
75 01 78 64
83 44 44 05
84 12 22 08
20 07 40 39
78 87 90 47 73
10 09 07 09 56
96 85 90 55 00
32 56 55 63 16
35 33 98 80 47
02 98 19 89 04
08 95 86 18 94
36 28 10 04 88
06 86 46 28 40
54 03 31 08 17
41 32 02 75 96
24 59 60 88 81
48 51 76 58 18
27 05 35 96 75
19 73 48 30 37
74
13
11
06
22
65 72 58 01
46 20 67 80
55 87 94 27
17 26 28 05
73 62 86 68
74 79 29 05 29
84 81 97 94 32
60 26 92 09 00
31 20 79 16 72
06 92 82 65 10
97 26 91 36 36
14 22 07 84 10
71 97 72 05 30
27 09 62 94 26
44 54 09 11 70
20 07 46 35 19
75 77 18 14 65
14 21 83 99 46
06 78 56 42 82
91 01 26 15 61
10 59 61 30 64
06 20 64 72 63
92 42 30 97 23
86 79 11 15 34
60 88 55 02 30
18 52 97 24 80
79 92 43 52 33
74 83 22 11 41
00 26 83 82 10
59 97 88 69 09
81 40 99 83 02
86 12 76 48 29
73 53 48 10 58
48 32 37 41 48
05 03 63 84 72
28 97 79 99 29
77 02 34 49 00
00 06 97 25 53
60 89 27 58 07
26 71 02 18 54
82
40
36
74
16
74 09 21 65 09
90 75 09 73 22
14 72 51 66 03
23 20 32 85 06
47 71 02 68 97
32 54 78 17 61
45 70 71 03 26
84 60 44 03 15
98 69 68 60 11
71 72 57 50 28
41 84 72 37 06
14 31 86 14 46
66 73 62 29 38
23 40 29 11 30
00 05 44 94 39
92 44 02 30 78
21 97 96 81 73
49 58 81 94 87
95 57 54 85 83
01 47 28 79 18
43 56 00 74 48
88 04 88 37 99
98 66 17 22 98
44 82 12 48 80
60 97 87 65 41
37 41 79 33
83 62 63 94
01 06 61 74
48 23 98 74
61 94 44 07
122
TABLE 7.1 (cont.)
Random Numbers Table
10 13 13 26 18
44 12 36 38 45
83 32 55 94 83
36 45 23 42 71
45 95 89 90 57
70
79
92
62
17
49 46 76 82
86 16 35 18
77 01 10 34
83 89 00 76
56 42 25 50
40 95 03 23 50
92 77 77 81 35
85 18 90 52 66
26 12 30 39 49
14 15 06 51 15
95 49 81 65 59
60 08 00 03 89
85 96 17 94 52
24 45 01 47 08
00 80 71 54 30
02 82 92 04
66 62 76 78
41 23 62 38
84 30 71 03
67 13 08 22
48
55
75
26
18
49 46 13 05
98 14 69 20
66 65 53 81
95 66 53 08
03 62 21 80
70 39 51 64 27
57 98 02 26 10
45 92 91 30 40
78 28 19 29 37
06 75 58 50 56
88 99 50 78 53
86 10 72 87 40
87 51 94 08 14
31 49 85 07 24
79 84 02 22 01
36 38 63 84 10
69 47 33 42 65
56 21 46 71 66
64 66 66 08 96
35 51 70 18 40
73 21 64 74 12
39 03 27 23 32
80 61 49 78 64
82 57 07 60 73
67 31 60 46 91
75 38 41 38 45
22 20 18 41 18
39 32 85 84 47
88 01 18 68 21
98 60 80 34 35
13 14 30 41
55 60 88 50
61 04 68 49
45 47 59 48
87 62 91 53
74 21 40 94 50
73 80 64 42 22
94 76 20 78 36
57 03 45 01 48
23 56 34 35 24
70 88 69 08 15
18 99 34 87 84
26 48 03 59 72
91 93 68 02 12
62 89 43 63 96
08 77 01 10 01
92 36 01 77 86
23 65 04 78 73
87 33 60 04 44
23 08 04 34 89
22 64 60 57 25
19 54 65 51 61
40 88 56 38 96
20 76 80 37 19
88 88 46 92 53
73 50 67 04 79
64 49 89 84 19
89 53 72 54 42
37 61 47 47 97
99 19 02 60 25
38
54
14
48
24
36
89
41
46
74
27 64 92 29
98 18 56 63
56 64 46 30
01 03 34 17
19 18 58 38
58
25
45
66
48
35 06
39 08
81 79
07 40
08 38
64
43
68
68
84
56 10 04 14
82 44 80 16
24 81 78 90
39 42 15 64
10 95 26 08
05 23 15 01 16
80 95 26 12 72
72 77 32 46 74
74 50 44 54 98
10 51 65 03 85
06
23
87
84
53
82 42 61 36
09 79 03 13
83 21 08 77
76 40 56 55
81 33 01 30
85
45
06
88
52
11 82
65 10
85 17
74 42
86 84
39 04 99 99 88
71 26 72 67 25
50 52 72 95 18
72 38 35 28 72
98 96 95 85 40
58
42
78
01
11
62 42 25 91
21 55 04 02
25 54 97 15
68 81 01 63
33 74 74 56
12
42
60
38
78
91 51 29 18
83 92 68 94
48 66 67 26
07 58 03 81
44 06 30 43
10
06
00
76
92
98 80 95 16
73 69 97 88
49 27 22 93
14 99 63 59
17 79 45 06
33
78
65
15
87
67 25 73 98
36 03 05 66
65 06 28 88
25 44 97 49
28 87 91 10
85 61 04 72 82
22 61 53 32 48
43 24 65 96 14
97 74 51 42 50
38 13 94 23 00
39 39 92 82 15
41 08 33 09 15
67 19 30 70 86
30 62 51 65 19
75 99 63 62 71
59 88 42 57 39
77 06 04 87 95
53 87 46 62 35
81 76 32 69 78
72 71 50 44 59
81
18
18
25
50
84 00 13 70
89 75 41 91
15 54 38 69
67 19 45 22
74 03 59 58
19
71
21
38
37
32 41 38 86
50 12 52 67
09 49 46 79
12 15 45 16
11 95 42 71
21
57
12
81
59
3551 07 72
9866 78 29
9688 12 82
9904 02 27
7350 41 56
51 18 95 67 31
43 48 79 20 82
84 14 34 49 60
92 61 27 50 95
44 28 25 37 88
64 36 88 94 08
19 92 17 17 81
61 00 64 97 75
10 50 92 28 93
57 95 44 07 53
15 37 18 71 81
58 27 81 52 71
02 70 61 10 26
36 18 22 13 05
42 30 70 57 81
88
11
57
55
67
27 91 67 77
31 35 35 25
74 71 29 69
97 35 51 73
85 97 28 63
29
77
89
98
68
54 01 17 25
71 63 11 20
14 30 80 19
46 20 62 13
71 11 44 71
05 97 46 65 22
19 38 49 10 37
62 16 62 63 42
14 65 07 76 62
13 69 22 02 86
80 66 93 02 41
12 67 47 40 60
82 19 61 55 80
32 94 60 23 27
72 33 05 61 53
77
51
57
02
05
86 84 73 38
32 38 45 08
00 03 36 50
89 67 41 36
20 98 54 89
73 00 71 55 84
73 84 75 01 65
15 57 08 41 82
20 49 73 04 57
50 70 82 03 13
40
34
62
04
45
05 62 84 23
49 99 21 38
60 59 57 15
84 24 58 16
99 78 23 57
25 06 13 79 75
85 43 74 41 83
98 97 72 35 69
46 65 45 92 44
17 03 45 52 77
11 59 73 59 61
22 39 78 24 26
67 88 05 23 33
98 38 63 50 84
34 18 29 96 08
75
34
17
78
13
0954
5252
8753
9548
9560
74
08
15
45
29
10
49
66
51
99
96 59 97 10
49 38 79 65
10 34 71 89
43 03 73 74
05 27 46 03
16 60 93 86 21
49 35 65 65 13
36 75 40 54 51
06 00 51 96 96
18 85 14 23 73
66 99 07 16 49
62 75 27 53 39
08 72 56 36 95
24 43 31 55 91
75 85 62 16 80
90 05
21 56
33 08
93 70
19 02
39 32 27 14
54 46 65 05
70 87 09 17
87 35 30 53
79 66 71 11
TABLE 7.2
Two-Tailed Binomial Test
Probability of X or more agreeing judgements in n trials (p=l/2)
6
5
625
312
688
062
219
453
727
9
10
11
12
13
031
125
299
508
IM
9
016
070
180
M9
7 74
008
039
109
227
388
581
791
IS
17
ia
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
42
43
46
50____
• NOTE Initial cMcimai pomi hat baan omuiad
004
021
095
287
424
807
804
u
to
002
Oil
039
092
180
302
4M
629
815
001
006
022
067
118
210
332
648
824
003
013
036
077
143
236
359
503
664
832
14
002
007
021
049
098
187
283
383
523
<78
839
001
004
013
031
064
115
189
296
405
Ml
890
845
15
001
002
006
019
041
078
134
210
307
424
567
701
861
18
001
004
012
027
062
093
152
230
327
442
572
711
856
17
19
20
21
22
22
24
25
29
27
28
001
002
004
008
014
024
038
060
088
126
174
233
302
382
471
568
672
001
003
006
009
017
028
044
066
096
135
184
243
312
392
480
29
30
31
32
33
001
001
003
005
010
016
026
040
059
085
119
001
002
004
007
Oil
019
029
044
065
001
001
002
005
006
013
021
033
35
3«
37
001
001
002
004
007
001
001
003
001
001
003
007
017
035
064
108
24®
346
4M
5®6
720
860
001
004
Oil
023
043
078
122
186
295
362
473
597
728
864
001
003
007
015
029
062
087
136
200
281
377
487
608
738
868
002
004
009
019
036
061
099
150
215
296
392
500
681
743
871
001
002
006
013
024
043
071
100
163
229
310
405
511
627
749
875
001
002
004
008
016
030
060
060
121
175
243
324
418
522
636
755
878
001
002
006
011
020
035
068
090
132
188
256
337
430
533
644
761
880
001
001
003
007
014
024
041
065
099
143
200
268
349
M2
652
766
883
001
002
005
009
017
029
047
073
108
IM
211
280
360
451
551
669
771
885
001
001
003
006
Oil
020
034
053
081
164
222
291
371
461
560
666
775
888
001
002
003
006
012
020
032
049
072
104
193
253
322
001
002
004
008
014
023
036
OM
079
152
203
001
002
003
006
009
015
GJ
124
TABLE 7.3
Critical Absolute Rank Sum Differences
for "All Treatments" Comparisons
at 5% Level of Significance
Number of samples
Panelists
3
44
5
66
77
8
99
10
11
12
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
55
60
65
70
75
80
85
90
95
100
6
7
8
9
10
10
10
11
11
12
12
13
13
14
14
15
15
15
16
16
16
17
17
17
18
18
18
19
19
19
20
20
20
20
21
21
21
21
22
22
22
22
23
23
23
23
24
24
25
26
27
28
29
30
31
32
33
34
8
10
11
12
13
13
14
14
15
15
16
17
18
18
19
19
20
20
21
21
21
22
22
23
23
24
24
25
25
26
26
27
27
27
28
28
29
29
29
30
30
31
31
31
32
32
32
33
33
33
34
35
37
38
40
41
42
44
45
46
47
11
13
14
15
17
18
19
20
21
22
23
24
24
25
26
26
27
28
28
29
30
30
31
32
32
33
33
34
34
35
36
36
37
37
38
38
39
39
40
40
41
41
41
42
42
43
43
44
46
48
50
52
53
55
57
58
60
61
13
13
15
15
17
17
19
19
20
20
22
22
23
23
24
24
26
26
27
27
28
28
29
29
30
30
31
31
32
32
32
32
33
33
34
34
35
35
36
36
37
37
37
38
39
40
40
41
41
42
42
43
44
44
44
45
46
46
47
48
48
49
49
50
51
51
51
51
52
62
52
53
53
54
56
59
61
64
66
68
70
72
74
76
15
15
18
18
21
21
22
22
24
24
26
26
27
27
29
29
30
30
32
32
33
33
34
34
36
36
37
37
38
38
39
39
40
40
41
41
42
42
43
43
44
44
45
45
46
46
46
46
47
47
48
48
49
49
50
50
5151
5151
52
52
53
53
5454
55
55
55
55
56
56
57
57
57
57
58
58
59
59
60
60
60
60
61
61
62
62
62
62
63
63
64
64
64
64
67
67
70
70
73
73
76
76
79
79
81
81
84
84
86
86
88
88
91
91
18
21
24
24
26
26
28
30
30
32
32
34
34
35
35
37
37
39
39
40
40
42
42
42
42
44
44
45
45
46
46
47
47
49
49
50
50
51
51
52
52
53
53
54
54
55
55
56
56
57
57
58
58
59
59
60
60
6161
62
62
63
63
63
63
64
64
65
65
66
66
67
67
68
68
69
69
69
69
70
70
71
71
72
72
72
72
73
73
74
74
7575
7878
8282
8585
8888
9191
9494
9797
100
100
103
103
105
105
20
20
24
24
27
27
30
30
32
32
34
34
36
36
38
38
40
40
42
42
44
44
46
46
47
47
49
49
50
50
51
51
53
53
54
54
56
56
57
57
58
58
59
59
61
61
62
62
63
63
64
64
65
65
66
66
67
67
68
68
70
70
71
71
72
72
73
73
74
74
75
75
76
76
76
76
77
77
78
78
79
79
80
80
81
81
82
82
83
83
84
84
85
85
85
85
90
90
94
94
97
97
101
101
105
105
108
108
111
111
114
114
118
118
121
121
23
27
30
34
36
39
41
43
45
48
50
52
53
55
56
58
60
61
63
64
65
67
68
70
71
72
73
75
76
77
78
79
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
101
105
110
114
118
122
125
129
133
136
25
25
30
30
34
34
37
37
40
40
43
43
46
46
48
48
51
51
53
53
55
55
57
57
59
59
61
61
63
63
65
65
66
66
68
68
70
70
71
71
73
73
74
74
76
76
77
77
79
79
80
80
82
82
83
83
85
85
86
86
87
87
89
89
90
90
91
91
92
92
94
94
95
95
96
96
97
97
98
98
99
99
101
101
102
102
103
103
104
104
105
105
106
106
107
107
112
112
117
117
122
122
127
127
131
131
136
136
140
140
144
144
148
148
151
151
28
33
37
42
44
47
50
53
56
58
61
63
66
67
69
71
73
75
77
79
80
82
84
85
87
89
90
92
93
95
96
98
99
100
102
103
105
106
107
109
110
111
112
114
115
116
117
118
124
130
135
140
145
150
154
159
163
167
•Exact values adapted from Hollander and Wolfe (1973) are used for up to 15 panel
ists
blnterpolation may be used for uns|ipecifiefi table values involving more than 50 panelists.
125
TABLE 7.4
Critical Absolute Rank Sum Differences
for "All Treatments" Comparisons
at 1 % Level of Significance
Number of samples
Panelists
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
55
60
65
70
75
80
85
90
95
100
3
4
5
8
9
10
11
12
13
13
14
15
15
16
16
17
17
18
18
19
19
20
20
21
21
22
22
22
23
23
23
24
24
25
25
25
26
26
26
27
27
27
28
28
28
28
29
29
29
30
31
32
34
35
36
37
38
40
41
42
9
11
13
14
15
16
17
18
19
20
21
22
22
23
24
25
25
26
27
27
28
28
29
29
30
31
31
32
32
33
33
34
34
35
35
36
36
36
37
37
38
38
39
39
39
40
40
41
43
45
46
48
50
51
53
54
56
57
12
14
16
18
19
21
22
23
24
26
27
28
28
30
31
31
32
33
34
35
35
36
37
38
38
39
40
40
41
42
42
43
44
44
45
45
46
47
47
48
48
49
49
50
50
51
51
52
54
57
59
61
64
66
68
70
71
73
6 __ 7_
8
__ 9_
17
20
23
25
28
30
32
33
35
37
38
40
41
43
44
45
46
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
66
67
68
69
70
70
71
72
73
74
74
75
79
82
86
89
92
95
98
101
103
106
19
23
26
29
32
34
36
38
40
42
44
46
48
49
51
52
54
55
56
58
59
60
62
63
64
65
66
67
69
70
71
72
73
74
75
76
77
78
79
80
81
82
82
83
84
85
86
87
91
95
99
103
106
110
113
116
120
123
22
26
30
33
36
39
41
44
46
48
50
52
54
56
58
60
61
63
64
66
67
69
70
71
73
74
75
77
78
79
80
82
83
84
85
86
87
88
90
91
92
93
94
95
96
97
98
99
104
108
113
117
121
125
129
132
136
140
14
17
19
21
23
25
27
28
30
31
32
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
48
49
50
51
52
52
53
54
55
55
56
57
57
58
59
60
60
61
62
62
63
63
66
69
72
75
78
80
83
85
87
89
10
24
29
33
37
40
43
46
49
51
54
56
58
60
63
65
67
69
70
72
74
75
77
79
80
82
83
85
86
87
89
90
92
93
94
95
97
98
99
100
102
103
104
105
106
108
109
110
111
116
121
126
131
136
140
144
149
153
157
11
12
27
32
37
41
45
48
51
54
57
60
62
65
67
70
72
74
76
78
80
82
84
85
87
89
91
92
94
95
97
99
100
102
103
105
106
107
109
110
112
113
114
115
117
118
119
121
122
123
129
135
140
146
151
156
160
165
169
174
30
36
41
45
49
53
56
59
63
66
68
71
74
77
79
81
84
86
88
90
92
94
96
98
100
101
103
105
107
108
110
112
113
115
117
118
120
121
123
124
126
127
128
130
131
133
134
135
142
148
154
160
166
171
176
181
186
191
•Exact values adapted from Hollander and Wolfe (1973) are used for up to 15 panel
ists.
binterpoiation may be used for unspecified table values involving more than 50 panel
ists.
126
TABLE 7.5
F Distribution
5% Level of Significance
‘'i
i
2
3
4
5
6
8
7
9
‘'i
1
2
3
4
161-45
18-513
10-128
7-7086
199-50 215-71
230-16
224-58
19 164
19-000
19-247
19-296
9-5521
9-2766
9-1172
9 0135
6-5914
6-9443
6-3883
6-2560
5
6
7
8
9
6-6079
5-9874
5-5914
5-3177
5-1174
5-7861
5 1433
4 7374
4-4590
4-2565
5-4095
4 7571
4 3468
4 0662
3-8626
5-1922
4 5337
4-1203
3-8378
3-6331
5-0503
4-3874
3-9715
3-6875
3-4817
4-9503
4-2839
3-8660
3 5806
3 3738
4-8759
4-2066
3-7870
3-5005
3-2927
4-8183
4-1468
3-7257
3 4381
3-2296
4-7725
4-0990
3-6767
3-3881
3-1789
10
11
12
13
14
4-9646
4-8443
4-7472
4-6672
4-6001
4-1028
3 9823
3-8853
3-8056
3-7389
3-7083
3-5874
3-4903
3 4105
3 3439
3-4780
3-3567
3-2592
3-1791
3-1122
3 3258
3-2039
3 1059
3-0254
2-9582
3-2172
3-0946
2-9961
2-9153
2-8477
3 1355
3 0123
2 9134
2-8321
2-7642
3-0717
2-9480
2-8486
2-7669
2-6987
3-0204
2-8962
2-7964
2-7144
2 6458
15
16
17
18
19
4 5431
4-4940
4 4513
4-4139
4-3808
3 6823
3-6337
3 5915
3-5546
3 5219
3-2874
3-2389
3-1968
3 1599
3 1274
3-0556
3-0069
2-9647
2-9277
2-8951
2-9013
2-8524
2-8100
2-7729
2-7401
2-7905
2-7413
2-6987
2 6613
2-6283
2-7066
2-6572
2 6143
2-5767
2-5435
2-6408
2-5911
2-5480
2-5102
2-4768
2-5876
2-5377
2 4943
2 4563
2-4227
20
21
22
23
24
4 3513
4 3248
4-3009
4 2793
4-2597
3-4928
3-4668
3 4434
3 4221
3-4028
3-0984
3-0725
3 0491
3-0280
3-0088
2-8661
2-8401
2-8167
2-7955
2-7763
2-7109
2-6848
2 6613
2-6400
2-6207
2-5990
2-5727
2 5491
2-5277
2-5082
2-5140
2-4876
2-4638
2-4422
2 4226
2-4471
2-4205
2-3965
2 3748
2-3551
2-3928
2 3661
2-3419
2 3201
2-3002
25
26
27
28
29
4 2417
4-2252
4-2100
4-1960
4-1830
3 3852
3-3690
3 3541
3-3404
3-3277
2 9912
2-9751
2-9604
2-9467
2-9340
2--75S7
2-7426
2-7278
2-7141
2-7014
2-6030
2-5868
2 5719
2-5581
2-5454
2-4904
2 4741
2-4591
2-4453
2-4324
2-4047
2-3883
2 3732
2-3593
2-3463
2-3371
2-3205
2-3053
2-2913
2-2782
2-2821
2-2655
2-2501
2-2360
2-2229
30
40
60
120
4-1709
4-0848
4-0012
3-9201
3-8415
3 3158
3-2317
3-1504
3 0718
2-9957
2 9223
2-8387
2-7581
2-6802
2-6049
2-6896
2-6060
2-5252
2-4472
2-3719
2 5336
2-4495
2-3683
2-2900
2-2141
2-4205
2-3359
2-2540
2 1750
2-0986
2 3343
2-2490
2-1665
2-0867
2-0096
2-2662
2-1802
2-0970
2-0164
1-9384
2-2107
2-1240
2-0401
1-9588
1-8799
00
233 99
236-77
238-88
240-54
19-330
19 371
19-353
19 385
8-9406
8-8868 8-8452 8-8123
6-1631
6-0942
6-0410 6-9988
This table gives the values of F for which
, v,) = 0-05.
127
TABLE 7.5 (cont.)
F Distribution
5% Level of Significance
io
12
15
20
24
30
40
60
120
oo
1
2
3
4
241-88
243 91
245-95
248-01
19-396
19-413
19-429
19-446
8-7855 8-7446
8-7029
8-6602
5-9644
5 9117
5-8578
5-8025
249-05
19-454
8-6385
5-7744
250-09
19-462
8-6166
5-7459
251-14
252-20 253 25
19-471
19-479
19-487
8-5720 8-5494
8-5944
5-7170
5-6878
5-6581
254 32
19-496
8-5265
5-6281
5
6
8
9
4 7351
4-0600
3 6365
3 3472
3 1373
4-6777
3-9999
3 5747
3-2840
3 0729
4-6188
3-9381
3 5108
3 2184
3 0061
4-5581
3 8742
3 4445
3-1503
2-9365
4-5272
3-8415
3 4105
3 1152
2-9005
4-4957
3-8082
3-3758
3-0794
2-8637
4-4638
3-7743
3-3404
3-0428
2-8259
4 4314
3-7398
3-3043
3-0053
2-7872
4-3984
3-7047
3-2674
2-9669
2-7475
4-3650
3-6688
3 2298
2 9276
2-7067
10
11
12
13
14
2-9782
2 8536
2-7534
2-6710
2-6021
2-9130
2-7876
2-6866
2-6037
2 5342
2 8450
2-7186
2 6169
2 5331
2 4630
2-7740
2-6464
2 5430
2-4589
2-3879
2-7372
2-6090
2-5055
2-4202
2-3487
2-6996
2-5705
2-4663
2-3803
2-3082
2-6609
2-5309
2-4259
2-3392
2-2664
2 621 1
2-4901
2-3842
2-2966
2-2230
2-5SOI
2-4480
2 3410
2-2524
2-1778
2-5379
2 4 045
2-2962
2-2064
2-1307
15
16
17
18
19
2 5437
2 4935
2 4499
2 4117
2-3779
2 4753
2 4247
2-3807
2 3421
2-3080
2 4035
2 3522
2 3077
2-2686
2 2341
2 3275
2-2756
2-2304
2-1906
2-1555
2-2878
2-2354
2-1898
2-1497
2 1141
2-2468
2-1938
2-1477
2-1071
2-0712
2-2043
2-I5O7
2-1040
2-0629
2-0264
2-1601
2-1058
2-0584
2-0166
1-9796
2 I 141
2-0589
2-0107
I-9681
1-9302
2-0658
2-0096
1-9604
1-9168
1-8780
20
21
22
23
24
2 3479
2-3210
2-2967
2-2747
2 2547
2-2776
2-2504
2-2258
2 2030
2-1834
2-2033
2-1767
2-1608
2-1282
2 1077
2-1242
2-0900
2-0707
2 0476
2-0267
2-0825
2-0540
2-0283
2 0050
1-9838
2-0391
2-0102
1-9842
1-9605
1-9390
1-9938
1-9645
1-9380
1-9139
1-8920
1-9464
1-9165
1-8895
1-8649
1-8424
1-8963
1-8657
1-8380
1-8128
1-7897
1-8432
1-8117
1-7831
1-7570
1-7331
25
26
27
28
29
2 2365
2-2197
2 2043
2-1900
2 1768
2 1649
2-1479
2-1323
2-1179
2 1046
2-0889
2-0716
2-0558
2-0411
2-0276
2-0075
1-9898
1-9736
1-9586
1-9446
1-9643
1-9464
1-9299
1-9147
1-9005
1-9192
1-9010
1-8842
1-8687
1-8543
1-8718
1-8633
1-8361
1-8203
1-8055
1-8217
1-8027
1-7851
1-7689
I 7537
1-7684
1-7488
1-7307
1-7138
1-6981
1-7110
1-6906
1-6717
1-6541
1-6377
30
40
60
120
co
21646
2-0772
1-9926
1 9105
1-8307
2-0921
2-0036
1-9174
1-8337
1-7622
2-0148
1-9245
1-8304
1-7506
1-6604
1-9317
1-8389
1-7480
1-6687
1-6705
1-8874
1-7929
1-7001
1-6084
1-5173
1-8409
1-7444
1-6491
1-6543
1-4591
1-7918
1-6928
1-6943
1-4952
1-3940
1-7396
1-6373
1-5343
1-4290
1-3180
1-6835
1-6766
1-4673
1-3519
1-2214
1-6223
1-5089
1-3893
1-2539
1-0000
F=
=
"I Si
128
TABLE 7.6
F Distribution
1% Level of Significance
‘i
2
3
4
5
G
7
8
9
■'t
I 4999 5
1
2
3
4
4052-2
98-503 | 99-000
34-116 ! 30-817
21-198 I 18-000
5403-3
99-160
29-457
16-694
5624 6
99-249
28-710
15-977
5763-7
99-299
28-237
15-522
5859 0
99-332
27-911
15-207
5928-3
99-356
27-672
14 976
5981-6
99-374
27-489
14-799
6022 5
99-388
27-345
14-659
5
6
7
8
9
16-258
13-745
12 246
1 I 259
10-561
13-274
10-925
9-5466
8-6491
8-0215
12 060
9-7795
8-4513
7-5910
6-9919
11-392
9-1483
7-8467
7-0060
6 4221
10-907
8-7459
7-4604
0-6318
6-0569
10-672
8-4661
7-1914
6-3707
5-8018
10 456
8 2600
6-9928
6-1770
5-6129
10 289
8-1016
6-8401
6-0289
6-4671
10-158
7-9761
6-7188
5- 9106
6- 3511
10
1I
12
13
14
10-044 I
9-6460 |
7-5594
7-2057
9-3302 ! 6-9266
9 0738 ' 6-7010
8-8616
6 5149
0-5523
6 2167
6-9526
5-7394
5-5639
5 9943
5-6683
5-4119
5-2053
5 0354
5-6363
5-3160
5-0643
4 8616
4 6950
6-3858
5-0692
4-8206
4-6204
4-4558
5-200!
4 8861
4-6395
4-4410
4-2779
5-0567
4 7445
4 4994
4 3021
4-1399
4 9424
4 6315
4-3875
4-1911
4-0297
15
16
17
18
19
8-6831
8-5310
8-3997
8 2854
8-1850
6-3589
6 2262
6-1 121
(I <1129
5 9259
5-4170
5- 2922
6- 1850
5-0919
5-0103
4-8932
4-7720
4-0090
4-5790 I
4-5003
4-5556
4 4374
4-3359
4 2479
4-1708
4-3183
4 2016
4-1015
4-0146
3-9386
4 1415
4-0259
3-9207
3-8406
3-7653
4 <104 5
3-8896
3-7910
3-7O54
3-6305
3-8948
3-7804
3-6822
3 6971
3-5225
20
21
22
23
24
8-0960
8-0166
7-9454
7-881 I
7-8229
5-8489
5-7804
5-7190
5-0637
5-6136
4-9382
4-8740
4-8I66
4-7049
4-7181
4 4307
4-3688
4 3134
4 2635
4-2184
4-1027
4-0421
3-9880
3-9392
3-8961
3-8714
3-8117
3-7683
3-7102
3-0007
3-0987
3 0396
3-5807
3 6390
3 4959
3 5644
3-5050
3-4530
3-4057
3 3629
3 4567
3-3981
3-3468
3-2980
3-2560
25
20
27
28
29
7-7698
7-7213
7-6767
7-6356
7-5976
5-5680
5-5263
5-4881
5-4529
5-4205
4-6755
4-6366
4-6009
4-6681
4-5378
4-1774
4-1400
4-1050
4 0740
4-0449
3-8550
3 8183
3-7848
3-7639
3 7254
3-0272
3 6911
3-5580
3-6276
3 4995
3-4508
3'4210
3-3882
3 3681
3 3302
3-3239
3-2884
3-2558
3 2269
3 1982
3-2172
3-1818
3 1494
3 1195
3-0920
30
40
60
120
7-5625
7-3141
7-0771
6-8510
6 6349
5-3904
5-1785
4-9774
4-7865
4-6052
4-6097
4-3126
4-1259
3-9493
3-7816
4-0179
3-8283
3 6491
3-4796
3 3192
3-6990
3 5138
3-3389
3-1735
3-4736
3-2910
3-1187
2-9669
2-8020
3 3045
3 1238
2-9630
2 7918
2-6393
3-1720
2 9930
2-8233
2-0629
2-6113
3-0665
2-8876
2-7185
2-6686
2-4073
co
3-0173
Thia table gives the values of F for which
, p,)
001.
129
TABLE 7.6 (cont.)
F Distribution
1% Level of Significance
10
12
15
20
24
30
40
60
120
ao
1
o
3
4
6055-8
99 399
27-229
14 546
6106'3
99-416
27 052
14 374
6157-3
99 432
20-872
14-198
6208-7
99-449
26 690
14-020
6234 6
99-458
26-598
13 929
6260-7
99-466
26-505
13-838
6286-8
99-474
26 41 I
13-745
6313 0
99 483
26-316
13-652
6339-4
99 491
26'221
13-558
6366 0
99-501
26125
13-463
5
6
7
8
9
10051
7-8741
6-6201
5-8143
5-2565
9-8883
7-7183
6-4691
5'6668
6 1114
9 7222
7'5590
6-3143
5 5151
4-9621
9-5527
7-3958
61554
5 3591
4-8080
9-4665
7 3127
6 0743
5-2793
4-7290
9-3793
7-2285
5-9921
51981
4-6486
9-2912
7 1432
5-9084
5-1 156
4 5667
9-2020
7 0568
5-8236
5 0316
4 4831
9 II IS
6-969O
5-7372
4-9460
4 3978
9 0204
6-8801
5 6495
4-8588
4 3105
10
11
12
4-8492
4'5393
4-2961
13
14
4 1003
3 9394
4-7059
4 3974
4 1553
3 9603
3-8001
4-5582
4 2509
4 0096
3 8154
3 6557
4-4054
4 0990
3-8584
3-6646
3 5052
4 3269
4 0209
3-7SO5
3-5868
3 4274
4-2469
3-94 I I
3-7OO8
3 5070
3'3476
41653
3 8596
30192
3-4253
3-2656
4OSI9
3-7761
3 5355
1 34 13
11813
3 9965
3-6904
3 4494
3 2548
30942
3 9090
30025
3 36OS
3 1654
3 <1040 ’
15
16
17
18
19
3-8049
36909
3-5931
3-5082
3 4338
3-6062
3-5527
3 4552
3 3700
3'2005
3-5222
3-4089
3 3117
3 2273
3 1533
3-3719
3-2588
31615
3 0771
30031
3-2940
31808
3-0835
2-9990
2-9249
3-2141
3 1007
3 0032
2 9185
2-8442
31319
3-0182
2-9205
2-8354
2-7608
30471
2-9330
2 8348
2-7493
2 6742
2-9595
2-8447
2-7459
2-6597
2-5839
2'8684
2-7528
2-6530
2'5660
2-4893
20
21
22
23
24
3 3082
3-3098
3-2570
3 2100
3-1081
3 2311
31729
31209
3 0740
3 0310
30880
3 0299
2-9780
2 9311
2-8887
2-9377
2-8796
2-8274
2-7805
2-7380
2-8594
2-8011
2-7488
2-7017
2 6591
2-7785
2-7200
2-0675
2-62O2
2-5773
20947
2 6359
2-5831
2 5355
2 4923
2-6077
2-5484
2 4951
2 4471
2 4035
2-51C8
2-4568
2-4029
2 3542
2-3099
2-4212
2-3603
2-3055
2-2559
2-2107
25
26
27
28
29
31294
3 0941
3 0018
30320
3 0045
2 9931
2'9679
2-9250
2-8959
2-8085
2-8502
2-8150
2-7827
2 7530
2'7260
2-6993
2-0640
2-6316
2-6017
2-5742
2-6203
2-5848
2-6522
2-5223
2'4946
2-5383
2-5026
2-4699
2-4397
2 4118
2 4530
2 4170
2-3840
2 3535
2 3253
2-3037
2-3273
2-2938
2-2629
2-2344
2-2095
2-2325
21984
21670
21378
21694
2 1315
2 0905
2 0642
2 0342
30
40
CO
120
2-9791
2-8005
2 0318
2-4721
2-3209
2 8431
2-6048
2-4961
2-3303
2-1848
2-7002
2-6210
2 3523
21916
20386
2-5487
2-3089
21978
2 0346
1-8783
2'4689
2-2880
2-1154
1-9500
1-7908
2-3860
2-2034
2 0285
1-8600
10964
2 2992
21 142
1 9360
1-7028
1-6923
2-2079
20194
b8363
1'6657
1-4730
21 107
1-9172
1-7263
1-5330
1-3246
2-0062
1-8047
1-6000
1-3805
I 0000
vx
i
co
r= =
TABLE 7.7
o
Critical Values (Q Values) for Duncan’s New Multiple Range Test
5% I >evel of Significance
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
1
2
3
4
5
6
7
8
9
10
17.97
6.085
4.501
3.927
3.635
3.461
3.344
3.261
3.199
3.151
17.97
6.085
4.516
4.013
3.749
3.587
3.477
3.399
3.339
3.293
17.97
6.085
4.516
4.033
3.797
3.649
3.548
3.475
3.420
3.376
17.97
6.085
4.516
4.033
3.814
3.680
3.588
3.521
3.470
3.430
17.97
6.085
4.516
4.033
3.814
3.694
3.611
3.549
3.502
3.465
17.97
6.085
4.516
4.033
3.814
3.697
3.622
3.566
3.523
3.489
17.97
6.085
4.516
4 .033
3.814
3.697
3.626
3.575
3.536
3.505
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.544
3.516
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.522
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.525
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
ii
12
13
14
15
16
17
18
19
20
3.113
3.082
3.055
3.033
3.014
2.998
2.984
2.971
2.960
2.950
3.256
3.225
3.200
3.178
3.160
3.144
3.130
3.118
3.107
3.097
3.342
3.313
3.289
3.268
3.250
3.235
3.222
3.210
3.199
3.190
3.397
3.370
3.348
3.329
3.312
3.298
3.285
3.274
3.264
3.255
3.435
3.410
3.389
3.372
3.356
3.343
3.331
3.321
3.311
3.303
3.462
3.439
3.419
3.403
3.389
3.376
3.366
3.356
3.347
3.339
3.480
3.459
3.442
3.426
3.413
3.402
3.392
3.383
3.375
3.368
3.493
3.474
3.458
3.444
3.432
3.422
3.412
3.405
3.397
3.391
3.501
3.484
3.470
3.457
3.446
3.437
3.429
3.421
3.415
3.409
3.506
3.491
3.478
3.467
3.457
3.449
3.441
3.435
3.429
3.424
3.509
3.496
3.484
3.474
3.465
3.458
3.451
3.445
3.440
3.436
3.510
3.498
3.488
3.479
3.471
3.465
3.459
3.454
3.449
3.445
3.510
3.499
3.490
3.482
3.476
3.470
3.465
3.460
3.456
3.453
3.510
3.499
3.490
3.484
3.478
3.473
3.469
3.465
3.462
3.459
3.510
3.499
3.490
3.484
3.480
3.477
3.473
3.470
3.467
3.464
3.510
3.499
3.490
3.485
3.481
3.478
3.475
3.472
3.470
3.467
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.472
3.470
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.473
3.472
24
30
40
60
120
2.919
2.888
2.858
2.829
2.800
2.772
3.066
3.035
3.006
2.976
2.947
2.918
3.160
3.131
3.102
3.073
3.045
3.017
3.226
3.199
3.171
3.143
3.116
3.089
3.276
3.250
3.224
3.198
3.172
3.146
3.315
3.290
3.266
3.241
3.217
3.193
3.345
3.322
3.300
3.277
3.254
3.232
3.370
3.349
3.328
3.307
3.287
3.265
3.390
3.371
3.352
3.333
3.314
3.294
3.406
3.389
3.373
3.355
3.337
3.320
3.420
3.405
3.390
3.374
3.359
3.343
3.432
3.418
3.405
3.391
3.377
3.363
3.441
3.430
3.418
3.406
3.394
3.382
3.449
3.439
3.429
3.419
3.409
3.399
3.456
3.447
3.439
3.431
3.423
3.414
3.461
3.454
3.448
3.442
3.435
3.428
3 .465
3.460
3.456
3.451
3.446
3.442
3.469
3.466
3.463
3.460
3.457
3.454
P
oo
v = df(Error)
p = number of means within range being compared
TABLE 7.7 (cont.)
Critical Values (Q Values) for Duncan’s New Multiple Range Test
5% Level of Significance
20
22
24
26
28
30
32
34
36
38
40
50
60
70
80
90
100
1
2
3
4
5
6
7
8
9
10
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.035
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.636
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.574
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
17.97
6.085
4.516
4.033
3.814
3.697
3.626
3.579
3.547
3.526
11
12
13
14
15
16
17
18
19
20
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.473
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3 474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
3.510
3.499
3.490
3.485
3.481
3.478
3.476
3.474
3.474
3.474
24
30
40
60
120
3.471
3.470
3.469
3.467
3.466
3.466
3.475
3.477
3.479
3.481
3.483
3.486
3.477
3.481
3.486
3.492
3.498
3.505
3.477
3.484
3.492
3.501
3.511
3.522
3.477
3.486
3.497
3.509
3.522
3.536
3.477
3.486
3.500
3.515
3.532
3.550
3.477
3.486
3.503
3.521
3.541
3.562
3.477
3.486
3.504
3.525
3.548
3.574
3.477
3.486
3.504
3.529
3.555
3.584
3.477
3.486
3.504
3.531
3.561
3.594
3.477
3.486
3.504
3.534
3.566
3.603
3.477
3.486
3.504
3.537
3.585
3.640
3.477
3.486
3.504
3.537
3.596
3.668
3 477
3.486
3.504
3.537
3.600
3.690
3.477
3.486
3.504
3.537
3.601
3.708
3.477
3.486
3.504
3.537
3.601
3.722
3.477
3.486
3.504
3.537
3.601
3.735
P
oo
W
TABLE 7.8
Critical Values (Q Values) for Duncan’s New Multiple Range Test
1% Level of Significance
GJ
N>
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
1
2
3
4
5
6
7
8
9
10
90.03
14.04
8.261
6.512
6.702
5.243
4.949
4.746
4.596
4.482
90.03
14 04
8.321
6.677
5.893
5.439
5.145
4.939
4.787
4.671
90.03
14.04
8.321
6.740
5.989
5.549
5.260
5.057
4.906
4.790
90.03
14.04
8.321
6.756
6.040
5.614
5.334
5.135
4 .986
4.871
90.03
14.04
8.321
6.756
6.065
5.655
5.383
5.189
5.043
4.931
90.03
14.04
8.321
6.756
6.074
5.680
5.416
5.227
5.086
4.975
90.03
14.04
8.321
6.756
6.074
5.694
5.439
5.256
5.118
5.010
90.03
14 .04
8.321
6.756
6.074
5.701
5.454
5.276
5.142
5.037
90.03
14.04
8.321
6.756
6.074
5.703
5.464
5.291
5.160
5.058
90.03
14 .04
8.321
6.756
6.074
5.703
5.470
5.302
5.174
5.074
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.309
5.185
5.088
90 03
14 04
8 321
6.756
6.074
5.703
5.472
5.314
5.193
5.098
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.316
5.199
5.106
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.203
5.112
90.03
14 .04
8.321
6.756
6.074
5.703
5.472
5.317
5.205
5.117
90.03
14 .04
8.321
6.756
6.074
5 703
5.472
5.317
5.206
5.120
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.122
90 03
14 04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
ii
12
13
14
15
16
17
18
19
20
4.392
4.320
4.260
4.210
4.168
4.131
4.099
4.071
4 046
4.024
4.579
4.504
4.442
4.391
4.347
4.309
4.275
4.246
4.220
4.197
4.697
4.622
4.560
4.508
4.463
4.425
4.391
4.362
4.335
4.312
4.780
4.706
4.644
4.591
4.547
4.509
4.475
4 445
4.419
4.395
4.841
4.767
4.706
4.654
4.610
4.572
4.539
4.509
4.483
4.459
4.887
4.815
4.755
4.704
4.660
4.622
4.589
4.560
4.534
4.516
4.924
4.852
4.793
4.743
4.700
4.663
4.630
4.601
4.575
4.552
4.952
4.883
4.824
4.775
4.733
4.696
4.664
4.635
4.610
4.587
4.975
4.907
4 .850
4.802
4.760
4.724
4.693
4.664
4.639
4.617
4.994
4.927
4.872
4.824
4.783
4.748
4 717
4.689
4.665
4.612
5.009
4 .944
4.889
4.843
4.803
4.768
4.738
4.711
4.686
4.664
5.021
4.958
4.901
4.859
4.820
4.786
4.756
4.729
4.705
4.684
5.031
4.969
4 .917
4.872
4.834
4.800
4.771
4.745
4.722
4.701
5.039
4 978
4.928
4.884
4.846
4.813
4.785
4.759
4.736
4 .716
5.045
4 .986
4.937
4.894
4.857
4.825
4 .797
4.772
4.749
4.729
5.030
4.993
4.944
4.902
4.866
4.835
4.807
4.783
4.761
4.741
5.054
4.998
4.950
4.910
4.874
4.844
4.816
4.792
4.771
4.751
5.057
5.002
4.956
4.916
4.881
4.851
4.824
4.801
4.780
4.761
24
30
40
60
120
oo
3.956
3.889
3.825
3.762
3.702
3.643
4.126
4.056
3.988
3.922
3.858
3.796
4.239
4.168
4.098
4.031
3.965
3.900
4.322
4.250
4.180
4.111
4.044
3.978
4.386
4.314
4.244
4.174
4.107
4 .010
4.437
4.366
4.296
4.226
4.158
4.091
4.480
4.409
4.339
4.270
4.202
4.135
4.516
4.445
4.376
4.307
4.239
4.172
4.546
4.477
4.408
4 .340
4.272
4.205
4.573
4.504
4.436
4.368
4.301
4.235
4.596
4.528
4.461
4.394
4.327
4.261
4.616
4.550
4.483
4 .417
4.351
4.285
4.634
4.569
4.503
4.438
4.372
4.307
4.651
4.586
4.521
4 .456
4.392
4.327
4.665
4.601
4.537
4.474
4.410
4.345
4.678
4.615
4.553
4.490
4.426
4.363
4.690
4.628
4.566
4.501
4.412
4.379
4.700
4.640
4.579
4.518
4.456
4.394
P
v = dF(Error)
p = number of means within range being compared
TABLE 7.8 (cont.)
Critical Values (Q Values) for Duncan’s New Multiple Range Test
1% Level of Significance
20
22
24
26
28
30
32
34
36
38
40
50
60
70
80
90
100
1
2
3
4
5
6
7
8
9
10
90.03
14 .04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14 .04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90 03
14 04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6 756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
90.03
14.04
8.321
6.756
6.074
5.703
5.472
5.317
5.206
5.124
ii
12
13
14
15
16
17
18
19
20
5.059
5.006
4.960
4 .921
4.887
4 .858
4.832
4.808
4.788
4.769
5.061
5.010
4.966
4.929
4.897
4 .869
4 .844
4.821
4.802
4.784
5.061
5.011
4.970
4.935
4.904
4 .877
4 .853
4.832
4.812
4.795
5.061
5.011
4.972
4.938
4.909
4.883
4 .860
4.839
4.821
4.805
5.061
5.011
4.972
4.940
4.912
4.887
4 .865
4.846
4.828
4.813
5.061
5.011
4.972
4.940
4.914
4.890
4.869
4.850
4.833
4.818
5.061
5.011
4.972
4.940
4 .914
4.892
4.872
4.854
4.838
4.823
5.061
5.011
4.972
4.940
4.914
4.892
4.873
4.856
4.841
4.827
5.061
5.011
4.972
4.940
4.914
4.892
4 874
4.857
4.843
4.830
5.061
5.011
4.972
4 .940
4.914
4.892
4 .874
4 .858
4.844
4 832
5.061
5.011
4.972
4.940
4.914
4 .892
4 .874
4.858
4.845
4.833
5.061
5.011
4.972
4.940
4 .914
4.892
4 .871
4.858
4 .845
4.833
5 061
5.011
4.972
4.940
4 .914
4 892
4 .874
4.858
4.845
4.833
5.061
5.011
4.972
4 .940
4.914
4.892
4 .874
4.858
4.845
4.833
5.061
5.011
4.972
4 940
4.914
4 892
4.874
4.858
4.845
4.833
5.061
5.011
4.972
4.940
4.914
4.892
4.874
4.858
4.845
4.833
5.061
5.011
4.972
4.940
4.914
4.892
4 .874
4.858
4.845
4.833
24
30
40
60
120
4 710
4 .650
4.591
4.530
4.469
4.408
4 .727
4.669
4.611
4.553
4 .494
4 .434
4.741
4.685
4.630
4.573
4.516
4.457
4 .752
4.699
4.645
4.591
4.535
4 .478
4.762
4.711
4.659
4.607
4.552
4.497
4 .770
4.721
4.671
4.620
4.568
4.514
4.777
4.730
4.682
4.633
4.583
4.530
4.783
4.738
4.692
4.645
4.596
4.545
4.788
4.744
4.700
4.655
4.609
4 .559
4 .791
4.750
4.708
4 .665
4 .619
4.572
4.794
4.755
4.715
4.673
4.630
4.584
4.802
4 .772
4.740
4.707
4.673
4.635
4 .802
4.777
4 .754
4 .730
4.703
4.675
4.802
4.777
4.761
4 .745
4.727
4.707
4.802
4 .777
4 764
4.755
4 745
4.734
4 .802
4.777
4.764
4.761
4 759
4 756
4 .802
4 .777
4.764
4.765
4.770
4 776
P
oo
CO
W
TABLE 7.9
One-Tailed Binomial Test
Probability of X or more correct judgements in n trials (p=l/3)
X 0
868
912
941
961
974
983
988
992
995
997
998
998
999
999
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
2b
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
2
3
4
5
539
649
210
320
429
532
623
701
76b
819
861
895
921
941
956
967
976
982
987
991
993
995
996
997
998
999
999
999
045
100
173
259
350
441
527
607
678
739
791
834
870
898
921
940
954
965
974
980
985
989
992
994
996
997
998
998
999
999
999
004
018
045
088
145
213
289
368
448
524
596
661
719
737
805
857
896
925
946
961
973
981
986
990
993
995
997
998
998
999
999
999
769
812
848
879
904
924
941
954
964
972
979
984
988
991
993
995
996
997
998
998
999
999
999
6
001
007
020
042
077
122
178
241
310
382
453
522
588
648
703
751
794
831
862
888
910
928
943
955
965
972
978
983
987
990
992
994
996
997
997
998
999
999
999
999
003
008
020
039
066
104
149
203
263
326
391
457
521
581
638
690
737
7 78
815
847
874
897
916
932
946
957
965
973
978
963
987
990
992
994
995
996
997
998
998
999
999
999
999
* NOTE: Initial decimal point hat been omitted
w
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
001
003
009
019
035
058
088
126
172
223
279
339
399
460
519
576
630
679
725
766
801
833
861
885
905
922
937
949
959
967
973
979
983
987
990
992
994
995
996
997
998
998
001
004
009
017
031
050
075
108
146
191
240
293
349
406
462
518
572
623
670
714
754
789
821
849
873
895
913
928
941
952
961
968
974
980
984
987
990
992
994
995
001
002
004
008
016
027
043
065
092
125
163
206
254
304
357
411
464
517
568
617
662
705
744
779
810
838
863
885
903
920
933
945
955
963
970
976
980
984
987
001
002
004
008
014
024
038
056
079
107
140
178
220
266
314
364
415
466
516
565
612
656
697
735
769
800
829
854
876
895
912
926
938
949
958
965
972
001
002
004
007
013
021
033
048
068
092
121
154
191
232
276
322
370
419
468
516
562
607
650
689
726
761
791
820
845
867
887
904
919
932
943
001
002
004
007
012
019
028
042
058
079
104
133
166
203
243
285
330
376
422
469
515
560
603
644
683
719
753
783
811
836
859
879
896
001
002
003
006
010
016
025
036
050
068
090
115
144
177
213
252
293
336
381
425
470
515
558
600
639
677
713
745
776
803
829
001
GO2
003
006
009
014
022
031
043
059
078
100
126
155
187
223
261
301
342
385
428
471
514
556
596
635
672
706
739
001
002
003
005
008
013
019
027
038
051
067
087
109
135
164
196
231
268
307
347
389
430
472
514
554
593
631
001
002
003
005
007
011
016
023
033
044
058
075
095
118
144
173
205
239
275
313
352
392
433
473
513
001
001
002
004
006
010
014
020
028
038
051
066
083
104
127
153
182
213
246
282
318
356
395
001
001
002
004
006
009
012
018
025
033
044
057
073
091
111
135
161
189
220
253
287
001
001
002
003
005
007
011
016
021
029
038
050
063
079
098
119
142
168
196
001
001
002
003
004
007
010
014
019
025
033
043
055
070
086
105
126
001
001
002
003
004
006
008
012
016
022
029
038
048
061
076
23
001
001
001
002
003
005
007
010
014
019
025
033
042
24
25
26
27
28
001
001
002
003
004
006
009
012
017
022
001
001
002
003
004
006
008
011
001
001
002
002
003
005
001
001
001
002
001
TABLE 7.10
Percentage Points of the Studentized Range
Upper 5% Points
\n
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
1
2
3
4
15-0 27 0
6-09
8-3
4-50 5 91
3-93 5-04
32-8
9-8
6-82
5-76
37-1
10-9
7-50
6-29
40-4
11-7
804
6-71
43 1
12-4
8-48
7-05
45-4
130
885
7-35
47-4
13-5
918
7-60
491
14-0
9-46
7-83
506
14-4
9-72
8-03
52-0
14 7
9-95
8-21
53-2
15 1
10-15
8-37
54-3
15-4
10-35
8-52
55-4
15-7
10-52
8-66
56 3
159
10-69
8-79
57-2
16 1
10-84
8 91
580
16-4
10-98
903
58-8
16 6
11-11
9 13
59 6
16-8
1121
9-23
5
6
7
8
9
3-64
3-46
3-34
3 26
3-20
4-60
4-34
4-16
4-04
3-95
5 22
4-90
4-68
4 53
4-42
5-67
5-31
506
4-89
4-76
603
5-63
5-36
517
502
6-33
5 89
5-61
540
5-24
6-58
612
5-82
5-60
5-43
6-80
6-32
600
5-77
5-60
6-99
6-49
6-16
5-92
5-74
7-17
6-65
6-30
6-05
5-87
7-32
6-79
6-13
6-18
5-98
7-47
6-92
655
6-29
6-09
7-60
7-01
6-66
639
6-19
7-72
7 14
6-76
6-48
6-28
7-83
7-24
6-85
6-57
6-36
7-93
7-31
6-94
6-65
641
8-03
7-13
7-02
6-73
6-51
8-12
7-51
7 09
6 80
6-58
8-21
7 59
7-17
6-87
664
10
11
12
13
14
3-15
3 11
3-08
3-06
3-03
3-88
3-82
3-77
3-73
3-70
4-33
4-26
4-20
4-15
4 11
4-65
4-57
4-51
4 45
4-41
4-91
4-82
4-75
4-69
4-64
512
503
4-95
4-88
4-83
5-30
5-20
512
505
4-99
5-46
535
527
519
513
5-60
5-49
5-40
5-32
5-25
5-72
5-61
5-51
5-43
5-36
5-83
5-71
562
5-53
5-46
5-93
5-81
571
5-63
5-55
6-03
5-90
5-80
5-71
5-64
6-11
5-99
5-88
5-79
5-72
6-20
6-06
5-95
5 86
5-79
6-27
6-14
6-03
5-93
5-b-5
6-34
620
609
600
5-92
6-40
626
6 15
6-05
5-97
6-47
6-33
6-21
6-11
6-03
15
16
17
18
19
3-01
3-00
2-98
2-97
2-96
3-67
3-65
3-63
3-61
3-59
4-08
4-05
4-02
4 00
3-98
4-37
4-33
4 30
4-28
4-25
4-60
4 56
4-52
4-49
4-47
4-78
4-74
4 71
4-67
4-65
4-94
4 90
4-86
4-82
4-79
508
503
4-99
4-96
4-92
5-20
5-15
5-11
5-07
5-04
5 31
5-26
5-21
5-17
5-14
5-40
5-35
5-31
5-27
5-23
5-49
5-44
5-39
5-35
5-32
5-58
5-52
5-47
5-43
5-39
5-65
5-59
5 55
5-50
5-16
5-72
5-66
5-61
5-57
5-53
5-79
5-72
5-68
5-63
5-59
5-85
5-79
5-74
5-69
5-65
5 90
5 84
5-79
5-74
5-70
5-96
5-90
5-81
5-79
5-75
20
24
30
40
2-95
2-92
2-89
2-86
3-58
3-53
3-49
3-44
3-96
3 90
3-84
3-79
4-23
4-17
4-10
404
4-45
4-37
4-30
4-23
4-62
4-54
4-46
4-39
4-77
4-68
4-60
4-52
4-90
4-81
4-72
4-63
5-01
4-92
4-83
4-74
511
5-01
4-92
4-82
5-20
5-10
5-00
4-91
5-28
5-18
5-08
4-98
5-36
5-25
5-15
5-05
5-43
5-32
5-21
5-11
5-49
5-38
5-27
5-16
5-55
5 44
5-33
5-22
5-61
5-50
5-38
5-27
5-66
5-54
5-43
5-31
5-71
5 59
5-48
5-36
60
120
2-83
2-80
2-77
3-40
3-36
3-31
3-74
3-69
3-63
3-98
3-92
3-86
4-16
4-10
4-03
4-31
4-24
4-17
4-44
4-36
4-29
4-55
4-48
439
4-65
4-56
4-47
4-73
4 6-1
4-55
4 81
4-72
4-62
4-88
4-78
4-68
4-94
4-8-1
4-74
5-00
4-90
4-80
5-06
4-95
4-85
511
500
4-89
5-16
5-05
4-93
5-20
5-09
4-97
5-24
5-13
5-01
CO
n is tLe size of sample from which the range is obtained and v is the number of degrees of freedom of sv.
w
U1
w
o>
TABLE 7.11
Percentage Points of the Studentized Range
Upper 1% Points
X
2
3
4
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
1
2
3
4
164
90-0 135
186
202
190
26-6
140
22-3
24-7
14-2
8-26 10-G
12-2
13-3
6 51
8-12
9-96 10-6
9-17
216
28-2
15-0
11-1
227
29-5
15-6
11-5
237
30-7
16-2
11-9
216
31-7
16-7
12-3
253
32-6
17 1
12-6
2G0
33-4
17-5
12-8
266
34-1
17-9
13-1
272
~3l-8
18-2
13 3
277
35-4
18-5
13-5
282
36-0
18-8
13-7
286
36-5
19-1
139
290
37-0
19-3
14-1
291
37-5
19-5
14-2
298
37-9
19 8
11-4
5
6
7
8
9
5-70
5 24
4-95
4-74
4-60
6-97
6 33
5-92
563
5-43
7-80
7 03
6-54
6-20
5-96
8-42
7-56
7-01
G-63
6-35
8-91
7-97
7-37
6-96
6-66
9-32
8-32
7-68
7-24
6-91
9-67
8-61
7-94
7-47
7-13
9-97
8-87
8-17
7-68
7-32
10-24
9-10
8-37
7-87
7-49
10-48
9-30
8-55
803
7-65
10-70
9 49
8-71
8-18
7-78
10-89
9-65
8-8G
8-31
7-91
11-08
9-81
9-00
8-44
8-03
11-24
9 95
9-12
8-55
8-13
11-40
10-08
9-21
8-66
8-23
11-55
10-21
9-35
8-76
8-32
11-68
10-32
9-46
8-85
8-41
11 81
10-43
9 55
8-94
8-49
11-93
10-54
9-65
9-03
8-57
10
11
12
13
14
4-48
439
432
426
4-21
5-27
514
5-04
4-96
4-89
5-77
5-62
5-50
5-40
5-32
6-14
5-97
5-84
5-73
5-63
6-43
6-25
6-10
5-98
5-88
6-67
6-48
6-32
6-19
6-08
687
6-67
6-51
6-37
6-2G
7-05
6-84
6-67
6-53
G-41
7-21
6-99
6-81
6-67
6-54
7-36
7-13
6-94
6-79
6-G6
7-48
7-25
7-06
6-90
6-77
7-60
7-36
7-17
7 01
6-87
7-71
7-46
7-26
7-10
6-96
7-81
7-56
7-36
7-19
7-05
7-91
7-65
7-44
7-27
7-12
7-99
7-73
7-52
7-3-1
7-20
8-07
7-81
7-59
7-42
7-27
8-15
7-88
7-66
7-48
7-33
8-22
7-95
7-73
7-55
7-39
15
16
17
18
19
417
4 13
4 10
407
405
4-83
4-78
4-74
4-70
4-67
5-25
5-19
5-14
5-09
505
5-56
5-49
5-43
5-38
5-33
5-80
5-72
5-66
5-60
5-55
5-99
5-92
5-85
5-79
5-73
6-16
6-08
6-01
5-94
5-89
6-31
6-22
6-15
608
6-02
6-44
6-35
6-27
6-20
6-14
6-55
6-46
6-38
6-31
6-25
6-66
656
6-48
6 41
6-3-1
6-76
666
6-57
6-50
643
6-84
6-74
6-6G
6-58
6-51
6-93
6-82
6-73
6-65
6-58
7-00
6-90
6-80
6-72
6-65
7-07
6-97
6-87
6-79
6-72
7-14
7-03
6-94
6-85
6-78
7-20
709
7 00
6 91
6-84
7-26
7-15
7 05
6-96
6-89
20
24
30
40
402
3-96
3-89
3-82
4-64
4-54
4-45
4-37
5-02
4-91
4-80
4-70
5-29
5-17
5-05
4-93
5-51
5-37
5-24
5-11
5-69
5-54
5-40
5-27
5-84
569
5-54
5-39
5-97
5 81
5-65
550
609
5-92
5-76
5-60
6-19
6-02
5-85
5-69
6-29
6-11
5-93
5-77
6-37
6-19
6-01
5-84
6-45
6-26
6-08
590
6-52
6-33
6-14
5-96
6-59
6-39
6-20
602
6-65
6-45
6-26
6-07
6-71
6-51
631
6-12
6-76
6-56
6-36
6-17
6-82
6-G1
6-41
6-21
60
120
3-76
3-70
3-64
4-28
4-20
4-12
4-60
4-50
4-40
4-82
4-71
4-GO
4-99
4-87
4-76
5-13
5-01
4-88
5-25
5-12
4-99
5-36
5 21
5-08
5-45
5-30
5-16
5-53
5-38
5-23
5-60
5-4-1
5-29
5-67
5-51
5-35
5-73
5-56
5-40
5-79
5-61
5-45
5-84
5-66
5-49
5-89
5-71
5-54
5-93
5-75
5-57
5-98
5-79
5-61
6-02
5-83
5-65
oo
5
n is the size of the sample from which the range is obtained and v is the number of degrees of freedom of s^.
References
1
1
I
1
139
REFERENCES CITED
4
Amerine, M.A., Pangborn, R.M. and Roessler, E.B. 1965.
"Principles
Sensory Evaluation of Food." Academic Press, New York.
/
I
of
ASTM Committee E-18,1986.
Physical requirement guidelines for sensory
evaluation laboratories. STP 913. Am. Soc. for Testing and Materials,
Philadelphia, Pa.
ASTM Committee E-18,1981.
Guidelines for the selection and training of
sensory panel members. STP 758. Am. Soc. for Testing and Materials,
Philadelphia, Pa.
Manual on consumer sensory evaluation. STP
ASTM Committee E-18,1979.
682. Am. Sex:, for Testing and Materials, Philadelphia, Pa.
ASTM Committee E-18,1968.
Manual on sensory testing methods. STP 434.
Am. Soc. for Testing and Materials, Philadelphia, Pa.
ASTM Committee E-18,1968.
Basic principles of sensory evaluation. STP
433. Am. Soc. for Testing and Materials, Philadelphia, Pa.
Basker, D. 1988.
Critical values of differences among rank sums for multiple
comparisons. Food Technol. 42(2):79.
Brandt, M.A., Skinner, E.Z. and Coleman, J.A. 1963.
thod. J. FoodSci. 28:404.
Texture profile me-
Cairncross, S.E. and Sjostrom, L.B. 1950.
Flavor profiles - A new approach
to flavor problems. Food Technol. 4(8):308.
Cardello, A.V. and Mailer, O. 1982
Relationships between food preferences
and food acceptance ratings. J. FoodSci. 47:1553.
140
Caul,J.F. 1957.
7:1.
The profile method of flavor analysis. Adv. Food Research
Civille, G.V. and Szczesniak, A.S. 1973.
Guidelines to training a texture
profile panel. J. Texture Studies 4:204.
Daget, N. 1977.
Sensory evaluation or sensory measurement? In "Nestle Re
search News lyienT. C. Boella (Ed.). Nestle Products Technical As
sistance Co. Ltd., Switzerland.
Ennis, D.M., Boelens, H., Haring, H. and Bowman, P. 1982.
Multivariate
analysis in sensory evaluation. Food Technol. 36(11):83.
Gacula, M.C. Jr. and Singh, J. 1984.
"Statistical Methods in Food and Con
sumer Research." Academic Press, Inc., New York.
IFT Sensory Evaluation Division. 1981.
Sensory evaluation guide for testing
food and beverage products. Food Technol. 35(ll):50.
Jellinek, G. 1985.
"Sensory Evaluation of Food. Theory and Practice." Ellis
Horwood, Chichester, England.
Joanes, D.N. 1985.
On a rank sum test due to Kramer. J. FoodSci. 50:1442.
Larmond, E. 1977. "Laboratory Methods for Sensory Evaluation of Food."
Research Branch, Canada Department of Agriculture, Ottawa. Publication
1637.
McPherson, R.S. and Randall, E. 1985.
Line length measurement as a tool
for food preference research. Ecol. FoodNutr. 17(2):149.
Mcilgaard, M. C., Civille, G.V. and Carr, B.T. 1987.
"Sensory
Evaluation
Techniques," Vols. I and II. CRC Press, Inc., Boca Raton, Florida.
Moskowitz, H.R. 1983.
"Product Testing and Sensory Evaluation of Foods,
Marketing and R & D Approaches." Food and Nutrition Press, Inc.,
Westport, Connecticut.
Newell, G.J. and MacFarlane, J.D. 1987.
Expanded tables for multiple com
parison procedures in the analysis of ranked data. ./. FoodSci. 52:1721.
141
O’Mahony, M. 1986.
"Sensory Evaluation of Food. Statistical Methods and
Procedures." Marcel Dekker, Inc., New York, N.Y.
O’Mahony, M. 1982.
Some assumptions and difficulties with common
statistics for sensory analysis. Food Technol. 36(11):75.
Pangborn, R.M.V. 1986.
Sensory techniques of food analysis. In "Food
Analysis Principles and Techniques. Vol. 1. Physical Characterization,"
D.W. Gruenwedcl and J.R. Whitaker, (Ed.). Marcel Dekker, Inc., New
York, N.Y.
Piggott, J.R. (Ed.), 1984.
"Sensory Analysis of Foods."
Science Publishers, London, England.
Elsevier Applied
Powers, J.J. 1984.
Using general statistical programs to evaluate sensory
data. Food Technol. 38(6):74.
Powers, J.J. 1981.
Multivariate procedures in sensory research: scope and
limitations. MBAA Technical Quarterly 18(1):11.
Snedecor, G.W. and Cochran, W.G. 1980.
"Statistical Methods," 7th ed.
Iowa State University Press, Ames, Iowa.
Steel, R.G.D. and Torrie, J.H. 1980.
"Principles and Procedures of Statis
tics," 2nd ed, McGraw-Hill, New York, N.Y.
Stone, H. and Sidel, J.L. 1985.
"Sensory Evaluation Practices."
Press, Inc., New York, N.Y.
Stone, H., Sidel, J.L. and Bloomquist, J. 1980.
Quantitative
analysis. Cereal Foods World 25(10):642.
Academic
descriptive
Stone, H., Sidel, J.L., Oliver, S., Woolsey, A. and Singleton, R.C. 1974. Senso
ry evaluation by quantitative descriptive analysis. Food Technol.
28(11):24.
Stungis, G.E. 1976.
Overview of applied multivariate analysis. In "Cor
relating Sensory Objective Measurements - New Methods for Answering
Old Problems". ASTM STP 594, Am. Soc. for Testing and Materials,
Philadelphia, Pa.
142
Szczesniak, A.S. 1963.
Sci. 28:385.
Classification of textural characteristics. J. Food
Szczesniak, A.S., Brandt, M.A. and Friedman, H.H. 1963.
Development of
standard rating scales for mechanical parameters of texture and correla
tion between the objective and the sensory methods of texture evaluation.
J. Food Sci. 28:397.
Zook, K. and Wessman, C. 1977.
The selection and use of judges for
descriptive panels. Food Technol. 31(11):56.
ADDITIONAL REFERENCES
AMSA. 1978.
Guidelines for cookery and sensory evaluation of meat.
American Meat Science Association and National Live Stock and Meat
Board.
Bieber, S.L. and Smith, D.V. 1986.
Multivariate analysis of sensory data: A
comparison of methods. Chemical Senses 11(1):19.
Bourne, M.C. 1978.
Texture profile analysis. Food Technol. 32(7):62.
Cardello, A.V., Mailer, O., Kapsalis, J.G., Segars, R.A., Sawyer, F.M., Murphy,
C. and Moskowitz, H.R. 1982.
Perception of texture by trained and
consumer panelists. J. FoodSci. 47:1186.
Civille, G.V. 1978.
Case studies demonstrating the role of sensory evalua
tion in product developments. Food Technol. 32(11):59.
Cross, H.R., Moen, R. and Stanfield, M.S. 1978.
Training and testing of
judges for sensory analysis of meat quality. Food Technol. 32(7):48.
Gatchalian, M.M. 1981.
"Sensory Evaluation Methods with Statistical
Analysis." College of Home Economics, University of the Philippines,
Diliman, Philippines.
Larmond, E. 1973.
27(11):28.
Physical requirements for sensory testing. Food Technol.
143
Moskowitz, H.R. 1978.
Magnitude estimation: notes on how, what, where
and why to use it. J. Food Quality 1:195.
Moskowitz, H.R. 1974.
Sensory evaluation by magnitude estimation. Food
Technol. 28(11):16.
Roessler, E.B., Pangborn, R.M., Sidel, J.L. and Stone, H. 1978.
Expanded
statistical tables for estimating significance in paired-preference, paired
difference, duo-trio and triangle tests. J. FoodSci. 43:940.
Sidel, J.L. and Moskowitz, H.R. 1971.
Magnitude and hedonic scales of
food acceptability. J. Food Sci. 36:677.
Sidel, J.L. and Stone, H. 1976.
Experimental design and analysis of sensory
tests. Food Technol. 30(ll):32.
Sidel, J.L., Stone, H. and Bloomquist, J. 1981.
Use and misuse of sensory
evaluation in research and quality control. J. Dairy Sci. 64:2296.
Szczesniak, A.S., Lowe, B.J. and Skinner, E.L. 1975.
profile technique. J. FoodSci. 40:1253.
Consumer
textural
Wolfe, K.A. 1979.
Use of reference standards for sensory evaluation of
product quality. Food Technol. 33(9):43.
Glossary
J
147
Explanations and definitions of terms used in this glossary were taken
from the following references:
Amerine, M.A., Pangborn, R.M. and Roessler, E.B. 1965. "Principles of
Sensory Evaluation ofFood." Academic Press, New York, N.Y.
Davies, P. (Ed.) 1970. "The American Heritage Dictionary of the English
Language." Dell Publishing Co., Inc. New York, N.Y.
Gacula, M.C. Jr. and Singh, J. 1984. "Statistical Methods in Food and
Consumer Research." Academic Press, Inc., Orlando, Florida.
Guralnik, D.B. (Ed.) 1963. "Webster’s New World Dictionary." Nelson,
Foster & Scott Limited, Toronto.
Hirsh, N.L. 1971. Sensory good sense. Food Product Devel. 5(6):27-29.
Huck, S.W., Cormier, W.H. and Bounds, W.GJr. 1974. "Reading Statistics
and Research." Harper & Row Publishers, New York, N.Y.
IFT Sensory Evaluation Division. 1981. Sensory evaluation guide for
testing food and beverage products. Food Technol. 35(ll):50.
Kramer, A. 1959. Glossary of some terms used in the sensory (panel)
evaluation of foods and beverages. Food Technol. 13(12):733-736.
Linton, M. and Gallo, P.S.Jr. 1975. "The Practical Statistician: Simplified
Handbook of Statistics." Brooks/Colc Publishing Company, California.
Lowry, S.R. 1979. Statistical planning and designing of experiments to
detect differences in sensory evaluation of beef loin steaks. J. FoodSci.
44:488-491.
O’Mahoney, M. 1986. "Sensory Evaluation of Food. Statistical Methods
and Procedures." Marcel Dekker, Inc., New York, N.Y.
Piggott, J.R. (Ed.), 1984. "Sensory Analysis of Foods" Elsevier Applied
Science Publishers, London, England.
148
Terms and Definitions
Acceptability (n) - Attitude towards a product, expressed by a consumer,
often indicating its actual use (purchase or eating).
Accuracy (n) - Closeness with which measurements taken approach the true
value; exactness; correctness.
Acuity (n) - Fineness of sensory recognition or discrimination; ability to
discern or perceive small differences in stimuli; sharpness or acuteness.
Affective Test - A test used to evaluate subjective attitudes such as
preference, acceptance and/or degree of liking of foods by untrained
panelists.
Aftertaste (ri) - The experience, which under certain conditions, follows the
removal of a taste stimulus.
Ageusia (n) - Lack or impairment of sensitivity to taste stimuli.
Analysis of Variance - A mathematical procedure for segregating the
sources of variability affecting a set of observations; used to test
whether the means of several samples differ in some way or are the
same.
Analytical Test - A test used for laboratory evaluation of products by
trained panelists in terms of differences or similarities and for
identification and quantification of sensory characteristics.
Anosmia (n) - Lack or impairment of sensitivity to odour stimuli.
Arbitrary (adj) - Based on or subject to personal or individual judgment.
Assess (v) - To evaluate.
Attribute (n) - A perceived characteristic; a distinctive feature, quality or
aspect of a food product.
Ballot (ri) - A form used by a panelist to record sample scores, decisions,
comments; usually includes instructions to the panelist related to the
type of test being performed.
149
Basic Taste - Sweet, sour, salty or bitter sensation.
Batch (n) - A definite quantity of some food product chosen from the
population of that food, and from which samples are withdrawn.
Bias (w) - A prejudiced or influenced judgment.
Binomial Test - A test of the frequency of occurrence in two categories;
used when only two possible outcomes are allowed.
Blind Control - A reference sample, whose identity is known only to the
researcher, coded and presented with experimental samples.
Category (n) - A defined division in a system of classification.
Category Scale - A scale divided into numerical and/or descriptive
classifications.
Characteristic (n) - Odour, flavour, texture or appearance property of a
product.
Chi-Square Test - Non-parametric test used to determine whether a
significant difference exists between an observed number and an
expected number of responses falling in each category designated by
the researcher; used to test hypotheses about the frequency of
occurrence in any number of categories.
Classification (n) - Category.
Classify (v) - To sort into predetermined categories.
Code (v) - Assignment of symbols, usually 3-digit random numbers, to
samples so that they may be presented to panelists without
identification.
Conditional (adj) - Implying a condition or prerequisite.
Conservative (adj) - Moderate; cautious.
Consistency (n) - Agreement or harmony of parts; uniformity.
150
Consumer (n) - An individual who obtains and uses a commodity.
Consumer Panel - A group of individuals representative of a specific
population whose behaviour is measured.
Conventional (adj) - Approved by or following general usage; customary.
Correlation Analysis - A method to determine the nature and degree of
relationship between variables.
Critical Value - A criterion or scientific cut-off point related to the chosen
level of significance.
Definition (ri) - A statement of the meaning of a word, phrase or term; the
act of making clear and distinct.
Descriptive Test - A test used to measure the perceived intensity of a
sensory property or characteristic.
Difference Test - A test used to determine if two samples are perceptibly
different.
Discriminate (v) - To perceive or detect a difference between two or more
stimuli.
Effect (/?) - Something brought about by a cause or agent; result.
Efficient (adj) - Acting or producing effectively with a minimum of waste
or effort.
Expectorate (v) - To eject from the mouth; spit.
Experimental Error - Measure of the variation which exists among
observations on samples treated alike.
Form (ri) - A document printed with spaces for information to be inserted.
Frequency (n) - The number of responses falling within a specified
category or interval.
Hedonic (adj) - Degree of liking or disliking.
151
Hedonic Scale - A scale upon which degree of liking and disliking is
recorded.
Hypothesis (n) - An expression of the researcher’s assumptions or
expectations concerning the outcome of his research, subject to
verification or proof; may be derived from a theory, may be based on
past observations or may merely be a hunch.
Illustrate (v) - To clarify by use of example or comparison.
Independent (adj) - Free from the influence, guidance or control of others.
Inference (n) - Scientific guess about a population based on sample data.
Intensity (n) - Perceived strength of a stimulus.
Interaction (n) - A measure of the extent to which the effect of changing the
level of one factor depends on the level or levels of another or others.
Label (n) - Means of identification, (v) - To attach a label to.
Liberal (adj) - Tolerant; generous.
Mask (v) - Disguise or conceal.
Mean (n) - Sum of all the scores divided by the number of scores.
Molar Teeth - Teeth with a broad crown for grinding food, located behind
the bicuspids.
Monitor (v) - To check, watch or keep track of.
Motivate (v) - To provide with an incentive or motive; maintain panel
interest and morale.
Noticeable (adj) - Readily observed; evident.
Objective (n) - Something aimed at; goal.
Odour (n) - Characteristic that can be perceived by the olfactory organ.
NUT'-IOO
i S')-S3__
152
Orient (v) - To familiarize participants with or adjust to a situation.
Palate (n) - The roof of the mouth.
Panel (ri) - A group of assessors who have been selected or designated in
some manner to participate in a sensory test.
Panel Leader - A person responsible for organizing, conducting and
directing a panel.
Panelist (n) - A member of a panel.
Perceive (v) - To become aware of a stimuli through the senses.
Perceptible (adj) - Capable of being perceived.
Perishable (adj) - Easily destroyed or spoiled.
Portion (ri) - A section or quantity within a larger amount.
(v) - To divide into parts.
Precise (adj) - Ability of repeated measurements to be identical, or almost
identical.
Precision (ri) - Closeness of repeated measurements to each other.
Preference (ri) - Expressed choice for a product or products rather than
another product or products.
Probability (n) - The likelihood or chance of a given event happening.
Psychological Factors - Involving the mind or emotions.
Qualitative (adj) - Pertaining to quality; involved in variation in kind rather
than in degree.
Quality (ri) - Degree of excellence.
Quantitative (adj) - Pertaining to number or quantity.
153
Random Sample - Batch or sample chosen such that all members of the
population have an equal chance of being selected.
Rank (v) - To order a series of three or more samples by the degree of some
designated characteristic, such as intensity or acceptability.
Recruit (v) - To seek and enroll individuals as participants.
Reference (n) - A constant sample with which others are compared or
against which descriptive terms are calibrated.
Reliability (n) - Extent to which the same characteristic can be measured
consistently upon repeated occasions.
Reliable (adj) - Measuring what the experimenter expects to measure;
dependable.
Replication (n) - Independent repetitions of an experiment under identical
experimental conditions.
Representative (adj) - Typical of others in the same category, group or
population. A representative sample of consumers should match the
population distribution of users by age, sex, socio-economic group,
occupation, etc.
Reproduce (v) - To make a copy of or re-create.
Sample (n) - A portion, piece or segment regarded as representative of a
whole and presented for inspection, (v) - To take a sample of.
Scale (n) - A system of ordered marks or divisions at fixed intervals used in
measurement, which may be graphic, descriptive or numerical.
Score (n) - Values assigned to specific responses made to a test item where
the scores have a defined and demonstrated mathematical relationship
to each other.
(v) - To rate the properties of a product on a scale or according to
some numerically defined set of criteria.
Scorecard (n) - Card or paper on which samples are scored.
154
Screen (v) - To separate out or eliminate individuals who are completely
unsuitable for sensory evaluation by testing for sensory impairment and
acuity.
Sense (/?) - Any of the functions of hearing, sight, smell, touch and taste.
Sensitivity (n) - Ability of individuals to detect or perceive quantitative
and/or qualitative differences in sensory characteristics; acuity.
Sensory (adj) - Pertaining to the action of the sense organs.
Sensory Analysis - A scientific discipline used to evoke, measure, analyze
and interpret reactions to those characteristics of foods and materials as
they are perceived by the senses of sight, smell, taste, touch and
hearing.
Sensory Evaluation - See sensory analysis.
Sensory Testing - See sensory analysis.
Significance (n) - Level of probability that the differences among samples
or treatments are real and not due to chance variation.
Simultaneously (adj) - Happening or done at the same time.
Sniff (v) - To evaluate an odour by drawing air audibly and abruptly through
the nose.
Stagger (v) - To arrange in alternating or overlapping time periods.
Statistics (n) - The mathematics of the collection, organization and
interpretation of numerical data, particularly the analysis of population
characteristics by inference from sampling.
Stimulus (n) - Anything causing or regarded as causing a response.
Tactile Senses - Pertaining to the sense of touch.
Tie (n) - An equality of scores between two or more samples.
155
Training (n) - Instruction and practice to familiarize panelists with test
procedures and to increase their ability to recognize, identify and recall
sensory characteristics.
Treatment (n) - Procedure whose effect is measured and compared with
other treatments.
Trial Run - The process of testing, trying and timing methods and
procedures through their actual use.
Valid (adj) - Drawing the proper and correct conclusions from the data.
Validity (ri) - Assurance that the specific characteristic that is supposed to
be measured is truly being measured. Degree to which results are
consistent with the facts.
Index
159
Acceptability 7, 48, 49, 51
Acceptance 59, 60, 63
Affective test 59
Analysis of variance 34, 55, 68, 70, 72, 91, 94, 99, 115
Analytical test 59
Binomial test 55, 61, 82
Blind control 42
Blocking 54, 57, 58, 69
Carriers 40
Category scale 49, 50, 55, 60, 63, 66, 68, 70, 90
Chi-square test 55
Consumer panel 8, 23, 48, 51, 53, 60, 65, 70
Consumer 7, 8, 9, 51, 60, 63, 70
Consumer-oriented test 7, 8, 48, 51, 53, 59, 60
Correlation 56
Discriminant analysis 56
Duncan’s multiple range test 55, 76,101
Duo-trio test 80
Endpoints 49, 114, 117
Experimental design 5, 56, 57, 58
Experimental error 58, 68, 69, 91
Factor analysis 56
Flavour profile 104
Food preparation 7, 11, 12, 13, 14, 17, 19, 23, 26
Friedman test 55, 65, 87
Hedonic 60, 66, 68, 70, 75, 94
Hidden reference 42
In-house panel 8, 29, 30, 62, 63, 65, 66, 70, 83
Kramer test 55
Least significant difference test 55
Line scale 49, 50, 90, 91, 92, 114
Monitoring performance 14, 29, 32, 33, 34, 35, 41, 54, 114, 115
Motivating panelists 29, 35
Multiple comparison test 55, 76, 100, 101
Multivariate analysis 56, 91
160
Non-parametric test 54, 55
Objectives 47, 56, 105, 106
Orienting panelists 30, 106
Paired-comparison test 80
Paired-preference test 60, 61, 62, 63, 80, 82
Panel leader 11, 13,17, 18, 25, 26, 29, 30, 32, 33, 34, 35, 48, 111, 114
Parametric test 54, 55
Preference 7, 8, 32, 48, 49, 51, 59, 60, 61, 62
Principal component analysis 56
Probability 52, 53, 61, 62, 78, 82, 83, 85, 103
Product-oriented test 7, 8, 9, 13, 48, 49, 51, 53, 59, 79
Quantitative descriptive analysis 104
Questionnaire 30
Random number 44, 57, 68, 81, 90
Random sample 8, 53
Randomization 44, 57, 68, 106
Recruiting panelists 29, 30, 106
Reference sample 13, 37, 41, 42, 80, 114
Regression 56
Replication 54, 57, 58, 91, 92, 94, 96, 97, 99, 100, 115
Sampling 19, 37, 53
Schcffe’s test 55
Screening 31, 86, 106
Sensory facilities 11, 18, 23
Significance 53, 61, 65, 69, 76, 77, 82, 83, 85, 97, 102
Standardization 37
Statistical analysis 5, 34, 47, 52, 54, 56, 105, 106
Statistical test 52, 54, 56
Supplies 11, 13, 19, 23, 25
Target population 8, 9
Texture profile 104
Trained panel 8, 9, 29, 31, 32, 54, 83, 86, 104
Training 9, 29, 32, 51, 106, 114
Triangle test 31, 79, 81, 82, 83
Tukey’s test 55, 101
Untrained panel 7, 8, 29, 30, 62, 65, 70
Head Office
IDRC, PO Box 8500, Ottawa, Ontario, Canada K1G 3H9
Regional Office for Southeast and East Asia
IDRC, Tanglin PO Box 101, Singapore 9124, Republic of Singapore
Regional Office for South Asia
IDRC, 11 Jor Bagh, New Delhi 110003, India
Regional Office for Eastern and Southern Africa
IDRC, PO Box 62084, Nairobi, Kenya
Regional Office for the Middle East and North Africa
IDRC/CRDI, PO Box 14 Orman, Giza, Cairo, Egypt
Regional Office for West and Central Africa
CRDI, BP 11007, CD Annexe, Dakar, Senegal
Regional Office for Latin America and the Caribbean
CUD, Casilla de Correos 6379, Montevideo, Uruguay
Please direct requests for information about IDRC and its activities to the
IDRC office in your region.
IDRC
CANADA
- Media
15252.pdf
Position: 1434 (6 views)