VALIDATION OF AN INSTRUMENT FOR THE NUTRITION MONITORING OF THE
ELDERLY TO BE USED BY CAREGIVERS WITHOUT SPECIFIC TRAINING
Lilian Fernanda Galesi1, Juliana A D B Campos2, José Eduardo Corrente,3 Maria Rita M. de Oliveira4
1
Doctorate student/Alimentos e Nutrição/ Fcfar-Unesp, Araraquara, Brasil, [email protected]
2
Alimentos e Nutrição/Fcfar-UNESP, Araraquara, Brasil, [email protected]
3
Bioestatística/ IBB -UNESP, Botucatu, Brasil, [email protected]
4
Educação/ IBB -UNESP, Botucatu, Brasil, [email protected]
reality. This monitoring should be based on an instrument
that can effectively detect individuals at nutritional risk
and be technically viable.
Abstract - A nutritional screening instrument to be used
by laypersons who provide care for elderly individuals
was established. The temporary data indicate that the
instrument has reproducibility and appropriate internal
consistency. If the instrument does not lose its good
characteristics after factor analysis, the instrument
development was successful.
Nevertheless, for a clinical diagnostic or
screening instrument to be used correctly and produce
valid, reliable and reproducible data, it needs to be
validated 13.
Keywords: elderly, nutritional assessment, validation
study.
2.PURPOSE
This study aims to validate an instrument for the
nutrition monitoring of the elderly to be used by
caregivers without specific training.
1. INTRODUCTION
Projections for 2025 indicate that the elderly
Brazilian population can exceed 30 million, which would
correspond to 14% of the total population 1. In 50 years
this number is estimated to be 58 million, which would
correspond to 23.6% of the Brazilian population 2.
3.METHODS
The entire validation process of the instrument
involved 124 caregivers and 100 elderly individuals
recruited from a long stay institution and homes. The
interviewers were trained and calibrated. The validation
consisted of face validation (done in a previous study),
content validation, reliability study and criterion validity.
Higher life expectancy associated with a higher
rate of chronic diseases has increased the rates of elderly
people with cognitive and mechanical disabilities,
generating a scenario of survival of dependent elderly 3.
A high prevalence of malnutrition can be
expected among the elderly that live in institutions 4,5 usually because they need care – ranging from 20 to 80%
5,6,7,8,9
. Because of its resemblance with the signs and
symptoms that are characteristic of the aging process,
malnutrition can be examined without meticulousness and
its importance underestimated 10,11.
For content validation, 10 judges from the
academic area and that work with the elderly classified
each question of the instrument as essential, useful but not
essential, and unnecessary. The number of judges that
deemed a question essential was assessed. When more
than half of the judges felt that the question was essential,
this question was given some content validity. The
content validity ratio was used for this assessment and
expressed as follows:
Caregivers without specific training often do not
notice nutritional problems and the need of personalized
nutritional care and little attention is given to the
identification of elderly individuals who have benefited
from having their malnutrition detected 6. If this morbid
condition is not detected, it can contribute to worsen the
clinical manifestations of numerous chronic diseases and
increase mortality 12.
Nutritional assessment presupposes diagnosis of
the nutritional status, an activity done by a professional
with higher education. However, there is a work of
monitoring the nutritional status that should be done
continuously and that does not require someone with a
higher degree to do it, thus embracing the Brazilian
Where CVR is the content validity ratio, n is the number
of judges who deemed the question to be essential and N
is the total number of judges.
The reliability study included internal
consistency (using the KR-20 Coefficient – Kuder-
1
Proceedings of the 9th Brazilian Conference on Dynamics Control and their Applications
Serra Negra, SP - ISSN 2178-3667
1119
Once the MNA and the instrument were filled
out, the calculations began to determine the cut-off point
and analyze the discriminatory power of the instrument.
The ROC (receiver operating characteristic) curve was
used for the analyses.
Richardson Coefficient – classified according to Sneed &
Herman (1990)14) and reproducibility (using Kappa
statistics, classified according to Landis & Koch
(1977)15). Of the 124 caregivers who participated in the
study, 109 were approached twice for intra-caregiver
assessment.
All scores obtained from the questionnaires filled
out by the caregivers (attributing one point for each
answer with positive characteristic for malnutrition) were
used to generate the ROC curve. Sensitivity and
specificity were determined for each score. A graph of
sensitivity as a function of specificity was then
constructed. The best cut-off point for the classification of
nutritional status was given by the balance between
sensitivity and specificity.
In all approaches, the caregivers were asked to
answer the instrument without providing any prior
explanation or help during the filling-out process. This
was done to assess how the caregivers understood the
questions, which would be confirmed in the
reproducibility study. A protocol in the form of a table
was prepared for the filling-out process so that the
caregivers could place an “x” on what was being asked.
The discriminatory power of the instrument was
verified by calculating the area under the ROC curve and
classified according to Hosmer & Lemeshow (2000)17.
The interval between approaches was of one
week so that it would be short enough to prevent the
clinical condition of usually frail elderly individuals who
require care to worsen quickly, thereby affecting their
nutritional status, and long enough for the answers given
by the caregiver in the second approach not to be
influenced by memory.
4. RESULTS
In the content validation process, the questions
were classified as essential, useful but not essential, and
unnecessary, and the content validity ratio was then
calculated as shown in Table 1.
Criterion validity was done with the MNA (Mini
Nutritional Assessment), a recognized and internationally
validated instrument for the nutritional assessment of the
elderly, considered “gold standard” because of its
sensitivity (96%) and specificity (98%) 16.
Table 1. Content validity ratio (CVR) of the questions of the instrument
Essential
Useful, but not
essential
Unnecessary
CVR
Does the elderly use a wheelchair?
9
1
0
0.8
Does the elderly urinate or defecate in his/her pants?
8
2
0
0.6
Does the elderly need diapers all the time?
7
3
0
0.4
Has the elderly been isolating him/herself and sad?
9
1
0
0.8
Questions
Has the elderly been worried and nervous?
7
3
0
0.4
Do you think the elderly is losing weight?
10
0
0
1.0
Is the elderly toothless and does not use a denture?
10
0
0
1.0
Can the elderly eat without somebody putting the food in his/her mouth?
10
0
0
1.0
Can the elderly walk without somebody’s help?
7
3
0
0.4
Can the elderly get out of bed by him/herself?
6
4
0
0.2
Can the elderly take a shower without help?
7
3
0
0.4
Can the elderly get dressed without help?
6
2
2
0.2
Does the elderly know what day of the week it is? (Monday, Tuesday,
Wednesday, Thursday...)?
7
2
1
0.4
Does the elderly know what day of the month it is? (1,2,3,4,5...)?
7
2
1
0.4
Does the elderly know what year it is?
6
2
2
0.2
Does the elderly have some idea of what time it is?
6
3
1
0.2
Does the elderly know the name of the city s/he lives in?
6
2
2
0.2
If you ask, can the elderly repeat the words “vase, car and window”?
6
2
2
0.2
Does the elderly have breakfast, lunch and supper every day?
10
0
0
1.0
Does the elderly eat egg or meat (any kind of meat such as beef, poultry,
pork, fish) every day?
10
0
0
1.0
Does the elderly drink at least 6 cups of filtered water per day?
9
1
0
0.8
2
Proceedings of the 9th Brazilian Conference on Dynamics Control and their Applications
Serra Negra, SP - ISSN 2178-3667
1120
Table 2. Analysis of internal consistency using the KR-20 coefficient
QUESTIONS
inter-item r
KR-20
Does the elderly use a wheelchair?
0.1785
0.8130
Does the elderly urinate or defecate in his/her pants?
0.1761
0.8104
Does the elderly need diapers all the time?
0.1724
0.8064
Has the elderly been isolating him/herself and sad?
0.2010
0.8342
Has the elderly been worried and nervous?
0.1971
0.8308
Do you think the elderly is losing weight?
0.1937
0.8278
Is the elderly toothless and does not use a denture?
0.1950
0.8289
Can the elderly eat without somebody putting the food in his/her mouth?
0.1802
0.8147
Can the elderly walk without somebody’s help?
0.1752
0.8094
Can the elderly get out of bed by him/herself?
0.1726
0.8067
Can the elderly take a shower without help?
0.1681
0.8016
Can the elderly get dressed without help?
0.1704
0.8042
Does the elderly know what day of the week it is? (Monday, Tuesday, Wednesday, Thursday...)?
0.1721
0.8061
Does the elderly know what day of the month it is? (1,2,3,4,5...)?
0.1702
0.8040
Does the elderly know what year it is?
0.1722
0.8062
Does the elderly have some idea of what time it is?
0.1742
0.8083
Does the elderly know the name of the city s/he lives in?
0.1794
0.8139
If you ask, can the elderly repeat the words “vase, car and window”?
0.1849
0.8194
Does the elderly have breakfast, lunch and supper every day?
0.1957
0.8295
Does the elderly eat egg or meat (any kind of meat such as beef, poultry, pork, fish) every day?
0.1964
0.8301
Does the elderly drink at least 6 cups of filtered water per day?
0.1981
0.8317
INSTRUMENT
0.1821
0.8238
results. Table 2 shows KR-20 and the inter-item
correlation of each question and the instrument.
Both the instrument and all the questions
presented excellent internal consistency according to
Sneed’s & Herman’s classification (1990) 14, that is, when
the value of KR-20 exceeds 0.8. In inter-item analysis,
relatively similar values were verified. These data show
that the instrument has good homogeneity.
The questions regarding getting out of bed,
getting dressed, knowing the year, time and city, and
being capable of repeating three words were the ones with
the worst CVR. However, these questions presented some
content validity and were therefore kept in the instrument.
Once content validation was finished, the
reliability study and criterion validation were done.
The reliability study, which included internal
consistency (using the KR-20 Coefficient) and
reproducibility (using Kappa statistics) presented good
3
Proceedings of the 9th Brazilian Conference on Dynamics Control and their Applications
Serra Negra, SP - ISSN 2178-3667
1121
Table 3. Analysis of the reproducibility of the instrument using Kappa statistics
κ
CLASSIFICATION
Does the elderly use a wheelchair?
1.0000
Perfect
Does the elderly urinate or defecate in his/her pants?
0.8705
Excellent
Does the elderly need diapers all the time?
0.9632
Excellent
Has the elderly been isolating him/herself and sad?
0.8688
Excellent
Has the elderly been worried and nervous?
0.7545
Good
Do you think the elderly is losing weight?
0.7538
Good
Is the elderly toothless and does not use a denture?
0.7704
Good
Can the elderly eat without somebody putting the food in his/her mouth?
0.6675
Good
Can the elderly walk without somebody’s help?
0.9057
Excellent
Can the elderly get out of bed by him/herself?
0.8817
Excellent
Can the elderly take a shower without help?
0.8827
Excellent
Can the elderly get dressed without help?
0.9259
Excellent
Does the elderly know what day of the week it is? (Monday, Tuesday, Wednesday,
Thursday...)?
0.9243
Excellent
Does the elderly know what day of the month it is? (1,2,3,4,5...)?
0.8703
Excellent
Does the elderly know what year it is?
0.9430
Excellent
Does the elderly have some idea of what time it is?
0.8032
Excellent
Does the elderly know the name of the city s/he lives in?
0.8688
Excellent
If you ask, can the elderly repeat the words “vase, car and window”?
0.6598
Good
Does the elderly have breakfast, lunch and supper every day?
0.5958
Regular
Does the elderly eat egg or meat (any kind of meat such as beef, poultry, pork, fish)
every day?
0.7315
Good
Does the elderly drink at least 6 cups of filtered water per day?
0.8291
Excellent
QUESTIONS
The test-retest reproducibility of the instrument is
shown in Table 3.
The reproducibility was considered appropriate,
with most of the questions presenting excellent
agreement, ranging from perfect (only the question on
wheelchair use) and regular (the question on meals),
according to the criteria established by Landis & Koch
(1977) 15.
The ROC curves were generated for two
distinctive situations. The first (Figure 1) illustrates the
power of the instrument for discriminating risk of
malnutrition and the second (Figure 2) for discriminating
malnutrition.
4
Proceedings of the 9th Brazilian Conference on Dynamics Control and their Applications
Serra Negra, SP - ISSN 2178-3667
1122
Figure 1. Sensitivity and specificity of the instrument for risk of malnutrition for each cut-off point determined by the ROC curve
Figure 2. Sensitivity and specificity of the instrument for malnutrition for each cut-off point determined by the ROC curve
5
Proceedings of the 9th Brazilian Conference on Dynamics Control and their Applications
Serra Negra, SP - ISSN 2178-3667
1123
those at risk of malnutrition; and 84.5 and 76.2 in the cut-off
point 9 to discriminate those at risk of malnutrition from the
malnourished.
Since this is a monitoring instrument, greater
sensitivity than specificity were preferred. From the possible
cut-off points that could be chosen in this study, a sensitivity
of 93.6 and a specificity of 65.2 in the cut-off point 4 were
chosen to discriminate well-nourished individuals from
Thus, by adding the points with the positive
characteristics for malnutrition, the final score of the
and it does not change within a week’s period. Meanwhile,
the question regarding having breakfast, lunch and supper
can be influenced by the elderly’s eating behavior on the
assessment day or in days close to the assessment, which
can easily change within a week’s period.
instrument can classify the elderly into the following:
individuals with less than 4 points are classified as wellnourished; individuals with 4 to 9 points are classified as
being at risk of malnutrition; and those with more than 10
points are malnourished.
High reproducibility does not ensure validity. A test
can be reproducible (have identical or close results when it
is repeated) but not capable of making correct
discriminations. If a diagnostic test of extremely low
validity is used, many people will be classified incorrectly
19
.
The discriminatory power of the instrument
determined by the area under the ROC curve was considered
good (area=0.852) for risk of malnutrition and excellent
(area=0.925) for malnutrition according to the classification
proposed by Hosmer & Lemeshow (2000) 17.
A good screening test should have high sensitivity
to avoid missing the disease cases in the tested population,
and high specificity to reduce the number of false-positive
results that require further investigation 13. However, since
the objective of this study is an instrument capable of
nutritional monitoring, the essential requirement is high
sensitivity to detect all elderly individuals with nutritional
deficits. Although more importance was given to sensitivity
and since specificity is also important, the cut-off point was
determined by finding the best balance between the two
characteristics. It should be emphasized that this study is
about the validation of a screening instrument and even a
cut-off point with greater specificity, capable of separating
malnourished individuals from other individuals, does not
have diagnostic power. This is a warning to demand further
investigations by qualified professionals.
5. DISCUSSION
The validation of this instrument is the second stage
of a doctorate project, whose first stage consisted of
developing it. Since this is a new instrument, there are no
data from other authors discussing its validation process.
Content validation allowed specialist dieticians to
assess the presence of each question in the instrument. The
questions regarding the capability of feeding oneself, weight
and teeth loss, number of meals and intake of protein
sources were unanimously classified as “essential” by the
judges. This classification may have occurred because these
questions are more directly related to food intake and
nutritional status.
It is necessary to make an exception to discuss the
final result of this validation. Exploratory factor analysis
was not done in this study. It should be done before
reliability study and criterion validity. This analysis should
be done because this instrument contains many variables.
Once it is done, dimensions in the instrument can be defined
and even the number of variables or questions can be
reduced. The results after an exploratory factor analysis can
differ from the current results, so the psychometric
characteristics of the instrument should not be taken as
definitive.
The screening test should also take only a few
minutes to be done and be easy to use 13. Concern with the
time required from the caregiver and ease of administration
was always present during the development of this work. If
factor analysis leads to a reduction in the number of
questions, it will also contribute to these aspects, improving
the quality of the instrument.
It is hoped that once the analyses are finished and
more reliable results are available, the instrument can
contribute to turn the caregiver without specific training into
someone who is capable of doing the nutritional monitoring
of elderly individuals exposed to nutritional risks, such as
those who require daily care.
The temporary data indicate an excellent internal
consistency, both for the instrument and for all the
questions. The internal consistency reflects the dimension
with which the items of a questionnaire can measure the
same phenomenon, thereby allowing its reliability to be
assessed 18.
6. CONCLUSION
The temporary data indicate that the instrument
proposed here is of good quality, has adequate
reproducibility and internal consistency, in addition to the
ability to discriminate between well-nourished, at risk of
malnourishment and malnourished elderly individuals, using
the MNA as the “gold standard.”
The test-retest reproducibility was also considered
adequate for all the questions of the instrument. Two
questions stood out because they were either classified as
having perfect reproducibility (the use of a wheelchair) or
regular reproducibility (meals per day). This profile for the
questions could be caused by variations in the caregiver’s
judgment during the assessment. The question regarding the
use of a wheelchair is very objective, its use is continuous
6
Proceedings of the 9th Brazilian Conference on Dynamics Control and their Applications
Serra Negra, SP - ISSN 2178-3667
1124
REFERENCES
1. BRASIL. Instituto Brasileiro de Geografia e Estatística. Censo
Demográfico de 2000, Brasília, IBGE. Disponível em: <http://
www.ibge.gov.br>. Acesso em: 27 set. 2006.
2. SCAZUFCA, M.; CERQUEIRA, A.T.A.R.; MENEZES, P.R.
Investigações epidemiológicas sobre demência nos países em
desenvolvimento. Revista de Saúde Pública, São Paulo, v.36, n.6, p. 773778, 2002.
3. KARSCH, U.M. Idosos dependentes: famílias e cuidadores. Cadernos de
Saúde Pública, Rio de Janeiro, v.19, n.3, p.861-6, mai./jun. 2003.
4. KNICKMAN, J.R.; SNELL, E. K. The 2030 Problem: Caring for Aging
Baby Boomers. Health Services Research, v.37, n.4, p.848-884, aug. 2002.
5. PAULY, L.; STEHLE, P.; VOLKERT, D. Nutritional situation of elderly
nursing home residents. Zeitschrift fur Gerontologie und Geriatrie: Organ
der Deutschen Gesellschaft fur Gerontologie und Geriatrie. v.40, p.3–12,
jan. 2007.
6. GUIGOZ, Y.; LAUQUE, S.; VELLAS, B.J. Identifying the elderly at
risk for malnutrition: The Mini nutritional Assessment. Clinics in Geriatric
Medicine, v.18, n.4, p.737-757, 2002.
7. DONINI, L.M.; SAVINA, C.; CANNELLA, C. Eating habits and
appetite control in the elderly: the anorexia of aging. International
Psychogeriatric, v.15, n.1, p.73-87, mar. 2003.
8. HARRIS, C.L.; FRASER, C. Malnutrition in the institutionalized elderly:
the effects on wound healing. Ostomy Wound Manage, v.50, n.10, p.54-63,
oct. 2004.
9. WOO, J.; CHI, I.; HUI, E.; et al. Low staffing level is associated with
malnutrition in long-term residential care homes. European Journal of
Clinical Nutrition, v.59, p.474-479, feb. 2005.
10. GALLAGHER-ALLRED, C. Geriatric Care. In: SIMKO, M.D.;
COWELL, C.; GILBRIDE, J.A. Nutrition Assessment: A Comprehensive
Guide for Planning Intervention. Aspen, 1995. p.301 a 326.
11. TORODOVIC, V. Detecting and managing undernutrition of older
people in the community. British Journal of Community Nursing, v.2, n.6,
p.54-60, feb. 2001.
12. SOUZA, V.M.C; GUARIENTO, M.E. Avaliação do idoso desnutrido.
Revista Brasileira de Clínica Médica, v.7, p.46-49, 2009.
13. GOULART, B. N. G.; CHIARI, B. M. Testes de rastreamento x testes
de diagnóstico: atualidades no contexto da atuação fonoaudiológica. PróFono Revista de Atualização Científica, Barueri, v. 19, n. 2, p. 223-232,
abr./jun. 2007.
14. SNEED, J.; HERMAN, C. M. Influence of job characteristics and
organizational commitment on job satisfaction of hospital food service
employees. Journal of the American Dietetic Association, v.90, p.1072–
1076, 1990.
15. LANDIS, J.R., KOCH, G.G. The measurement of observer agreement
for categorical data. Biometrics, v.33, p.159-74, 1977.
16. VELLAS, B.; GUIGOZ, Y.; J. GARRY, P.J. et al. The mini nutritional
assessment (MNA) and its use in grading the nutritional state of elderly
patients. Nutrition, v.15, n.2 , p.166-122. feb. 1999.
17. HOSMER, D.W., LEMESHOW, S. Applied logistic regression. 2 ed.
New York: John Wiley & Sons Inc, 2000.
18. ABREU, A.M.; FARIA, C.D.C.M.; CARDOSO, S.M.V. et al. Versão
brasileira do Fear Avoidance Beliefs Questionnaire. Cadernos de Saúde
Pública, Rio de Janeiro, v. 24, n.3, p.615-623, mar, 2008.
19. PEREIRA, M.G. Aferição dos Eventos. In______. Epidemiologia:
teoria e prática. 6.ed. Rio de Janeiro: Guanabara Koogan, 2002, p.358-76.
7
Proceedings of the 9th Brazilian Conference on Dynamics Control and their Applications
Serra Negra, SP - ISSN 2178-3667
1125