© International Epidemiological Association 2002
Printed in Great Britain
International Journal of Epidemiology 2002;31:112–114
Searching the Literatura Latino Americana e do
Caribe em Ciências da Saúde (LILACS) database
improves systematic reviews
Otavio Augusto Camara Clarka,b and Aldemar Araujo Castroc
Background An unbiased systematic review (SR) should analyse as many articles as possible
in order to provide the best evidence available. However, many SR use only
databases with high English-language content as sources for articles. Literatura
Latino Americana e do Caribe em Ciências da Saúde (LILACS) indexes 670
journals from the Latin American and Caribbean health literature but is seldom
used in these SR. Our objective is to evaluate if LILACS should be used as a
routine source of articles for SR.
Methods
First we identified SR published in 1997 in five medical journals with a high
impact factor. Then we searched LILACS for articles that could match the
inclusion criteria of these SR. We also checked if the authors had already
identified these articles located in LILACS.
Results
In all, 64 SR were identified. Two had already searched LILACS and were
excluded. In 39 of 62 (63%) SR a LILACS search identified articles that matched
the inclusion criteria. In 5 (8%) our search was inconclusive and in 18 (29%)
no articles were found in LILACS. Therefore, in 71% (44/72) of cases, a LILACS
search could have been useful to the authors. This proportion remains the same
if we consider only the 37 SR that performed a meta-analysis. In only one case
had the article identified in LILACS already been located elsewhere by the authors’
strategy.
Conclusion
LILACS is an under-explored and unique source of articles whose use can improve
the quality of systematic reviews. This database should be used as a routine
source to identify studies for systematic reviews.
Keywords
Bibliographic databases, information systems, bias, reviews, epidemiologic
research design, meta-analysis, online systems
Accepted
2 October 2001
An unbiased systematic review (SR) should include all relevant
articles in order to offer the best evidence available.1 However,
some SR use a narrow search strategy for articles, limiting their
search to English literature or to databases like MEDLINE and
EMBASE2 that index mostly journals from the developed countries.
In 1995, Gibbs asserted that there is a ‘lost science’ in the
developing countries, largely ignored in developed countries
because of (1) the lack of an organized way to assess its scientific
a Instituto do Radium, Campinas, São Paulo, Brazil.
b Present affiliation: H Lee Moffitt Cancer Center, Tampa, FL, USA.
c Universidade Federal de São Paulo/Escola Paulista de Medicina, São Paulo,
Brazil.
Correspondence: Otavio AC Clark, H Lee Moffitt Cancer Center, BMT
PROGRAM, 12902 Magnolia Drive, Tampa, FL 33612–9497, USA. E-mail:
[email protected]
production and (2) the low impact factor of the rare indexed
periodicals edited in these countries.3
Latin American and Caribbean medical literature is a part of
this lost science, but there is an organized database in health
science and biological literature, with a potential role in the
identification of articles for SR.
Literatura Latino Americana e do Caribe em Ciências da
Saúde (LILACS) is a database produced by Biblioteca Regional
de Medicina (BIREME) and the Pan-American Health Organization (PAHO) at the Latin American and Caribbean Health
Sciences Information Center in São Paulo, Brazil, since 1982.4
In the year of 2000, 670 journals were indexed. LILACS is available in CD-ROM and on the internet, free of charge.4 There are
interfaces in three languages (Portuguese, Spanish and English).
Most of the journals indexed are published in Portuguese or
Spanish. All titles and some abstracts are translated into English.
112
LILACS DATABASE FOR SYSTEMATIC REVIEWS
Despite its broad reach, the LILACS database is seldom used
as a source of articles in SR. In 1998, only 3 of 50 collaborative
review groups in the Cochrane Collaboration used this database.
If LILACS contains articles that could be included in SR it
should be used as a routine source of articles for SR.
113
references listed in the SR to verify if the identified articles had
already been located. Two of the authors extracted data blinded
to the results of the other. Discrepancies were resolved by reaching a consensus.
Results
Methods
We performed a search in MEDLINE, according to the strategy
described by McKibbon,5 to locate the SR published in 1997 in
five medical journals with high impact factors: Annals of Internal
Medicine (Annals), British Medical Journal (BMJ), Journal of the
American Medical Association (JAMA), The Lancet (LANCET) and
The New England Journal of Medicine (NEJM).
All records retrieved were assessed in full text. We included
studies that used any systematic way of searching literature,
with explicit criteria of selection of articles. Those SR that had
used LILACS were excluded.
The following data were extracted: first author, journal,
databases searched, languages included, inclusion criteria and if
a meta-analysis was performed.
For each included SR, we searched LILACS (30th edition on
CD-ROM, January,1998) to locate articles that matched the SR’s
inclusion criteria. The LILACS search was classified as positive, if
we found at least one article that could fit the inclusion criteria;
as negative if we located no such articles, and as inconclusive if
we found at least one article that may or may not have been
included, depending on additional information. For LILACS
searches that were positive or inconclusive, we checked the
Figure 1 Results of additional searches in LILACS
We located 64 SR (the complete list is available from the authors).
Two had performed a LILACS search and were excluded (Figure 1).
Sixty-two SR were included in this study: 34% (21/62) were
published in Annals; 32% in (20/62) BMJ; 31% (19/62) in
JAMA; 3% (2/62) in LANCET and none in the NEJM.
Of the SR, 37% (23/62) limited their search to Englishlanguage articles; in 32% (20/62) we could not determine if
language restrictions were used. Only 29% (18/62) explicitly
had no language restrictions. Meta-analysis was performed in
60% (37/62) of the SR.
MEDLINE was searched in 95% (59/62) of the SR and in
76% (47/62) it was the only database searched. EMBASE was
searched in 16% (10/62) and other databases in 14% (9/62). Of
the SR, 5% (3/62) did not mention the databases searched.
The LILACS search was positive in 63% (39/62) of them, inconclusive in 8% (5/62) and negative in 29% (18/62) (Figure 1).
Among those 44 SR with a positive or inconclusive LILACS
search, only one had located the article identified in LILACS
by other methods. Therefore, in 70% of the cases (43/62) a
LILACS search was effective in identifying new articles suitable
for inclusion and not located by authors. This percentage
remains the same if we consider only the 37 SR that performed
114
INTERNATIONAL JOURNAL OF EPIDEMIOLOGY
meta-analysis. In this subgroup, a LILACS search was positive in
60% (22/37), inconclusive in 11% (4/37) and negative in 30%
(11/37).
Discussion
Systematic reviews must locate all suitable articles to offer the
best evidence to the reader.1 Unfortunately some SR use a narrow
search strategy making them subject to bias.2 As we have shown,
a search in the ‘lost science’ can offer useful articles for SR.
In our study, only 2 of 64 SR (3%) had searched LILACS, but
in 70% (44/62) of them a search in this database could have
identified articles potentially relevant to the review.
The Brazilian Cochrane Center6 and BIREME4 have been
working in an effort to facilitate access to and searching of
LILACS. A highly sensitive search strategy for clinical trials
adapted to cover three languages was developed7 and free
access to LILACS via the internet is available.4 A program to
re-tag randomized studies is also ongoing.
Concerns regarding the quality of publications in Portuguese
or Spanish can arise, but a previous study found no significant
qualitative differences among trials published in English or nonEnglish languages,2 Spanish included.
This study demonstrates that LILACS could have added
information to 70% of SR surveyed and that these articles could
not be identified by other search methods.
In conclusion, LILACS is an under-explored and unique source
of articles, its use can improve the quality of SR. This database
should be used as a routine source of studies in the preparation
of SR.
Acknowledgements
Source of support: Universidade Federal de São Paulo, São Paulo,
SP-Brazil. The authors also thank Dr Benjamin Djulbegovic for
the critical review of the article.
KEY MESSAGES
•
Systematic reviews (SR) should use a wide search strategy to locate all relevant information, but seldom use
information from developing countries.
•
The Literatura Latino Americana e do Caribe em Ciências da Saúde (LILACS) database indexes 670 health science
journals from the Latin American and Caribbean regions.
•
From 64 SR published in five medical journals in 1997, only 2 (3%) had searched LILACS. For the other 62 SR,
we could find articles in LILACS that matched the inclusion criteria of 44 (71%) SR. In only one of these had the
article been located by author’s search strategy.
•
LILACS is a unique and relevant source of articles and should be used routinely in SR.
References
1 Cook DJ, Mulrow CD, Haynes RB. Systematic reviews: synthesis of
5 McKibbon KA, Walker-Dilks CF, Wilczynski NL, Haynes RB. Beyond
best evidence for clinical decisions. Ann Intern Med 1997;126:376–80.
ACP Journal Club: how to harness MEDLINE for review articles. ACP
J Club 1996;124:A12–13.
2 Moher D, Fortin P, Jadad AR et al. Completeness of reporting of trials
published in languages other than English: implications for conduct
and reporting of systematic reviews. Lancet 1996;347:363–66.
6 Available from: URL: www.centrocochranedobrasil.org.br [Accessed in
3 Gibbs W. Lost Science in the Third World. Scientific American 1995;273:
7 Castro AA, Clark OA, Atallah AN. Optimal search strategy for clinical
76–83.
4 Available from: URL: www.bireme.br [Accessed in July 2001].
July 2001].
trials in the Latin American and Caribbean Health Science Literature
database (LILACS database): update. São Paulo Med J 1999;117:138–39.