TUDOR-CASPi
Central-Eastern European Evidence Based Health Care Workshop

December 2-6, 2002, Visegrad, Hungary

Searching for randomized clinical trials and controlled clinical trials in russian medical journals: Handsearching vs Medline

V. RAFALSKI, I. ANDREEVA, E. RYABKOVA

Department of Clinical Pharmacology, Smolensk State Medical Academy, Russia

ABSTRACT

Objective: to evaluate the possibility of identification the controlled trials in Russian medical journals by MEDLINE.

Methods: Handsearching was used as a referent method. Systematic reviews (SRs), meta-analyses (MAs), randomized controlled trials (RCTs) and controlled clinical trials (CCTs) in "Antibiotiki i Khimioterapia" (AiK) journal from 1990 to 2000 were registered. The result of handsearching was compared with MEDLINE search for the identical period and the same journal. MEDLINE search was performed by the PubMed interface. Optimal Cochrane Collaboration Search strategy adapted for PubMed was used for RCTs and CCTs retrieval.

Results: A total number of 1500 articles were studied. No SRs and MAs, 10 (0.7%) RCTs and 78 (5.2%) CCTs were found by handsearch. MEDLINE search retrieved 1475 articles with no SRs and MAs among them. Also 10 RCTs and 52 CCTs were found. As for RCTs 7 publications (70%) agreed with handsearching results; 3 articles (30%) false positive and 3 articles (30%) false negative, for CCTs - 15 articles actually described CCTs (19.2%), 35 (45%) false positive and 63 (80.8%) false negative CCTs. Thus, sensitivity of MEDLINE search for RCTs was 0.7, for CCTs – 0.19; specificity – 0.99 and 0.98 respectively.

Conclusion: MEDLINE search alone could loss up to 30% of RCTs and 80% CCTs in Russian medical journals indexed in MEDLINE.

INTRODUCTION AND OBJECTIVES

The identification of non-English-language trials is important to reduce the impact of the foreign language bias, one of the main sources of selection bias in meta-analyses. Obviously the identification of clinical trial by medical database such as MEDLINE and EMBASE is the most optimal decision.

The objectives of the current study were to to assess the possibility of identification the controlled trials in Russian medical journals by MEDLINE.

MATERIALS AND METHODS

We used handsearching as a referent method. Handsearching was performed according to "Cochrane hand search manual" in one of the leading journal on antimicrobials and infectious diseases of Russia and former USSR "Antibiotiki i Khimioterapia" - (AiK) from 1990 to 2000.

Systematic reviews (SRs), meta-analyses (MAs), randomized controlled trials (RCTs) and controlled clinical trials (CCTs) were registered. Handsearching was carried out by three trained handsearchers. The result of handsearching was compared with MEDLINE search for the identical period and the same journal.

MEDLINE search was performed by the PubMed interface (http://www.ncbi.nlm.nih.gov/PubMed). Optimal Cochrane Collaboration Search strategy adapted for PubMed was used for RCTs and CCTs retrieval:

(((((("Randomized Controlled Trial" [Publication Type] OR "Controlled Clinical Trial" [Publication Type]) OR "Randomized Controlled Trials" [MeSH Terms]) OR "Random Allocation" [MeSH Terms]) OR "Double Blind Method" [MeSH Terms]) OR "Single Blind Method" [MeSH Terms]) NOT ("Animal" [MeSH Terms] NOT ("Human" [MeSH Terms] AND "Animal" [MeSH Terms])) AND "Antibiot Khimioter"[Jour] AND 1990:2000 [dp].

RESULTS AND DISCUSSION

A total number of 1500 articles were studied by handsearch. .The most frequent types of publications were laboratory studies (46.1%) and nonsystematic reviews (22.8%), followed by observational studies (9.5%) and uncontrolled clinical trials (9.1%). No SRs and MAs, 10 (0.7%) RCTs and 78 (5.2%) CCTs were found.

When the MEDLINE search was performed we found 1475 articles. There were no SRs and MAs among them. MEDLINE search identified 10 RCTs: 7 publications (70%) concomitant with handsearching results; 3 articles (30%) false positive and 3 articles (30%) false negative.

MEDLINE search for CCTs identified 52 articles, among them 15 articles actually described CCTs (19.2%), 37 (45%) were false positive and 63 (80.8%) false negative.

Thus, sensitivity of MEDLINE search for RCTs was 0.7, for CCTs – 0.19; specificity – 0.99 and 0.98 respectively. Taking into account that the COCHRANE CONTROLLED TRIALS REGISTER (CCTR) uses the similar search strategy our results may be applied to CCTR as well.

Fig. 1. Types of trials published in "Antibiotiki i Khimioterapia".
("Antibiotics and Chemotherapy"), 1990-2000

Fig. 2. MEDLINE vs HANDSEARCHING

Fig. 3. MEDLINE vs HANDSEARCHING

CONCLUSION

• The use MEDLINE alone for searching RCTs and CCTs in Russian medical journals indexed in MEDLINE could loss up to 30% of RCTs and 80% CCTs.
• We suppose that high specificity of MEDLINE search is caused by the low rate of RCTs and CCTs rather true specificity of MEDLINE citing.