TY - JOUR
TI - Comparison of evidence of treatment effects in randomized and
nonrandomized studies
AU - Ioannidis, JPA
AU - Haidich, AB
AU - Pappa, M
AU - Pantazis, N
AU - Kokori,
AU - SI
AU - Tektonidou, MG
AU - Contopoulos-Ioannidis, DG
AU - Lau, J
JO - JAMA - JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION - INTERNATIONAL EDITION
PY - 2001
VL - 286
TODO - 7
SP - 821-830
PB - AMER MEDICAL ASSOC
SN - 0098-7484
TODO - 10.1001/jama.286.7.821
TODO - null
TODO - Context There is substantial debate about whether the results of
nonrandomized studies are consistent with the results of randomized
controlled trials on the same topic.
Objectives To compare results of randomized and non randomized studies
that evaluated medical interventions and to examine characteristics that
may explain discrepancies between randomized and non-randomized studies.
Data Sources MEDLINE (1966-March 2000), the Cochrane Library (Issue 3,
2000), and major journals were searched.
Study Selection Forty-five diverse topics were identified for which both
randomized trials (n = 240) and nonrandomized studies (n = 168) had been
performed and had been considered in meta-analyses of binary outcomes.
Data Extraction Data on events per patient in each study arm and design
and characteristics of each study considered in each meta-analysis were
extracted and synthesized separately for randomized and nonrandomized
studies.
Data Synthesis Very good correlation was observed between the summary
odds ratios of randomized and nonrandomized studies (r = 0.75; P<.001);
however, nonrandomized studies tended to show larger treatment effects
(28 vs 11; P = .009). Between-study heterogeneity was frequent among
randomized trials alone (23%) and very frequent among nonrandomized
studies alone (41%). The summary results of the 2 types of designs
differed beyond chance in 7 cases (16%). Discrepancies beyond chance
were less common when only prospective studies were considered (8%).
Occasional differences in sample size and timing of publication were
also noted between discrepant randomized and nonrandomized studies. In
28 cases (62%), the natural logarithm of the odds ratio differed by at
least 50%, and in 15 cases (33%), the odds ratio varied at least
2-fold between nonrandomized studies and randomized trials.
Conclusions Despite good correlation between randomized trials and
nonrandomized studies-in particular, prospective studies-discrepancies
beyond chance do occur and differences in estimated magnitude of
treatment effect are very common.
ER -