|
Sign In to gain access to subscriptions and/or personal tools.
|
An exploratory test for an excess of significant findings
John PA Ioannidis
Clinical Trials and Evidence Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece, jioannid{at}cc.uoi.gr, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Tufts University School of Medicine, Boston, Massachusetts, USA
Thomas A Trikalinos
Clinical Trials and Evidence Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, Tufts University School of Medicine, Boston, Massachusetts, USA
Background The published clinical research literature may be distorted by the pursuit of statistically significant results.
Purpose We aimed to develop a test to explore biases stemming from the pursuit of nominal statistical significance.
Methods The exploratory test evaluates whether there is a relative excess of formally significant findings in the published literature due to any reason (e.g., publication bias, selective analyses and outcome reporting, or fabricated data). The number of expected studies with statistically significant results is estimated and compared against the number of observed significant studies. The main application uses  = 0.05, but a range of thresholds is also examined. Different values or prior distributions of the effect size are assumed. Given the typically low power (few studies per research question), the test may be best applied across domains of many meta-analyses that share common characteristics (interventions, outcomes, study populations, research environment).
Results We evaluated illustratively eight meta-analyses of clinical trials with >50 studies each and 10 meta-analyses of clinical efficacy for neuroleptic agents in schizophrenia; the 10 meta-analyses were also examined as a composite domain. Different results were obtained against commonly used tests of publication bias. We demonstrated a clear or possible excess of significant studies in 6 of 8 large meta-analyses and in the wide domain of neuroleptic treatments.
Limitations The proposed test is exploratory, may depend on prior assumptions, and should be applied cautiously.
Conclusions An excess of significant findings may be documented in some clinical research fields. Clinical Trials 2007; 4: 245—253; http://ctj.sagepub.com
Clinical Trials, Vol. 4, No. 3,
245-253 (2007)
DOI: 10.1177/1740774507079441
 CiteULike  Complore  Connotea  Del.icio.us  Digg  Reddit  Technorati  Twitter What's this?
This article has been cited by other articles:
|
|
|
|
 
J. P.A. Ioannidis
Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses
Can. Med. Assoc. J.,
October 13, 2009;
181(8):
488 - 493.
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. Liberati, D. G. Altman, J. Tetzlaff, C. Mulrow, P. C. Gotzsche, J. P.A. Ioannidis, M. Clarke, P. J. Devereaux, J. Kleijnen, and D. Moher
The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration
Ann Intern Med,
August 18, 2009;
151(4):
W-65 - W-94.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
T. Terasawa, E. M. Balk, M. Chung, A. C. Garlitski, A. A. Alsheikh-Ali, J. Lau, and S. Ip
Systematic Review: Comparative Effectiveness of Radiofrequency Catheter Ablation for Atrial Fibrillation
Ann Intern Med,
August 4, 2009;
151(3):
191 - 202.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. J. Khoury, L. Bertram, P. Boffetta, A. S. Butterworth, S. J. Chanock, S. M. Dolan, I. Fortier, M. Garcia-Closas, M. Gwinn, J. P. T. Higgins, et al.
Genome-Wide Association Studies, Field Synopses, and the Development of the Knowledge Base on Genetic Variation and Human Diseases
Am. J. Epidemiol.,
August 1, 2009;
170(3):
269 - 279.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. Liberati, D. G Altman, J. Tetzlaff, C. Mulrow, P. C Gotzsche, J. P A Ioannidis, M. Clarke, P J Devereaux, J. Kleijnen, and D. Moher
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration
BMJ,
July 21, 2009;
339(jul21_1):
b2700 - b2700.
[Abstract]
[Full Text]
|
|
|
|
|
|
|
D. G Contopoulos-Ioannidis, A. Karvouni, I. Kouri, and J. P A Ioannidis
Reporting and interpretation of SF-36 outcomes in randomised trials: systematic review
BMJ,
January 12, 2009;
338(jan12_1):
a3006 - a3006.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. Vineis, M. Manuguerra, F. K. Kavvoura, S. Guarrera, A. Allione, F. Rosa, A. Di Gregorio, S. Polidoro, F. Saletta, J. P. A. Ioannidis, et al.
A Field Synopsis on Low-Penetrance Variants in DNA Repair Genes and Cancer Susceptibility
J Natl Cancer Inst,
January 7, 2009;
101(1):
24 - 36.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
F. K. Kavvoura, M. B. McQueen, M. J. Khoury, R. E. Tanzi, L. Bertram, and J. P. A. Ioannidis
Evaluation of the Potential Excess of Statistically Significant Findings in Published Genetic Association Studies: Application to Alzheimer's Disease
Am. J. Epidemiol.,
October 15, 2008;
168(8):
855 - 865.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
P. Boffetta, J. K. McLaughlin, C. La Vecchia, R. E. Tarone, L. Lipworth, and W. J. Blot
False-Positive Results in Cancer Epidemiology: A Plea for Epistemological Modesty
J Natl Cancer Inst,
July 16, 2008;
100(14):
988 - 995.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. R. Munafo, E. C. Johnstone, M. F.G. Murphy, and P. Aveyard
"Intention to Analyze" in Pharmacogenomics Studies
Cancer Epidemiol. Biomarkers Prev.,
March 1, 2008;
17(3):
740 - 741.
[Full Text]
[PDF]
|
|
|
|
|
