Background Published pharmaceutical industry–sponsored trials are more likely than non-industry-sponsored trials to report results and conclusions that favor drug over placebo. Little is known about potential biases in drug–drug comparisons. This study examined associations between research funding source, study design characteristics aimed at reducing bias, and other factors that potentially influence results and conclusions in randomized controlled trials (RCTs) of statin–drug comparisons. Methods and Findings This is a cross-sectional study of 192 published RCTs comparing a statin drug to another statin drug or non-statin drug. Data on concealment of allocation, selection bias, blinding, sample size, disclosed funding source, financial ties of authors, results for primary outcomes, and author conclusions were extracted by two coders (weighted kappa 0.80 to 0.97). Univariate and multivariate logistic regression identified associations between independent variables and favorable results and conclusions. Of the RCTs, 50% (95/192) were funded by industry, and 37% (70/192) did not disclose any funding source. Looking at the totality of available evidence, we found that almost all studies (98%, 189/192) used only surrogate outcome measures. Moreover, study design weaknesses common to published statin–drug comparisons included inadequate blinding, lack of concealment of allocation, poor follow-up, and lack of intention-to-treat analyses. In multivariate analysis of the full sample, trials with adequate blinding were less likely to report results favoring the test drug, and sample size was associated with favorable conclusions when controlling for other factors. In multivariate analysis of industry-funded RCTs, funding from the test drug company was associated with results (odds ratio = 20.16 [95% confidence interval 4.37–92.98], p < 0.001) and conclusions (odds ratio = 34.55 [95% confidence interval 7.09–168.4], p < 0.001) that favor the test drug when controlling for other factors. Studies with adequate blinding were less likely to report statistically significant results favoring the test drug. Conclusions RCTs of head-to-head comparisons of statins with other drugs are more likely to report results and conclusions favoring the sponsor's product compared to the comparator drug. This bias in drug–drug comparison trials should be considered when making decisions regarding drug choice.
References
[1]
Ioannidis JP (2005) Why most published research findings are false. PLoS Med 2: e124. doi:10.1371/journal.pmed.0020124.
[2]
Bero LA, Rennie D (1996) Influences on the quality of published drug studies. Int J Technol Assess Health Care 12: 209–237.
[3]
Smith R (2005) Medical journals are an extension of the marketing arm of pharmaceutical companies. PLoS Med 2: e138. doi:10.1371/journal.pmed.0020138.
[4]
Chan AW, Altman DG (2005) Identifying outcome reporting bias in randomised trials on PubMed: Review of publications and survey of authors. BMJ 330: 753.
[5]
Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. JAMA 291: 2457–2465.
[6]
Chan AW, Krleza-Jeric K, Schmid I, Altman DG (2004) Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 171: 735–740.
[7]
Dickersin K (1990) The existence of publication bias and risk factors for its occurrence. JAMA 263: 1385–1389.
[8]
Chalmers TC, Block JB, Lee S (1972) Controlled studies in clinical cancer research. N Engl J Med 287: 75–78.
[9]
Colditz GA, Miller JN, Mosteller F (1989) How study design affects outcomes in comparisons of therapy. I: Medical. Stat Med 8: 441–454.
[10]
Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273: 408–412.
[11]
Ioannidis JP, Trikalinos TA, Ntzani EE, Contopoulos-Ioannidis DG (2003) Genetic associations in large versus small studies: An empirical assessment. Lancet 361: 567–571.
[12]
Ioannidis JP, Cappelleri JC, Lau J (1998) Issues in comparisons between meta-analyses and large trials. JAMA 279: 1089–1093.
[13]
Misakian AL, Bero LA (1998) Publication bias and research on passive smoking: Comparison of published and unpublished studies. JAMA 280: 250–253.
[14]
Als-Nielsen B, Chen W, Gluud C, Kjaergard LL (2003) Association of funding and conclusions in randomized drug trials: A reflection of treatment effect or adverse events? JAMA 290: 921–928.
[15]
Rochon PA, Gurwitz JH, Simms RW, Fortin PR, Felson DT, et al. (1994) A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med 154: 157–163.
[16]
Safer DJ (2002) Design and reporting modifications in industry-sponsored comparative psychopharmacology trials. J Nerv Ment Dis 190: 583–592.
[17]
Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, et al. (2000) The uncertainty principle and industry-sponsored research. Lancet 356: 635–638.
[18]
Johansen HK, Gotzsche PC (1999) Problems in the design and reporting of trials of antifungal agents encountered during meta-analysis. JAMA 282: 1752–1759.
[19]
Brook O, Van Hout H, Nieuwenhuysea H, De Haan M (2003) Effects of coaching by community pharmacists on psychological symptoms of antidepressant users; a randomised controlled trial. Eur Neuropsychopharmacol 13: 347–354.
[20]
Gomez Cerezo J, Lubomirov Hristov R, Carcas Sansuan AJ, Vazquez Rodriguez JJ (2003) Outcome trials of COX-2 selective inhibitors: Global safety evaluation does not promise benefits. Eur J Clin Pharmacol 59: 169–175.
[21]
Yaphe J, Edman R, Knishkowy B, Herman J (2001) The association between funding by commercial interests and study outcome in randomized controlled drug trials. Fam Pract 18: 565–568.
[22]
Lexchin J, Bero LA, Djulbegovic B, Clark O (2003) Pharmaceutical industry sponsorship and research outcome and quality: Systematic review. BMJ 326: 1167–1170.
[23]
Bekelman JE, Li Y, Gross CP (2003) Scope and impact of financial conflicts of interest in biomedical research: A systematic review. JAMA 289: 454–465.
[24]
Montgomery JH, Byerly M, Carmody T, Li B, Miller DR, et al. (2004) An analysis of the effect of funding source in randomized clinical trials of second generation antipsychotics for the treatment of schizophrenia. Control Clin Trials 25: 598–612.
[25]
Bhandari M, Busse JW, Jackowski D, Montori VM, Schuneman H, et al. (2004) Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ 170: 477–480.
[26]
Procyshyn RM, Chau A, Fortin P, Jenkins W (2004) Prevalence and outcomes of pharmaceutical industry-sponsored clinical trials involving clozapine, risperidone, or olanzapine. Can J Psychiatry 49: 601–606.
[27]
Cho MK, Bero LA (1996) The quality of drug studies published in symposium proceedings. Ann Intern Med 124: 485–489.
[28]
Kjaergard LL, Als-Nielsen B (2002) Association between competing interests and authors' conclusions: Epidemiological study of randomised clinical trials published in the BMJ. BMJ 325: 249.
[29]
Barnes DE, Bero LA (1998) Why review articles on the health effects of passive smoking reach different conclusions. JAMA 279: 1566–1570.
[30]
Heres S, Davis J, Maino K, Jetzinger E, Kissling W, et al. (2006) Why olazepine beats risperidone, risperidone beats quetiapine, and quetiapine beats olazepine: An exploratory analysis of head-to-head comparison studies of second generation antipsychotics. Am J Psychiatry 163: 185–194.
[31]
Law M, Wald N, Rudnicka A (2003) Quantifying effect of statins on low density lipoprotein cholesterol, ischaemic heart disease, and stroke: Systematic review and meta-analysis. BMJ 326: 1423–1429.
[32]
Kelley C, Helfand M, Good C, Ganz M (2006) Drug class review: Hydroxymethylglutaryl-coenzyme A reductase inhibitors (statins). VHA Pharmacy Benefits Management Strategic Healthcare Group and Medical Advisory Panel. Available at: http://www.pbm.va.gov/reviews/hmgstatins？04-09-03.pdf. Accessed 18 August 2006.
[33]
International Committee of Medical Journal Editors (1997) Uniform requirements for manuscripts submitted to biomedical journals. Ann Intern Med 126: 36–47.
[34]
Ioannidis JP, Cappelleri JC, Sacks HS, Lau J (1997) The relationship between study design, results, and reporting of randomized clinical trials of HIV infection. Control Clin Trials 18: 431–444.
[35]
Ioannidis JP, Polycarpou A, Ntais C, Pavlidis N (2003) Randomised trials comparing chemotherapy regimens for advanced non-small cell lung cancer: Biases and evolution over time. Eur J Cancer 39: 2278–2287.
[36]
Institute for Scientific Information (2005) Science citation index: Journal citation reports. Philadelphia (Pennsylvania): Institute for Scientific Information. Available at: http://portal.isiknowledge.com/portal.cg？i?DestApp=JCR&Func=Frame. Accessed 4 May 2007.
[37]
The World Bank Group (2005) Country group data and statistics by income. Geneva: The World Bank. Available at: http://web.worldbank.org/WBSITE/EXTERNAL？/DATASTATISTICS/0,,contentMDK:20421402~p？agePK:64133150~piPK:64133175~theSitePK:2？39419,00.html#Low_income. Accessed 4 May 2007.
[38]
Chalmers I, Adams M, Dickersin K, Hetherington J, Tarnow-Mordi W, et al. (1990) A cohort study of summary reports of controlled trials. JAMA 263: 1401–1405.
[39]
Friedman L, Richter E (2004) Relationship between conflicts of interest and research results. J Gen Intern Med 19: 51–56.
[40]
Dickersin K, Min Y-I, Meinert C (1992) Factors influencing publication of research results. JAMA 267: 374–378.
[41]
Gotzsche PC (1989) Multiple publication of reports of drug trials. Eur J Clin Pharmacol 36: 429–432.
[42]
Tramer M, Reynolds D, Moore R, McQuay H (1997) Impact of covert duplication publication on meta-analysis: A case study. BMJ 315: 635–640.
[43]
Davidson R (1986) Source of funding and outcome of clinical trials. J Gen Intern Med 3: 155–158.
[44]
Moher D, Cook D, Jadad A, Tugwell P, Moher M, et al. (1999) Assessing the quality of reports of randomised trials: Implications for the conduct of meta-analysis. Health Technol Assess 3: 1–98.
[45]
Juni P, Witschi A, Bloch R, Egger M (1999) The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 282: 1054–1060.
[46]
Balk E, Bonis P, Moskowitz H, Schmid CH, Ioannidis J, et al. (2002) Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 287: 2973–2982.
[47]
Schulz K, Chalmers I, Altman D (2002) The landscape and lexicon of blinding in randomized trials. Ann Intern Med 136: 254–259.
[48]
Schulz K, Chalmers I, Altman D, Grimes D, Dore C (1995) The methodologic quality of randomization as assessed from reports of trials in specialist and general medical journals. Online J Curr Clin Trials 187: 81.
[49]
Krimsky S, Rothenberg LS (2001) Conflict of interest policies in science and medical journals: Editorial practices and author disclosures. Sci Eng Ethics 7: 205–218.
[50]
Krimsky S, Rothernberg L (1998) Financial interest and its disclosure in scientific publications. JAMA 280: 225–226.
[51]
Grundy S, Cleeman J, Merz C, Brewer H, Clark L, et al. (2004) Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III Guidelines. Arterioscler Thromb Vasc Biol 24: e149–e161.
[52]
Quilliam B, Perez H, Andros V, Jones P (2004) Quantifying the effect of applying the NCEP ATP III criteria in a managed care population treated with statin therapy. J Manag Care Pharm 10: 244–250.
[53]
Buchkowsky S, Jewesson P (2004) Industry sponsorship and authorship of clinical trials over 20 years. Ann Pharmacother 38: 579–585.
[54]
Dorman P, Counsell C, Sandercock P (1999) Reports of randomized controlled trials in acute stroke, 1955–1995. What proportions were commercially sponsored? Stroke 30: 1995–1998.
[55]
Patsopoulos N, Ioannidis J, Analatos A (2006) Origin and funding of the most frequently cited papers in medicine: Database analysis. BMJ 332: 1061–1064.
[56]
Chaudhry S, Schroter S, Smith R, Morris J (2002) Does declaration of competing interests affect readers' perceptions? A randomised trial. BMJ 325: 1391–1392.
