Industry sponsorship and research outcome

Types of studies

This review includes reports of empirical studies that investigate samples of primary research studies. To avoid confusion we will use the terms 'studies' for the primary research studies and 'papers' for the reports of empirical studies of primary research studies. We will use the term trials to describe studies of a randomized clinical trial design.

We included papers of cross‐sectional studies, cohort studies, systematic reviews or meta‐analyses that quantitatively compared primary research of human drug or medical device studies sponsored by the pharmaceutical or device industries with studies that had other sources of sponsorship. These papers could report the results of methodological studies or systematic reviews that had a pre‐specified subgroup or sensitivity analysis by sponsorship source. We also included papers investigating sources of heterogeneity (e.g. using meta‐regression) if sponsorship was investigated. Drugs were defined as medications that require approval by a regulatory authority as a prescription drug, recognizing that these approval standards vary worldwide. Devices were defined based on the Food and Drug Administration (FDA) definition as instruments intended for use in the diagnosis, treatment or prevention of disease.

We excluded papers without quantitative data related to our primary or secondary outcomes. We excluded papers of the effects of sponsorship by non‐pharmaceutical or non‐device (e.g. tobacco, food or chemical) industries, and papers that evaluated the effectiveness of herbal supplements or medical procedures. Papers examining mixed interventions (e.g. pharmaceuticals and educational interventions) were included if drug or device data were reported separately or could be obtained from the authors.

We excluded papers that quantitatively compared the association of sponsorship and results of syntheses of research studies (i.e. systematic reviews or meta‐analyses) or pharmacoeconomic studies of drugs or devices. We also excluded analyses of pharmacokinetic studies and studies restricted to non humans (e.g. animal or cell cultures).

Only papers published in full, including structured research letters, were included. We excluded unstructured letters to the editor and conference abstracts. This decision was based on the poor reporting quality of data in letters and conference abstracts encountered in a previous version of our review (Lexchin 2003). A comment to the previous version of our review (Lundh 2012) suggested that the exclusion of conference abstracts and letters could have introduced publication bias. Therefore, we included conference abstracts and all types of letters in a sensitivity analysis (see below). We had no language restrictions.

Types of data

Drug and device papers including human research studies comparing drug to placebo, device to sham, drug to drug, drug to device, device to device, or mixed comparisons where the effectiveness, efficacy or harms of the drug or device were evaluated. A few papers included data from both unpublished and published studies. If data were reported separately for the published studies or were available from the authors we used data from published studies only. The reason for this decision was that published studies represent what is available to users of the medical literature and our focus was on assessing biases in published studies.

Types of methods

We defined sponsorship as funding or provision of free drug or devices. Drug or device studies with pharmaceutical or device industry funding versus those with other or undisclosed funding were included. We extracted the definition of industry funding verbatim from the included papers (see Data extraction and management) and reported this in the 'Characteristics of included studies' table. For analysis, we grouped the definitions into a variety of categories, including 100% pharmaceutical or device company funding, 100% non‐industry funding, mixed funding (e.g. non‐industry and industry collaboration), free provision of drug or device only, and undisclosed funding.

We included papers that compared industry sponsored studies with non‐industry sponsored studies and also papers that compared studies of products by competing manufacturers (i.e. studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the control treatment); we analyzed the two types of papers separately.

Types of outcome measures

Electronic searches

In this update, we searched Ovid MEDLINE (R) In‐Process and other non‐indexed citations and Ovid MEDLINE (R) (2010 to February 2015), Ovid Embase (2010 to February 2015) and the Cochrane Methodology Register (2015, Issue 2) (Wiley InterScience Online). We searched the Web of Science (June 2015) for papers that cited any of the papers included in our review.

Searching other resources

Other sources of data included author files, searches of reference lists of included papers and previous systematic reviews.

Selection of studies

Two pairs of assessors  (LB and JS or BM and JL) screened the titles and abstracts, when available, of all retrieved records for obvious exclusions, and assessed the remaining papers based on full text. Any disagreements were resolved by consensus and reasons for exclusions of potentially eligible papers are described in the 'Characteristics of excluded studies' table. There was no need for translation of non‐English papers.

Data extraction and management

Two pairs of assessors (AL and JS or BM and JL) independently extracted data from included papers; differences in data extraction were resolved by consensus. 

We extracted data on the following.

• Year published.

• Country of corresponding author.

• Study objective.

• Study design used in the paper (cohort, cross‐sectional, systematic review or meta‐analysis, other).

• Study domain ‐ descriptive (e.g. oncology drug trials).

• Study domain ‐ category (drug/device class, specific disease, medical specialty/type of diseases, mixed).

• Type of studies (drug, device, drug and device, mixed).

• Type of comparisons (drug versus drug, drug versus placebo, device versus device, device versus sham, device versus drug, mixed, other).

• Sample strategy used to locate research studies (electronic search only, electronic plus other, sampling of journals, sampling by venue (e.g. conference abstracts)).

• Whether there were language restrictions on the search.

• Number of studies included in the sample.

• Time period covered by studies in the paper.

• Sponsorship categories coded in the paper. Categories were:

• • 100% pharmaceutical/device company funded;

  • 100% non‐profit funded;

  • mixed funding ‐ e.g. non‐industry and industry collaboration;

  • provision of drug or device only; and

  • undisclosed funding.

• Sponsorship categories used in analysis in the paper (e.g. 100% industry funded grouped with mixed funding for industry category).

• Description of role of the sponsor (if any). For example, definition of the sponsor’s role in the design, implementation or reporting in the sample of studies.

• Criteria used to assess risk of bias of the studies included in the paper.

• Primary purpose of the study.

• Whether the paper commented on appropriateness of comparators.

• Data on sponsorship and results.

• Data on sponsorship and conclusions.

• Data on sponsorship and effect size.

• Data on sponsorship and risk of bias.

• Data on sponsorship and concordance between study results and conclusions.

• Additional relevant data.

• Funding source of included paper. As this item was not included in the previous version of the review we also extracted data from papers included in the previous version.

Assessment of risk of bias in included studies

Since there are no validated criteria for assessing risk of bias in these types of papers, we developed our own criteria. We reviewed papers for high, low or unclear risk of bias for each of four criteria. If a criterion was met, it was regarded as having low risk of bias, and high risk of bias otherwise. If we could not determine whether a criterion was met, we coded it as unclear. We used the following criteria:

• whether explicit and well‐defined criteria that could be replicated by others were used to select studies for inclusion/exclusion;

• whether there was an adequate study inclusion method, with two or more assessors selecting studies;

• whether the search for studies was comprehensive; and

• whether methodological differences and other characteristics that could introduce bias were controlled for or explored.

Measures of the effect of the methods

We performed a meta‐analysis of the papers that reported the association of sponsorship with favorable study outcomes in cases where a pooled risk ratio (RR) and its 95% confidence interval could be computed.

The definition of a favorable outcome varied among papers. In some papers favorable outcomes were defined as those that were favorable to the sponsor's product and in others favorable to the test treatment. This difference in terminology did not matter when the comparison was between active treatment and placebo, since the sponsor's product was the active treatment and not placebo. For head‐to‐head comparisons, however, the sponsor could be either the manufacturer of the test treatment or the control treatment. In these cases, when data were available, we recoded outcomes as to whether they were favorable to the sponsor's product.

We separately analyzed papers of industry sponsored head‐to‐head studies, comparing studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the comparator treatment. This was done by assigning the newest treatment (most recent FDA approval date) as the 'test' treatment and the older treatment as the 'comparator' treatment using similar methods as described by Bero and colleagues (Bero 2007) and comparing the number of studies favorable to the test treatment in the two groups (i.e. sponsor produces test treatment or sponsor produces comparator treatment).

At the time many of the papers were published, the approach was to assess the methodological quality of studies as opposed to an assessment of the risk of bias of studies. We therefore recoded the data on methodological quality into 'Risk of bias' categories. So, for example, a trial with adequate concealment of allocation was coded as low risk of bias and a trial with inadequate concealment of allocation as high risk of bias. Some papers assessed risk of bias by summarizing results for individual domains into an overall methodological quality score (i.e. a scale approach). There are substantial methodological problems related to quality scales (Jüni 1999), and their use is not recommended. We therefore did not combine the results obtained with these scales, but report the results descriptively. The included papers assessed blinding using different approaches. Some papers rated blinding for the study overall, for example whether a study used matching placebo tablets (which could be considered blinded) and some papers assessed who was blinded, similar to the Cochrane 'Risk of bias' tool. The Cochrane 'Risk of bias' tool assesses blinding to protect against performance bias (e.g. clinicians or patients are blinded) and to protect against detection bias (e.g. outcome assessors are blinded). We therefore categorized each 'Risk of bias' assessment into the items blinding‐overall, blinding‐performance bias and blinding‐detection bias.

Dealing with missing data

We contacted authors of the original papers in an attempt to obtain missing data. If papers included studies reporting conflicts of interest, but not the source of funding, we contacted the authors in order to obtain separate data for funding. In total, we contacted authors of eight papers included in this update and received additional data for five of these papers.

Assessment of heterogeneity

We assessed heterogeneity using I². We defined substantial statistical heterogeneity as an I² > 50% (Higgins 2011a).

Data synthesis

We used Review Manager (RevMan 2014) to analyze data. For dichotomous data we used the Mantel‐Haenszel random‐effects model to create a pooled RR. In the previous version of this review (Lundh 2012), we used a fixed‐effect model as default and a random‐effects model when substantial heterogeneity was observed. However, due to the large clinical heterogeneity between papers (e.g. study domains, study designs and definition of outcomes), we decided that a random‐effects model was more appropriate.

Subgroup analysis and investigation of heterogeneity

We considered the following factors as potential explanations for heterogeneity and investigated them in separate subgroup analyses for our primary outcomes.    

• We hypothesized that the association of industry sponsorship and favorable outcomes may be larger in high risk of bias papers. We assessed overall risk of bias of the included papers using the criteria described in 'Assessment of risk of bias in included studies'. We regarded papers with adequate study inclusion, a comprehensive search and controlling for bias as having a low risk of bias; others as having a high risk. We compared low risk of bias papers with high risk of bias papers in a subgroup analysis.

• We compared papers of drug studies with device studies, as the mechanisms of influencing study outcomes may differ between the industries. For example, drug trials are more regulated than device trials, which could have an influence on biases in the design, conduct and reporting of the trials. We compared this in a subgroup analysis.   

• As the study domain might contribute to heterogeneity, we compared papers on specific treatments or diseases with papers of mixed domains in another subgroup analysis.

Sensitivity analysis

We undertook the following sensitivity analyses to test the robustness of our findings for our primary outcomes.

• The primary analyses compared the number of favorable results and conclusions in papers with industry sponsorship to those with other sources of sponsorship; 'industry sponsorship' included 100% pharmaceutical/device company funding, mixed funding and provision of drug or device only. 'Non‐industry sponsorship' included 100% government funding, 100% non‐industry funding and undisclosed funding. In a sensitivity analysis, we excluded those studies with mixed funding sources and those with funding consisting solely of free product from the 'industry sponsorship' category, and excluded studies with undisclosed funding from the category of 'non‐industry sponsorship', to determine if these had an impact on the initial analysis. As noted under 'Data extraction and management', we were reliant on how the studies in our review defined 'funding'.

• A sensitivity analysis restricted to papers that adjusted for confounders (e.g. adjusted for sample size and concealment of allocation using logistic regression) using adjusted estimates. We used the generic inverse variance method to pool adjusted odds ratios in a random‐effects model.

• A sensitivity analysis where all analyses were based on a fixed‐effect model.

• One paper (Finucane 2004), included unpublished abstracts from conference proceedings. One paper (Lynch 2007), included manuscripts submitted to a medical journal of which the majority were never published. However, based on the reported data it was not possible to extract data from the published studies separately. As these two papers included unpublished data and we planned to analyze only published data, we conducted a sensitivity analysis that excluded these two papers.

• Many of the included papers investigated similar domains (e.g. antidepressants or oncology drugs) or included studies from similar journals in overlapping time‐periods. This is likely to lead to double counting if data from the same studies are included more than once and to an overestimate of the precision of effect estimates. Due to the way data were reported in the papers, it was not possible ensure that data was not double counted. Instead we performed a sensitivity analysis restricted to papers on specific treatments or diseases where none of the other included papers were related to the same domain.

• We included letters and conference abstracts reporting quantitative data in a sensitivity analysis.

Quality of Evidence

We created a 'Summary of findings' table using the GRADE approach (Higgins 2011a) and using the MAGICapp software (MAGICapp; Vandvik 2013).

Results of the search

See: Figure 1.

After removal of duplicates, 3052 references were identified. From reading titles and abstracts, 2925 were eliminated as being not relevant to the review. Full‐text papers were obtained for 127 references. From these 127 papers, 104 papers were excluded (see Characteristics of excluded studies) and 23 included (see Characteristics of included studies). Four additional papers were included from searching additional sources and 48 were included from the previous version of the review (see Characteristics of included studies). In total, 75 papers were included.

Included studies

See: Characteristics of included studies.

The 75 papers were published between 1986 and 2015. Seventy‐two papers included mainly published studies, one included studies presented at a conference, one included studies submitted to a medical journal, and one included studies submitted to eight medical journals. Fifty‐seven papers included only drug studies, three only device studies, two drug and device studies and 13 included different types of interventions (e.g. drugs, devices, behavioral interventions). Thirty‐four papers included studies related to specific drug classes, 16 related to specific medical specialties or types of diseases (e.g. endocrinology), 10 related to a specific disease, three related to a specific type of device, 11 included all types of research studies, and one did not state the domain. Various aspects of medicine were covered, but 16 (21%) papers were restricted to psychiatric diseases or drugs and eight (11%) to cancer treatment. Fifty‐eight papers included only clinical trials, two only observational studies, and 15 both clinical trials and observational studies. Thirteen papers included only drug versus drug comparisons, eight only drug versus placebo, 49 mixed comparisons (e.g. drug versus drug, drug versus placebo) and five did not describe the kind of comparisons. The median number of included studies per paper was 105 (range: nine to 930). Of the 75 papers, 27 reported data on both favorable outcomes and risk of bias, 44 on favorable outcomes only, and four on risk of bias only. Twenty‐five papers were non‐industry funded, two were industry funded (Freemantle 2000; van Lent 2014), one was funded by both industry and non‐industry sources (Lynch 2007), 17 reported that they did not receive funding and 30 did not reported on funding.

Favorable results: industry sponsored versus non‐industry sponsored studies

See: Table 1

Twenty‐six papers, including 3081 studies (3062 drug studies and 19 device studies), reported on sponsorship and efficacy results; 25 could be combined in a pooled analysis. An analysis based on these 25 papers, including 2923 studies, found that industry sponsored studies more often had favorable efficacy results (e.g. those with significant P values) compared with non‐industry sponsored studies, risk ratio (RR): 1.27 (95% confidence interval (CI): 1.17 to 1.37), I²: 28% (Analysis 1.1). The paper that could not be included in the pooled analysis (Bhandari 2004), which had included 158 drug studies in general medicine, found similar results, odds ratio (OR): 1.6 (95% CI: 1.1 to 2.8).

Four papers, including 826 studies, did not find a difference in favorable harms results in industry sponsored studies compared with non‐industry sponsored studies, RR: 1.37 (95% CI: 0.64 to 2.93), I²: 96%. (Analysis 1.2). The results of one paper (Als‐Nielsen 2003), were opposite in direction to the other papers and resulted in the substantial heterogeneity. .

Favorable results: industry sponsorship by test treatment company versus industry sponsorship by comparator treatment company

Three papers, including 151 studies (all drug trials), compared efficacy results of trials sponsored by the manufacturer of the test treatment with trials sponsored by the manufacturer of the comparator treatment; two could be combined in a pooled analysis. An analysis based on these two papers (Bero 2007; Rattinger 2009), which included 131 industry sponsored trials of statins and thiazolidinediones, found that trials were much more likely to favor the test treatment when they were sponsored by the manufacturer of the test treatment than when they were sponsored by the manufacturer of the comparator treatment, RR: 3.88 (95% CI: 1.26 to 11.94), I²: 50% (Analysis 2.1). The paper that could not be included in the pooled analysis, which had included 20 selective serotonin reuptake inhibitor head‐to‐head trials, found that two trials favored the sponsor's drug, 18 had similar efficacy and none favored the comparator drug (Gartlehner 2010).

Favorable conclusions: industry sponsored versus non‐industry sponsored studies

See: Table 1

Thirty‐two papers, including 5258 studies (4761 drug studies and 497 device studies), reported on sponsorship and conclusions, 29 of which could be combined in a pooled analysis. An analysis based on these 29 papers, including 4583 studies (4179 drug studies and 404 device studies), found that industry sponsored studies more often had favorable conclusions than non‐industry sponsored studies, RR: 1.34 (95% CI: 1.19 to 1.51), I²: 92% (Analysis 3.1). Three papers could not be included in the pooled analysis due to the reporting of data. Of these, one paper reporting on 301 psychiatric drug studies (Kelly 2006) found that industry sponsored studies more often had favorable conclusions than non‐industry sponsored studies (P < 0.001) and similar findings were reported in a paper of 59 trials of antipsychotics (P = 0.02) (Montgomery 2004). A paper on 315 gastroenterology trials (222 drug trials and 93 device trials) did not find a difference in conclusions between industry sponsored trials and non‐industry sponsored trials (industry: 86% favorable, non‐industry: 83% favorable; P = 0.57) (Brown 2006).

Favorable conclusions: industry sponsorship by test treatment company versus sponsorship by comparator treatment company

Five papers, including 348 drug trials, compared conclusions of studies sponsored by the manufacturer of the test treatment with studies sponsored by the manufacturer of the comparator treatment, and three could be combined in a pooled analysis. These three papers (Bero 2007; Heres 2006; Rattinger 2009) including 154 industry sponsored trials of statins, antipsychotics and thiazolidinediones, found that trials were much more likely to favor the test treatment when they were sponsored by the manufacturer of the test treatment than when they were sponsored by the manufacturer of the control treatment, RR: 5.92 (95% CI: 2.80 to 12.54). No heterogeneity was observed (Analysis 4.1). A paper including 138 psychiatric drug studies (Kelly 2006) had similar findings, RR 2.80 (95% CI: 2.02 to 3.88), and a paper on 56 non‐steroidal anti‐inflammatory drug (NSAID) trials (Rochon 1994), found that 16 trials favored the sponsor's drug, 40 concluded that the drugs had similar effect and none favored the comparator drug.

Effect size: industry sponsored versus non‐industry sponsored studies

Twenty‐four papers, including 1517 studies (1476 drug studies and 41 device studies), reported on sponsorship and effect size, but could not be pooled due to differences in reporting of data. The results were heterogeneous and are described in Table 1 below.

Table 1. Effect size in industry and non‐industry sponsored studies.

Most papers (19 out of 24) compared effect sizes for efficacy results. Twelve of these papers, including 1131 drug studies, did not find a difference in effect sizes for efficacy estimates between industry sponsored studies and non‐industry sponsored studies. In contrast, seven papers, including 262 studies (221 drug studies and 41 device studies) found higher effect sizes of efficacy estimates in industry sponsored studies. Two papers, including 30 drug studies, did not find a difference in effect size of harms between industry sponsored studies and non‐industry sponsored studies. In contrast, two papers, including 36 drug studies, found lower effect size of harms in industry sponsored studies. Lastly, a paper on 58 industry‐sponsored head‐to‐head trials of antidepressants (Sinyor 2012), found that the sponsor's antidepressants were often given at a higher dose than the comparator. Effects on remission were higher when the sponsor's drug was given at a higher dose, as compared to trials in which the sponsor's drug was given in comparable or lower dose OR: 1.28 (95% CI: 1.11 to 1.47) versus OR: 1.06 (95% CI: 0.96 to 1.17) (P = 0.04).

Risk of bias: industry sponsored versus non‐industry sponsored studies

Twelve papers, including 1660 studies (1482 drug studies and 178 device studies), compared risk of bias in industry versus non‐industry studies using six different composite quality scales (Brown, Cho, Cochrane, Jadad, PEDro or Sackett) and the results were heterogeneous. Seven papers did not find a difference in risk of bias between industry sponsored and non‐industry sponsored studies (Cho 1996; Clark 2002; Corona 2014; Jefferson 2009; Lynch 2007; Sung 2013; Vlad 2007), whereas five papers found lower risk of bias (i.e. higher methodological quality scores) in industry sponsored studies (Brown 2006; Djulbegovic 2000; Montgomery 2004; Pengel 2009; Perlis 2005a).   

Nine papers, including 913 drug trials, did not find a difference in risk of bias from sequence generation in industry sponsored trials compared with non‐industry sponsored trials, RR: 0.99 (95% CI: 0.78 to 1.27), I²: 73% (Analysis 5.1). Sixteen papers, including 1886 trials (1867 drug trials and 19 device trials), did not find a difference in risk of bias from concealment of allocation in industry sponsored trials compared with non‐industry sponsored trials, RR: 1.06 (95% CI: 0.85 to 1.31), I²: 71% (Analysis 5.2). Thirteen papers, including 1578 trials (1559 drug trials and 19 device trials), found that industry sponsored trials more often had low risk of bias from overall blinding compared with non‐industry sponsored trials, RR: 1.25 (95% CI: 1.05 to 1.50), I²: 72% (Analysis 5.3). Three papers, including 128 drug trials, did not find a difference in risk of performance bias (e.g. blinding of clinicians or patients) in industry sponsored trials compared with non‐industry sponsored trials, RR: 1.26 (95% CI: 0.60 to 2.62), I²: 66% (Analysis 5.4). Four papers, including 307 drug trials, found that industry sponsored trials more often had low risk of detection bias (e.g. blinding of outcome assessors) compared with non‐industry sponsored trials, RR: 1.47 (95% CI: 1.02 to 2.12). No heterogeneity was observed (Analysis 5.5). Six papers, including 416 drug trials, did not find a difference in risk of bias from loss to follow‐up in industry sponsored trials compared with non‐industry sponsored trials, RR: 1.05 (95% CI: 0.92 to 1.18), I²: 2% (Analysis 5.6). Two papers, including 193 drug trials, did not find a difference in risk of reporting bias in industry sponsored trials compared with non‐industry sponsored trials, RR: 1.49 (95% CI: 0.61 to 3.60), I²: 79% (Analysis 5.7).

Concordance between study results and conclusions: industry sponsored versus non‐industry sponsored studies

Six papers, including 751 drug studies, reported on concordance between study efficacy results (e.g. as judged by their P values) and conclusions. Industry sponsored studies were less concordant than non‐industry sponsored studies, RR: 0.83 (95% CI: 0.70 to 0.98), I²: 63% (Analysis 6.1). One paper (Alasbali 2009), including 39 drug studies, found markedly higher lack of concordance in industry studies than the other four papers, and this was the reason for the high heterogeneity between papers.

One paper, of 211 corticosteroid studies with statistically significant harms results, found that industry sponsored studies more often concluded that the drug was safe than non‐industry sponsored studies, RR: 3.68 (95% CI: 2.14 to 6.33) (Nieto 2007).

Reasons for observed heterogeneity

For the association between sponsorship and favorable efficacy results of drug and device studies the data had acceptable heterogeneity, but heterogeneity for the association between sponsorship and harms results and study conclusions was substantial with an I² of 96% and 92%, respectively.

The reason for the large heterogeneity related to harms results is attributed to the results of the Als‐Nielsen paper (Als‐Nielsen 2003), that was opposite in direction to the other papers (i.e. industry had less favorable harms results as opposed to more favorable harms results) (Halpern 2005; Kemmeren 2001; Nieto 2007), and the interaction tests in the subgroup analyses were statistically significant. The Als‐Nielsen paper differs in some aspects from the three other included papers. First, it samples trials from various therapeutic areas, whereas the other papers each deal with harms results of single‐drug classes (HIV drugs, oral contraceptives and inhaled corticosteroids). Second, the three papers related to single‐drug classes had harms results as their primary outcome and only included studies with quantitative data, but Als‐Nielsen had harms as a secondary outcome and also included trials without quantitative harms data. The number of trials without harms data was high, particularly in the non‐industry group, 28% and 52%, respectively. Third, Als‐Nielsen only included trials, whereas Halpern 2005 included both trials and observational studies and Kemmeren 2001 and Nieto 2007 only included observational studies. Finally, we assessed Als‐Nielsen 2003 as having an overall low risk of bias, compared to high risk of bias in the other three papers.

In relation to the substantial heterogeneity for study conclusions, one reason was likely that the coding of favorable results was similar across the different papers, using statistical significance as the cut‐off, but coding varied for favorable conclusions. Some papers did not describe what they considered a favorable conclusion and this would involve some judgement. Others used scales, but for similar scales the cut‐off varied between papers. For example, on the same six‐point scale one paper used four as the cut‐off (Djulbegovic 2000) and another used six as the cut‐off (Als‐Nielsen 2003).

Also, the proportion of studies with favorable conclusions in the non‐industry sponsored group might have contributed to the size of the association and thereby the heterogeneity. For example, while the Chard and Liss papers (Chard 2000; Liss 2006) had a similar proportion of favorable industry sponsored studies (both 98%), they reported very different proportions of favorable non‐industry sponsored studies (32% and 97%) and this explains why the risk ratios reported in the two studies were not the same: RR: 3.03 in Liss and RR: 1.01 in Chard. Variations in the definition of favorable conclusions might explain why the risk ratios reported in the two papers were not similar. For example, in the Chard paper, a conclusion was coded as favorable if the study authors supported the use of the treatment, even in the absence of a statistically significant result.

Our subgroup analyses stratifying papers in relation to risk of bias (low versus high), type of intervention (drug versus device) or study domain (mixed versus specific treatments or diseases) did not explain the observed heterogeneity, though this was a simplistic comparison and other factors might also contribute to heterogeneity.

We found mixed results on the relationship between sponsorship and effect size, with most papers not finding a difference. All but one of these papers were restricted to specific treatments, which may explain the different findings. A recent study of systematic reviews of nine different drugs found that the influence of reporting biases on effect sizes varied considerably between drugs (Hart 2012). Furthermore, one paper found that even when adjusting for effect size, industry sponsored studies more often had favorable conclusions, compared with non‐industry sponsored studies (Als‐Nielsen 2003). Therefore, while the direction of the relationship between sponsorship and favorable outcomes was consistent, the size of the effect likely varies depending on the type of treatment or treated condition.

Reasons for favorable outcomes in industry sponsored studies

The pharmaceutical and medical device industries have strong interests in scientific publications that present their products positively, as publications are the basis of regulatory, purchasing, and medical decisions. These interests can influence the design, conduct and publication of studies in ways that make the sponsor’s product appear better than the comparator product (Bero 1996).

Several possible factors can explain the relationship between industry sponsorship and favorable outcomes. It has been argued that since many industry sponsored studies are undertaken to fulfill regulatory requirements, industry sponsored studies could have a lower risk of bias than non‐industry sponsored studies (Rosefsky 2003). Even if this were true, it would not explain the association of industry sponsorship and favorable efficacy results and conclusions. In addition, we did not find evidence for differences in risk of bias except in relation to blinding, where industry sponsored trials tended to have a lower risk of bias, even when restricted to head‐to‐head trials (Bero 2007). The papers comparing blinding between trials with different sponsorship often used a description of double blinding as an indicator for low risk of bias. Double blinding is an inconsistent term and does not ensure that, for example, outcome assessors are blinded (Devereaux 2001). The more frequent use of double blinding may therefore be a reporting issue, with industry trials being better reported. This is further substantiated by the fact that nearly all the papers finding a higher methodological quality score in industry studies used the Jadad scale, a scale which has been criticized for having more focus on the quality of reporting than on methodological quality (Lundh 2008).   

A few papers assessing a more specific definition of blinding related to performance bias and detection bias also found that industry sponsored studies had lower risk of bias. Evidence also suggests that for non‐industry trials, companies may prevent proper blinding by restricting access to placebo drugs (Christensen 2012), and therefore differences in adequate blinding may be real. In addition, double blinding can be used as a proxy for low risk of bias and trials without double blinding are on average more likely to have favorable results (Pildal 2007). The effect of this bias is in the opposite direction of our findings, as it would lead to industry sponsored studies having less favorable results and conclusions, and our findings can therefore, not be explained by differences in risk of bias related to blinding between industry and non‐industry sponsored studies.

Another possible explanation for our findings could be that industry studies have larger sample sizes, and would have a higher chance of achieving statistically significant results. Although industry trials seem in general to be of larger size (Als‐Nielsen 2003; Booth 2008; Bourgeois 2010; Djulbegovic 2013; Etter 2007; Flacco 2015; Perlis 2005a), when we restricted our analysis to studies controlling for sample size and other confounders, the relationship between industry sponsorship and favorable results or conclusions was still present.

Industry representatives argues that the trials they sponsor are more likely to have favorable outcomes because they fund research that has a high chance of achieving success (Palmer 2003). However, when independent investigators conduct non‐industry sponsored trials, they in most cases test treatments that have been approved based on favorable industry trial results. Non‐industry sponsored trials would therefore also be expected to achieve successful results, unless they are designed to answer different questions than industry sponsored trials. For example, non‐industry sponsored studies may test a new treatment against a well‐established treatment, while industry sponsored studies might test the new treatment against placebo or against an outdated, inferior treatment.

Accordingly, it seems most plausible that industry achieves overly positive results through a variety of biasing choices in the design, conduct and reporting of their studies. For example, industry protocols might include inferior comparators that will increase the chance of their product’s success. Djulbegovic and colleagues (Djulbegovic 2003) have argued that industry sponsored studies violate equipoise by choosing inferior competing treatment alternatives. Previous studies have found that industry sponsored trials more often use placebo control (Als‐Nielsen 2003; Djulbegovic 2000; Dunn 2013; Estellat 2012; Katz 2006; Lathyris 2010), active comparators in inferior doses (Rochon 1994; Safer 2002; Sinyor 2012), or inappropriate administration of the drugs (Johansen 1999). Industry may also selectively choose less clinical relevant outcomes as their primary outcome in order to get a higher chance of achieving an effect. For example, in the paper by Djulbegovic (Djulbegovic 2013) industry sponsored trials had higher effect size than non‐industry sponsored trials on primary outcomes, but not overall survival. This could also be one of the possible explanations as to why industry sponsored trials more often have favorable results and conclusions while the effect sizes are often similar when comparing similar outcomes.

Industry sponsored studies may also be biased in the coding of events and their data analysis (Furukawa 2004; Psaty 2008; Psaty 2010). Industry and its sponsored investigators also may selectively report favorable outcomes, fail to publish whole studies with unfavorable results, or publish studies with favorable results multiple times (Chan 2004; Dwan 2008; Gøtzsche 2011; McGauran 2010; Melander 2003; Rising 2008; Vedula 2009). While such biases in analyses and reporting have been documented in a number of cases, the papers included in this review focused on comparisons of published studies. Only two papers (Killin 2014; Naci 2014) included in our review compared risk of selective reporting between industry sponsored trials and non‐industry sponsored trials and found no difference. However, confidence intervals were wide and the analysis was limited to two types of drugs (donepezil for Alzheimer's disease and statins). Therefore, we are unable to determine the extent to which selective analysis or reporting contribute to our findings. Similarly, we found no difference in harms results between industry sponsored and non‐industry sponsored studies. Under‐reporting of harms seems to be a major problem in both industry and non‐industry sponsored studies with 28% of trials included in Als‐Nielsen 2003 not reporting any harms data, which is in line with a recent systematic review that found that a median of 54% studies do not report harms data (Golder 2016).

Favorable conclusions in industry‐sponsored trials may also be reached by over‐interpreting results and use of spin in conclusions (Boutron 2010). We found that industry sponsored studies had less concordance between results and conclusions compared with non‐industry sponsored studies, suggesting that conclusions of industry sponsored studies are less reliable.

It should also be noted that some studies in the non‐industry group likely had authors with conflicts of interest related to the pharmaceutical or device industry, which may have influenced their interpretation of study results (Stelfox 1998; Wang 2010), thereby diluting the measured effect of industry bias on study conclusions. Also, we coded studies as non‐industry sponsored if they did not state who sponsored the study. As some of these studies were likely industry sponsored, this misclassification will have led to similar bias towards the null. However, in our sensitivity analyses, we excluded studies without sponsorship statements and did not see a change in results.

Further evidence for industry bias stems from our comparison of studies sponsored by the manufacturer of the test treatment with those sponsored by the manufacturer of the control treatment. These studies had the advantage of comparing like with like, as they are restricted to specific drug classes or types of devices and have similar methodologies. Though limited to only three papers on drug trials, the findings show associations that are stronger than the comparison between industry and non‐industry sponsored studies. These comparisons are restricted to drugs competing for the same market, which may put pressure on companies to influence outcomes to a greater degree than what is needed in placebo‐controlled trials to present the drug in a good light.

In sum, the industry bias associated with favorable efficacy results and conclusions may be mediated by factors other than traditional measures of the risk of bias (e.g. lack of concealment of allocation, blinding and dropout) and sample size. This industry bias may be partially mediated by such factors as the choice of comparators, dosing and timing of comparisons, choice of outcomes, selective analysis, and selective reporting.