Cancer, meta-analysis and reporting biases: the case of erythropoiesis-stimulating agents

DOI: https://doi.org/10.4414/smw.2013.13776

Meta-analyses and the threat of reporting and publication biases

The large number of studies conducted in medical research and their sometimes conflicting results make it difficult for healthcare professionals and policy makers to reach well-informed decisions. Meta-analyses offer a possible solution, as they provide an overall effect estimate by pooling results from different studies dealing with the subject in question. In this way they may overcome problems that individual studies might have, such as small sample size and low statistical power, and may provide answers when the results of individual studies are conflicting. A correctly conducted meta-analysis should be preceded by a systematic review of the literature, which aims to identify all the available studies on the subject. It is conducted in a transparent and replicable manner, and the studies are selected and evaluated according to predefined criteria.

Meta-analyses, however, might also yield misleading results. There are three main reasons why this could happen: (a) poor quality of the included studies; (b) heterogeneity between the studies; and (c) publication and reporting biases [1]. As each individual study affects the overall result of a meta-analysis, its quality should be assessed and taken into account. Study quality could be evaluated by using various criteria such as method of randomisation and allocation concealment, blinding of patients and outcome assessors, use of intention-to-treat analysis, etc. In addition, a high degree of heterogeneity between studies can also affect the overall result and researchers should seek to explain heterogeneity by stratifying results according to prespecified groups, for instance according to type of cancer, or type of chemotherapy, etc.

The term “publication bias” refers to the fact that not all studies are published and that publication of studies follows a pattern that cannot be attributed to chance: studies with positive results tend to be published more often and earlier than studies with negative results [2, 3]. A longitudinal study of the publication outcome of clinical trial protocols submitted to five ethics committees showed that trials were 2.6 times more likely to be published if they reported statistically significant results [4]. Bourgeois and colleagues [5], on the basis of a sample of more than 500 registered clinical trials, found that studies funded by industry were more likely to be published if they reported positive results. Reporting bias refers to the selective reporting of outcomes in a published study. For instance, some outcomes that had been defined in the protocol might not be reported [6], or the results reported are derived from per-protocol analyses rather than intention-to-treat analyses [7]. This kind of bias also seems to follow a pattern, whereby mainly the most statistically significant results or the ones meeting the authors’ assumptions are reported [8].

The case of erythropoiesis-stimulating agents in cancer patients

In 2002 we conducted a meta-analysis on the effects of erythropoiesis-stimulating agents (ESAs) in cancer patients. ESAs are recombinant human proteins resembling erythropoietin, which is a human hormone produced mainly in the kidney and which stimulates the production of red blood cells. ESAs were licensed to treat cancer-related anaemia in 1990. There are several short- or long-acting ESAs, with the most commonly used ones being epoetin-α or epoetin-β and darbepoetin. Since their introduction for the treatment of cancer-related anaemia, numerous randomised controlled trials and systematic reviews have been conducted, in order to assess their efficacy and their potential side effects [9–28]. Our first meta-analysis in 2002 [12] showed that besides reducing the risk and the need for red blood cell transfusions, ESAs might also improve overall survival (see fig. 1). The result was mainly driven by one large study conducted by Littlewood and colleagues in 2001 [29]. The study produced some evidence that patients with solid and haematological malignancies who received ESAs experienced improved overall survival compared with controls. The p-value of the trial was of borderline significance (p = 0.052). However, the endpoint of overall survival was introduced as a protocol amendment after the study had been started. Including adjusted data from this trial and data from another 18 trials with a total of 2,805 patients, our meta-analysis showed some evidence for improved overall survival in patients receiving ESAs (hazard ratio (HR) 0.81, 95% confidence interval (CI) 0.67‒0.99 using adjusted survival estimates and HR 0.84, 95% CI 0.69‒1.02 using unadjusted survival estimates from Littlewood et al. [29]). In other words, when we pooled the results from individual studies for our first meta-analysis, we found evidence for an overall survival benefit for cancer patients receiving ESAs. Of note, there was no evidence of a publication bias (p-value for regression test = 0.481) (see fig. 2). This finding was supported by biological hypotheses. There was evidence from animal models and studies in humans that anaemia might result in a poorer response to chemotherapy or radiotherapy owing to increased tumour hypoxia [30–34]. Consequently, researchers hypothesised that targeting cancer-related anaemia might also improve tumour response and overall survival.

Figure 1

Forest plot for overall survival based on data from our first meta-analysis [12].

Figure 2

Funnel plot for overall survival based on data from our first meta-analysis [12].

However, already in 2003 the findings of our first meta-analysis were contradicted by two large randomised controlled trials that showed reduced survival for the ESA group in patients with metastatic breast cancer receiving chemotherapy [35] and patients with head and neck cancers receiving radiotherapy [36]. These unfortunate findings were followed by several other trials showing detrimental survival results for cancer patients receiving ESAs, as compared with controls [37–41]. When updating our meta-analysis on overall survival in 2006, these findings from single studies were consolidated but still uncertain: the HR for overall survival was 1.08 (95% CI 0.99‒1.18, 42 trials and 8,167 patients), making the overall survival benefit we had previously reported questionable. In this situation we decided to retrieve the individual patient data (IPD) for all randomised controlled trials (RCTs) of the effects of ESAs compared with standard care in cancer patients. We invited independent investigators and pharmaceutical companies (Johnson & Johnson, Roche, Amgen) to participate in this project and to contribute their trial data. In 2008 we received the IPD as requested, as well as corresponding study protocols, protocol amendments and clinical study reports for 53 RCTs including 13,933 patients [28]. Based on these data and documents we conducted a meta-analysis on the effects of ESAs on on-study mortality and overall survival, which was published in 2009 [28]. The use of IPD allowed us to overcome certain limitations of literature-based meta-analyses, such as the use of aggregated data, heterogeneous endpoints across studies and the inability to assess differences across subgroups. For this update we differentiated mortality during the active study period from overall survival, defined as deaths during the longest follow up available. This update confirmed the previously noted trends: ESAs increased on-study mortality in cancer by a factor of 1.17 (HR 1.17, 95% CI 1.06‒1.30) and there was some evidence for reduced overall survival (HR 1.06, 95% CI 1.00‒1.12).

A hypothetical study

Introduction

To determine whether the results of our first meta-analysis [12] had been affected by publication and reporting biases, and if timely access to clinical study reports and IPD could have prevented this, we conducted a hypothetical meta-analysis for overall survival and on-study mortality including studies and study data which could have been available in 2002, at the time when we conducted our first meta-analysis.

Figure 3

Forest plot for overall survival based on data from our first meta-analysis [12] adding missing studies and using results from individual patient data review [27, 28].

Figure 4

Funnel plot for overall survival based on data from our first meta-analysis [12] adding missing studies and using results from individual patient data review [27, 28].

Methods

In this hypothetical meta-analysis, we included studies that recruited patients until the end of 1999. We chose this timeframe because it closely corresponds to the studies we included in the first, literature-based meta-analysis, that is, studies published up to the beginning of 2002. We identified eligible studies on the basis of the meta-analyses conducted previously [12, 14, 28]. In these meta-analyses [12, 14, 28] we included randomised controlled trials comparing ESAs with placebo or best usual care in cancer patients who were anaemic or at risk of developing cancer-related anaemia, and who were or were not receiving anticancer treatment. We conducted systematic searches using electronic databases (Medline, Embase, Cochrane Library and databases of conference proceedings), as well as documents from Oncology Drug Advisory Committee (ODAC) hearings [42]. For the first, literature-based review, we also obtained unreported data from the investigators. For the IPD review we obtained raw study data, study protocols, their amendments and clinical study reports from manufacturers and independent investigators. For the literature-based meta-analyses we used overall survival defined as the longest follow up available. For the IPD meta-analysis we used overall survival and in addition on-study mortality defined as death from any cause during the active study period plus 30 days. Details of these meta-analyses are reported elsewhere [12, 14, 28].

For the hypothetical meta-analysis we extracted overall survival and on-study mortality data from these three meta-analyses [12, 14, 28]. Whenever we had more than one result for a given study the following hierarchy applied: (1.) data and results taken from IPD review; (2.) data and results provided by study authors for previous meta-analyses; (3.) data and results as reported in the literature. We compared results from the different reviews for a given study, discrepant data and results were noted and reasons for discrepancies investigated and reported. Overall survival and on-study mortality was assessed as time-to-event analysis using hazard ratios. We calculated effect estimates and 95% confidence intervals (CIs) using fixed effects models.

Results

Our first, literature-based meta-analysis evaluated 19 trials including 2,805 patients for overall survival [12]. We obtained overall survival data either from the literature [29, 43–45] or from study investigators [46–60]. We found a HR of 0.81 (95% CI 0.67‒0.99, figs 1 and 2), favouring ESAs. For the hypothetical meta-analysis we identified 25 eligible studies (3,664 patients in total; 2,155 randomised to ESA; 1,509 randomised to control), which recruited patients up to the end of 1999. Of these, 19 trials had been included in the overall survival analysis of the first meta-analysis. Another six studies were not included in the first meta-analysis for the following reasons: One study [61] had been included in the first review [12], however, since survival data were not reported or provided by the study authors, the study was not included in the overall survival analysis. Another study [62] was published after 2002 and was therefore not identified for the first review [12]. Four studies [63–66] were completed at the time of the first review [12], but survival or any other study data had not yet been published. The survival results for these studies became available through ODAC reports published in 2003 [67] and the IPD meta-analysis published in 2009 [28]. Five of these six studies showed detrimental findings, that is, worse survival in patients receiving ESAs than in controls. For the 19 studies which had already been included in the first review [12] we compared the results reported in the different sources. For the six studies [43, 45, 49, 54, 55, 58], no additional data were retrieved for the IPD review because the studies were too small for inclusion [27, 28]. The results for these six studies [43, 45, 49, 54, 55, 58] are therefore identical in the first review and the hypothetical meta-analysis. For 13 studies [29, 44, 46–48, 50–53, 56, 57, 59, 60] we had results stemming from the first review [12] and the IPD meta-analysis [28]. For three studies the number of patients randomised and the number of events were identical and there were only minor differences in effect estimates (HRs or CIs) [29, 46, 48]. For the other ten studies we noted discrepant results, that is, the data reported in the literature or provided by the study authors for the first review [12] were not identical to the overall survival estimates generated in the IPD review [28]. Discrepancies concerned the number of patients randomised per study arm, the number of deaths per study arm and the reported HRs plus CIs, for details see attached data table 1. Differences were explained by differing definitions of end of study in three studies [47, 50, 52], longer follow up or updated study information available for IPD for four studies [44, 57, 59, 60]. For two studies IPD analysis was based on intention-to-treat whereas previously included results were based on a safety population including only patients who had received at least one dose of the drug [53, 56]. For another study differences are explained by use of different inclusion criteria in the first review and the IPD review [51]. The results obtained for the IPD review [28] were worse (a HR which was less favourable than the previously reported HR) in seven studies [44, 47, 50, 51, 56, 57, 59] and better (a HR which was more favourable compared to the previously reported HR) in three studies [52, 53, 60].

For the hypothetical meta-analysis we included the available data stepwise into the original review conducted in 2002. Firstly, we added published data for “late” publications [62] and IPD for unreported outcomes from published [61, 63] and unpublished studies [64–66]. This resulted in the addition of six more studies, which means that in 2002 24% of the available studies were missing. Five out of these six studies were sponsored by pharmaceutical companies. By including these missing studies the hazard ratio changed from 0.81 (95% CI 0.67‒0.99) to 0.88 (95% CI 0.73‒1.05). Moving one step further, we replaced the published data of our first analysis [12] with IPD [27, 28]. This resulted in replacing data from 13 studies (or from 52% of all the included studies). The new HR was 0.97 (95% CI 0.83‒1.12, figs 3 and 4), showing no overall survival benefit for patients receiving ESAs compared to controls. In a sensitivity analysis we conducted a similar analysis for on-study mortality and found similar results (data on file).

Discussion

Compared with our original meta-analysis, which suggested an overall survival benefit for cancer patients receiving ESAs (HR 0.81, 95% CI 0.67‒0.99) [12], our hypothetical meta-analyses based on the results of all studies conducted at the time of the first analysis did not show evidence for a beneficial effect of ESAs on overall survival (HR 0.97; 95% CI 0.83‒1.12). This suggests that our first meta-analysis [12] showed misleading overall survival benefit owing to publication and reporting biases, which could have been prevented by timely access to IPD and clinical study reports. Based on a series of meta-analyses with access to published reports and IPD, our study demonstrates the impact of publication and reporting biases on the perceived effects of ESAs on survival in cancer patients. We were able to show this effect by using a large number of studies. Compared with our original analysis we increased the number of studies by 25% and replaced 52% of the data with results generated in an IPD meta-analysis [27, 28]. Our case study, however, has limitations. For six studies [43, 45, 49, 54, 55, 58] we did not have IPD; however, these studies were small and a large effect on the overall results is unlikely. We investigated overall survival and on-study mortality, and we do not know whether the publication and reporting biases would affect other outcomes in a similar way. However, we recently updated several clinical outcomes for our Cochrane Review [14, 68] based on the published literature. With few exceptions study outcomes were underreported leading to overestimates of both beneficial and harmful effects of ESAs [68].

How to address publication and reporting biases?

Our case study adds another example to the large body of evidence on publication and reporting biases [2–4, 8, 69–71]. Funnel plots offer a way to explore the probability of publication bias; however, they only demonstrate but do not solve potential publication biases. They can also produce false negative results. For example, the funnel plot generated for our first meta-analysis did not indicate a publication bias. In 2004, the International Committee of Medical Journal Editors produced a statement requiring trials to have been registered in a registry before being considered for publication [72]. Two years after its implementation, more trials were registered than in the past [73]. However, this measure did not help to reduce selective reporting of outcomes leading to reporting bias, since this statement did not require reporting of study results. A study by Huic et al. [74] assessed about 150 reports from RCTs and found that almost 40% of the assessed trials had discrepancies between their registered and reported primary outcomes, and almost 65% had discrepancies in the secondary outcomes. Most of these discrepancies were due to post-hoc reporting of outcomes in the publications that had not been listed in the registry. There is some evidence that, specifically for adverse events, unpublished data might allow a more accurate estimate than the published data [75]. This kind of discrepancy has led to the recognition that not only registration of the trial but also complete reporting of study results needs to be ensured. The US FDA Amendments Act of 2007 requires the publication of the main trial results in a standardised summary report [76–79]. However, it does not require the publication of the entire clinical study report or the release of the IPD. The European Medicines Agency (EMA), recognising that the current state of disclosure of important study documents is no longer desirable [80], has changed its policy on document access [81]. With the new policy the EMA will release documents upon request provided that the EMA’s authorisation procedures on a given drug have been finalised.

There is an on-going debate as to whether the release of clinical study reports including tabulated listings of anonymised IPD is sufficient, or whether IPD in electronic databases suitable for re-analysis of the data are needed [80, 82]. Access to IPD permits researchers to define uniform study endpoints across studies, to conduct intention-to-treat analyses standardised across trials and to include additional information gained through longer follow up [83]. Access to well-reported clinical study protocols and reports in addition to the IPD are essential to fully understand the nature and quality of the IPD [84, 85]. Whether, in contrast, access to clinical study reports providing summary estimates would be sufficient to allow an adequate evaluation of the drug remains questionable and has to our best knowledge not yet been investigated.

Is meta-analysis losing its credibility?

In a recently published opinion paper in Nature Medicine the authors pointed out that with a lack of robust evidence and a reasonable minimum amount of data, meta-analyses could lose credibility [86]. We agree that robust data are needed to generate meta-analyses that help to make well-informed decisions on healthcare interventions. We also agree that meta-analyses based on biased data sets can produce distorted and misleading results. However, we would like to emphasise that a lack of robust data, either due to a lack of high quality studies or due to reporting and publication biases, is not a problem of meta-analysis per se; it is a problem of healthcare and medical research. Meta-analyses together with funnel plots and regression tests are instruments that allow the assessment of the quality of the studies and detection of publication or reporting biases, actually making the absence of studies and study results visible. If the published data are so incomplete that we cannot use them for a meta-analysis how can we possibly use them to make healthcare decisions for patients, with or without meta-analysis?

Our study is yet another indication that publication and reporting biases are main obstacles to the efficient use of healthcare interventions. We strongly believe that these problems cannot be eliminated by the registration of trials and the standardised but limited reporting of trial results. Unrestricted access to clinical study protocols including amendments, to clinical study reports as well as to IPD is needed to ensure timely detection of both beneficial and harmful effects of health care interventions.

References

  1 Nordmann A, Kasenda B, Briel M. Meta-analyses: what they can and cannot do. Swiss Med Wkly. 2012;142(w13518).

  2 Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ. 1997;315(7109):640–5.

  3 Song F, Parekh S, Hooper L, Loke Y, Ryder J, Sutton A, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):1–220.

  4 von Elm E, Rollin A, Blumle A, Huwiler K, Witschi M, Egger M. Publication and non-publication of clinical trials: longitudinal study of applications submitted to a research ethics committee. Swiss Med Wkly. 2008;138(13-14):197–203.

  5 Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010;153(3):158–66.

  6 Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753.

  7 Hollis S, Campbell F. What is meant by intention to treat analysis? Survey of published randomised controlled trials. BMJ. 1999;319(7211):670–4.

  8 Norris SL, Holmer HK, Ogden LA, Fu R, Abou-Setta AM, Viswanathan MS, et al. Selective Outcome Reporting as a Source of Bias in Reviews of Comparative Effectiveness. 2012 Aug.

  9 Aapro M, Scherhag A, Burger HU. Effect of treatment with epoetin-beta on survival, tumour progression and thromboembolic events in patients with cancer: an updated meta-analysis of 12 randomised controlled studies including 2301 patients. Br J Cancer. 2008;99(1):14–22.

10 Bennett CL, Silver SM, Djulbegovic B, Samaras AT, Blau CA, Gleason KJ, et al. Venous thromboembolism and mortality associated with recombinant erythropoietin and darbepoetin administration for the treatment of cancer-associated anemia. JAMA. 2008;299(8):914–24.

11 Seidenfeld J, Piper M, Flamm C, Hasselblad V, Armitage JO, Bennett CL, et al. Epoetin treatment of anemia associated with cancer therapy: a systematic review and meta-analysis of controlled clinical trials. J Natl Cancer Inst. 2001;93(16):1204–14.

12 Bohlius J, Langensiepen S, Schwarzer G, Seidenfeld J, Piper M, Bennett C, et al. Recombinant human erythropoietin and overall survival in cancer patients: results of a comprehensive meta-analysis. J Natl Cancer Inst. 2005;97(7):489–98.

13 Wilson J, Yao GL, Raftery J, Bohlius J, Brunskill S, Sandercock J, et al. A systematic review and economic evaluation of epoetin alpha, epoetin beta and darbepoetin alpha in anaemia associated with cancer, especially that attributable to cancer treatment. Health Technol Assess. 2007;11(13):1-iv.

14 Bohlius J, Wilson J, Seidenfeld J, Piper M, Schwarzer G, Sandercock J, et al. Erythropoietin or darbepoetin for patients with cancer. Cochrane Database Syst Rev 2006;3:CD003407.

15 Bohlius J, Wilson J, Seidenfeld J, Piper M, Schwarzer G, Sandercock J, et al. Recombinant human erythropoietins and cancer patients: updated meta-analysis of 57 studies including 9353 patients. J Natl Cancer Inst. 2006;98(10):708–14.

16 Seidenfeld J, Piper M, Bohlius J, Weingart O, Trelle S, Engert A, et al. Comparative effectiveness of epoetin and darbepoetin for managing anemia in patients undergoing cancer treatment. Comparative effectiveness Review No. 3. (Prepared by Blue Cross and Blue Shield Association Technology Evaluation Center. Evidence-based Practice Center under contract No. 290-02-0026). Rockville, MD. Agency for Healthcare Research and Quality. May 2006. Available at: www.effectivehealthcare.ahrq.gov/reports/final.cfm.

17 Tonelli M, Hemmelgarn B, Reiman T, Manns B, Reaume MN, Lloyd A, et al. Benefits and harms of erythropoiesis-stimulating agents for anemia related to cancer: a meta-analysis. CMAJ. 2009;180(11):E62–E71.

18 Tonelli M, Lloyd A, Lee H, Wiebe N, Hemmelgarn B, Reiman T, et al. Erythropoiesis-stimulating agents for anemia of cancer or of chemotherapy: systematic review and economic evaluation. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2009. Report No.: Technology report number 119. -. p.

19 Ross SD, Allen IE, Henry DH, Seaman C, Sercus B, Goodnough LT. Clinical benefits and risks associated with epoetin and darbepoetin in patients with chemotherapy-induced anemia: a systematic review of the literature. Clin Ther. 2006;28(6):801–31.

20 Minton O, Richardson A, Sharpe M, Hotopf M, Stone P. A systematic review and meta-analysis of the pharmacological treatment of cancer-related fatigue. J Natl Cancer Inst. 2008;20;100(16):1155–66.

21 Minton O, Richardson A, Sharpe M, Hotopf M, Stone P. Drug therapy for the management of cancer-related fatigue. Cochrane Database Syst Rev. 2010;(7):CD006704.

22 Aapro M, Coiffier B, Dunst J, Osterborg A, Burger HU. Effect of treatment with epoetin beta on short-term tumour progression and survival in anaemic patients with cancer: A meta-analysis. Br J Cancer. 2006;95(11):1467–73.

23 Aapro M, Osterwalder B, Scherhag A, Burger HU. Epoetin-beta treatment in patients with cancer chemotherapy-induced anaemia: the impact of initial haemoglobin and target haemoglobin levels on survival, tumour progression and thromboembolic events. Br J Cancer. 2009;101(12):1961–71.

24 Ludwig H, Crawford J, Osterborg A, Vansteenkiste J, Henry DH, Fleishman A, et al. Pooled analysis of individual patient-level data from all randomized, double-blind, placebo-controlled trials of darbepoetin alfa in the treatment of patients with chemotherapy-induced anemia. J Clin Oncol. 2009;27(17):2838–47.

25 Glaspy J, Crawford J, Vansteenkiste J, Henry D, Rao S, Bowers P, et al. Erythropoiesis-stimulating agents in oncology: a study-level meta-analysis of survival and other safety outcomes. Br J Cancer. 2010;19;102(2):301–15.

26 Aapro M, Jelkmann W, Constantinescu SN, Leyland-Jones B. Effects of erythropoietin receptors and erythropoiesis-stimulating agents on disease progression in cancer. Br J Cancer. 2012;106(7):1249–58.

27 Bohlius J, Schmidlin K, Brillant C, Schwarzer G, Trelle S, Seidenfeld J, et al. Recombinant human erythropoiesis-stimulating agents and mortality in patients with cancer: a meta-analysis of randomised trials. Lancet. 2009;373(9674):1532–42.

28 Bohlius J, Schmidlin K, Brillant C, Schwarzer G, Trelle S, Seidenfeld J, et al. Erythropoietin or Darbepoetin for patients with cancer – meta-analysis based on individual patient data. Cochrane.Database.Syst.Rev. 2009;(3):CD007303.

29 Littlewood TJ, Bajetta E, Nortier JW, Vercammen E, Rapoport B. Effects of epoetin alfa on hematologic parameters and quality of life in cancer patients receiving nonplatinum chemotherapy: results of a randomized, double-blind, placebo-controlled trial. J Clin Oncol. 2001;19(11):2865–74.

30 Frommhold H, Guttenberger R, Henke M. The impact of blood hemoglobin content on the outcome of radiotherapy. The Freiburg experience. Strahlenther Onkol. 1998;174(Suppl 4):31–4.

31 Hockel M, Knoop C, Schlenger K, Vorndran B, Baussmann E, Mitze M, et al. Intratumoral pO2 predicts survival in advanced cancer of the uterine cervix. Radiother Oncol. 1993;26(1):45–50.

32 Knocke TH, Weitmann HD, Feldmann HJ, Selzer E, Potter R. Intratumoral pO2-measurements as predictive assay in the treatment of carcinoma of the uterine cervix. Radiother Oncol. 1999;53(2):99–104.

33 Glaser CM, Millesi W, Kornek GV, Lang S, Schull B, Watzinger F, et al. Impact of hemoglobin level and use of recombinant erythropoietin on efficacy of preoperative chemoradiation therapy for squamous cell carcinoma of the oral cavity and oropharynx. Int J Radiat Oncol Biol Phys. 2001;50(3):705–15.

34 Silver DF, Piver MS. Effects of recombinant human erythropoietin on the antitumor effect of cisplatin in SCID mice bearing human ovarian cancer: A possible oxygen effect. Gynecol Oncol. 1999;73(2):280–4.

35 Leyland-Jones B. Breast cancer trial with erythropoietin terminated unexpectedly. Lancet Oncol. 2003;4(8):459–60.

36 Henke M, Laszig R, Rube C, Schafer U, Haase KD, Schilcher B, et al. Erythropoietin to treat head and neck cancer patients with anaemia undergoing radiotherapy: randomised, double-blind, placebo-controlled trial. Lancet. 2003;362(9392):1255–60.

37 Smith RE, Jr., Aapro MS, Ludwig H, Pinter T, Smakal M, Ciuleanu TE, et al. Darbepoetin alpha for the treatment of anemia in patients with active cancer not receiving chemotherapy or radiotherapy: results of a phase III, multicenter, randomized, double-blind, placebo-controlled study. J Clin Oncol. 2008;26(7):1040–50.

38 Hedenus M, Adriansson M, San MJ, Kramer MH, Schipperus MR, Juvonen E, et al. Efficacy and safety of darbepoetin alfa in anaemic patients with lymphoproliferative malignancies: a randomized, double-blind, placebo-controlled study. Br J Haematol. 2003;122(3):394–403.

39 Overgaard J, Hoff CM, Hansen HS, Specht L, Overgaard M, Grau C, et al. Randomized study of darbepoetin alfa as modifier of radiotherapy in patients with primary squamous cell carcinoma of the head and neck (HNSCC): Final outcome of the DAHANCA 10 trial. In 2009. p. abstr 6007.

40 Wright JR, Ung YC, Julian JA, Pritchard KI, Whelan TJ, Smith C, et al. Randomized, double-blind, placebo-controlled trial of erythropoietin in non-small-cell lung cancer with disease-related anemia. J Clin Oncol. 2007;20;25(9):1027–32.

41 Goss G, Feld R, Bezjak A, Perry G, Melosky B, Smith C, et al. O-154 Impact of maintaining Hb with epoetin alfa on time toprogression (TTP), overall survival (OS), quality of life (QOL) and transfusion reduction in limited disease SCLC patients. Lung Cancer 49[Supplement 2], S53. 2005.

42 Luksenburg H, Weir A, Wager R. Safety concerns associated with Aranesp (darbepoetin alfa) Amgen, Inc. and Procrit (epoetin alfa) Ortho Biotech, L.P., for the treatment of anemia associated with cancer chemotherapy. 2004. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research, Oncologic Drugs Advisory Committee.

43 Dunphy FR, Harrison BR, Dunleavy TL, Rodriguez JJ, Hilton JG, Boyd JH. Erythropoietin reduces anemia and transfusions: A randomized trial with or without erythropoietin during chemotherapy. Cancer. 1999;86(7):1362–7.

44 Thatcher N, De Campos ES, Bell DR, Steward WP, Varghese G, Morant R, et al. Epoetin alpha prevents anaemia and reduces transfusion requirements in patients undergoing primarily platinum-based chemotherapy for small cell lung cancer. Br J Cancer. 1999;80(3-4):396–402.

45 Thompson JA, Gilliland DG, Prchal JT, Bennett JM, Larholt K, Nelson RA, et al. Effect of recombinant human erythropoietin combined with granulocyte/ macrophage colony-stimulating factor in the treatment of patients with myelodysplastic syndrome. GM/EPO MDS Study Group. Blood. 2000;95(4):1175–9.

46 Dammacco F, Castoldi G, Rodjer S. Efficacy of epoetin alfa in the treatment of anaemia of multiple myeloma. Br J Haematol. 2001;113(1):172–9.

47 Abels R. Erythropoietin for anaemia in cancer patients. Eur J Cancer. 1993;29A(Suppl 2):S2-8.:S2–S8.

48 Case DC, Jr., Bukowski RM, Carey RW, Fishkin EH, Henry DH, Jacobson RJ, et al. Recombinant human erythropoietin therapy for anemic cancer patients on combination chemotherapy. J Natl Cancer Inst. 1993;19;85(10):801–6.

49 Cascinu S, Fedeli A, Del Ferro E, Fedeli SL, Catalano G. Recombinant Human Erythropoietin Treatment in Cisplatin-Associated Anemia: A Randomized, Double-Blind Trial With Placebo. J Clin Oncol. 1994;12:1058–62.

50 Rose E, Rai K, Revicki D, et al. Clinical and health status assessments in anemic chronic lymphocytic leukemia (CLL) patients treated with epoetin alfa (EPO). Blood. 1994;84(10 Suppl 1):526a.

51 Cazzola M, Messinger D, Battistel V, Bron D, Cimino R, Enller-Ziegler L, et al. Recombinant human erythropoietin in the anemia associated with multiple myeloma or non-Hodgkin’s lymphoma: dose finding and identification of predictors of response. Blood. 1995;86(12):4446–53.

52 Henry DH, Abels RI. Recombinant human erythropoietin in the treatment of cancer and chemotherapy-induced anemia: results of double-blind and open-label follow-up studies. Semin Oncol. 1994;21(2 Suppl 3):21–8.

53 Osterborg A, Boogaerts MA, Cimino R, Essers U, Holowiecki J, Juliusson G, et al. Recombinant human erythropoietin in transfusion-dependent anemic patients with multiple myeloma and non-Hodgkin’s lymphoma – a randomized multicenter study. The European Study Group of Erythropoietin (Epoetin Beta) Treatment in Multiple Myeloma and Non-Hodgkin’s Lymphoma. Blood. 1996;87(7):2675–82.

54 Del Mastro L, Venturini M, Lionetto R, Garrone O, Melioli G, Pasquetti W, et al. Randomized phase III trial evaluating the role of erythropoietin in the prevention of chemotherapy-induced anemia. J Clin Oncol. 1997;15(7):2715–21.

55 Kurz C, Marth C, Windbichler G, Lahousen M, Medl M, Vavra N, et al. Erythropoietin treatment under polychemotherapy in patients with gynecologic malignancies: a prospective, randomized, double-blind placebo-controlled multicenter study. Gynecol Oncol. 1997;65(3):461–6.

56 Oberhoff C, Neri B, Amadori D, Petry KU, Gamucci T, Rebmann U, et al. Recombinant human erythropoietin in the treatment of chemotherapy-induced anemia and prevention of transfusion requirement associated with solid tumors: a randomized, controlled study. Ann Oncol. 1998;9(3):255–60.

57 ten Bokkel Huinink WW, de Swart CA, van Toorn DW, Morack G, Breed WP, Hillen HF, et al. Controlled multicentre study of the influence of subcutaneous recombinant human erythropoietin on anaemia and transfusion dependency in patients with ovarian carcinoma treated with platinum-based chemotherapy. Med Oncol. 1998;15(3):174–82.

58 Throuvalas NA, Antonadou D, Boufi M, Lavey R. Erythropoietin decreases Transfusion Requirements during Radiochemotherapy. Proc Am Soc Clin Oncol. 2000;19:2000.

59 Boogaerts M, Coiffier B, Kainz C. Impact of epoetin beta on quality of life in patients with malignant disease. Br J Cancer. 2003;88(7):988–95.

60 Osterborg A, Brandberg Y, Molostova V, Iosava G, Abdulkadyrov K, Hedenus M, et al. Randomized, double-blind, placebo-controlled trial of recombinant human erythropoietin, epoetin Beta, in hematologic malignancies. J Clin Oncol. 2002;20(10):2486–94.

61 Quirt I, Micucci S, Moran LA, Pater J, Browman G. The role of recombinant human erythropoietin (EPO) in reducing red blood cell transfusions and maintaining quality of life (QOL) in patients with lymphoma and solid tumors receiving cytotoxic chemotherapy. Results of a randomized, double-blind, placebo-controlled clinical trial. Blood. 1996;88(10 (Suppl 1)):347a. Blood. 1996;88[10 (Suppl 1)], 347a.

62 Bamias A, Aravantinos G, Kalofonos C, Timotheadou N, Siafaka V, Vlahou I, et al. Prevention of anemia in patients with solid tumors receiving platinum-based chemotherapy by recombinant human Erythropoietin (rHuEpo): a prospective, open label, randomized trial by the Hellenic Cooperative Oncology Group. Oncology. 2003;64(2):102–10.

63 Huddart RA, Welch RS, Chan S, Perren T, Atkinson R. A prospective randomised comparative-group evaluation of epoetin alfa for the treatment of anaemia in UK cancer patients receiving platinum-based chemotherapy. [Miscellaneous]. Annals of Oncology Abstract Book of the 27th ESMO Congress, Nice, France, 2002;13(Suppl 5):177.

64 INT-1. 2004. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research, Oncologic Drugs Advisory Committee.

65 INT-3. 2004. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research, Oncologic Drugs Advisory Committee.

66 P-174. 2004. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research, Oncologic Drugs Advisory Committee.

67 ODAC 2003. 2004. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research, Oncologic Drugs Advisory Committee.

68 Tonia T, Mettler A, Robert N, Schwarzer G, Seidenfeld J, Weingart O, Hyde C, Engert A, Bohlius J. Erythropoietin or darbepoetin for patients with cancer. Cochrane Database of Systematic Reviews 2012.

69 Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365.

70 Poppe K, Doughty R, Yu C, Quintana M, Moller J, Klein A, et al. Understanding differences in results from literature-based and individual patient meta-analyses: An example from meta-analyses of observational data. Int J Cardiol. 2011;148:209–13.

71 McCormack K, Grant A, Scott N. Value of updating a systematic review in surgery using individual patient data. Br J Surg. 2004;91(4):495–9.

72 DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Archives of Otolaryngology – Head & Neck Surgery. 2005;131(6):479–80.

73 International Committee of Medical Journal Editors (ICMJE), Laine C, Horton R, De AC, Drazen JM, Frizelle FA, Godlee F, et al. Clinical trial registration: looking back and moving ahead. [Danish]. Ugeskrift for Laeger. 2007;169(26):2505–6.

74 Huic M, Marusic M, Marusic A. Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS ONE [Electronic Resource] 2011;6(9):e25258.

75 Golder S, Loke YK, Bland M. Unpublished data can be of value in systematic reviews of adverse effects: methodological overview. [Review]. J Clin Epidemiol. 2010;63(10):1071–81.

76 FDA Amendments Act of 2007. http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=110_cong_public_laws&docid=f:publ085.110. Pdf

77 Wieseler B, McGauran N, Kaiser T. Finding studies on reboxetine: a tale of hide and seek. BMJ. 2010;341:c4942.

78 Godlee F, Loder E. Missing clinical trial data: setting the record straight. BMJ. 2010;341:c5641.

79 Jefferson T, Jones MA, Doshi P, Del Mar CB, Heneghan CJ, Hama R, et al. Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. [Review]. Cochrane Database of Systematic Reviews 2012;1:CD008965.

80 Eichler H, Abadie E, Breckenridge H, Leufkens H, Rasi G. Open clinical trial data for all? A view from regulators. PLoS.Med. 2012;9(4):e1001202.

81 European Medicines Agency widens public access to documents: Policy on access to documents also sets out new approach for proactive disclosure of documents. http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2010/11/news_detail_001158.jsp&mid=WC0b01ac058004d5c1

82 Wieseler B, McGauran N, Kerekes M, Kaiser T. Access to regulatory data from the European Medicines Agency: the times they are a-changing. Systematic Reviews 2012;1:50.

83 Hrynaszkiewicz I, Norton ML, Vickers AJ, Altman DG. Preparing raw clinical data for publication: guidance for journal editors, authors, and peer reviewers. BMJ. 2010;340:c181.

84 Chan AW, Tetzlaff JM, Altman DG, Laupacis A, Gotzsche PC, Krleza-Jeric K, et al. SPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials. Ann Intern Med. 2013 Jan 8.

85 Guideline for industry: structure and content of clinical study reports. http://www.fda.gov/downloads/regulatoryinformation/guidances/ucm129456.pdf. 2013.

86 Humaidan P, Polyzos NP. (Meta)analyze this: Systematic reviews might lose credibility. Nat Med. 2012;18(9):1321.

87 Bohlius J, Langensiepen S, Schwarzer G, Seidenfeld J, Piper M, Bennett C, Engert A. Recombinant human erythropoietin and overall survival in cancer patients: results of a comprehensive meta-analysis. J Natl Cancer Inst. 2005;97(7):489–98.

88 Littlewood TJ, Bajetta E, Nortier JW, Vercammen E, Rapoport B. Effects of epoetin alfa on hematologic parameters and quality of life in cancer patients receiving nonplatinum chemotherapy: results of a randomized, double-blind, placebo-controlled trial. J Clin Oncol. 2001;19(11):2865–74.