Funding characteristics of randomised clinical trials supported by the Swiss National Science Foundation : a retrospective cohort study

Amstutz Alaina, Schandelmaier Stefanae, Frei Roya, Surina Jakuba, Agarwal Arnavb, Alturki Reema, von Niederhäusern Belindac, von Elm Erikd, Briel Matthiasae, on behalf of the MAking Randomized Trials Affordable (MARTA) Group a Basel Institute for Clinical Epidemiology and Biostatistics, Department of Clinical Research, University of Basel and University Hospital Basel, Switzerland b Department of Medicine, University of Toronto, Canada c Clinical Trial Unit, Department of Clinical Research, University of Basel and University Hospital Basel, Switzerland d Cochrane Switzerland, Institute of Social and Preventive Medicine (IUMSP), Lausanne University Hospital, Lausanne, Switzerland e Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada


Introduction
Randomised clinical trials (RCTs) are the gold standard for assesing healthcare interventions and a cornerstone of evidence-based medicine [1,2].Over the last decade, new regulations and guidelines for RCTs have been published, including initiatives to better protect research participants, improve trial methodology and harmonise research across countries [3][4][5].These initiatives may have increased research quality, but also increased trial complexity and cost.An estimated USD 99.6 billion was spent on biomedical research by the public sector worldwide in 2012 [6].Funding in biomedical research has more than doubled from USD 37.1 billion in 1994 to USD 94.3 billion in 2003, and it has been estimated that clinical trial research costs are increasing by 7.5% per year above the rate of inflation [4,7,8].Most clinical research is funded by industry (nearly 75% of US trials in 2000) [9].Public-sector funded research has increased over the past 10 years.In 2012, it accounted for 37% of the global expenditure on biomedical research [6].A UK study analysed 122 RCTs funded by two major public healthcare sponsors between 1994 and 2002.Grant sum per participant ranged from GBP 16 (CHF 36) up to GBP 4522 (CHF 10 245) with a median of GBP 641 (CHF 1452) [10].At the same time, a substantial amount of public research money is wasted because results of RCTs (or lessons learned from discontinued RCTs) are not published.Across medical disciplines and countries, the proportion of unpublished trials is high and ranges between 30 and 60% [11].Expressing the resulting waste in monetary terms is difficult because data on funding characteristics of unpublished RCTs are rarely available.In a previous publication, we showed that 26 out of 101 (25%) RCTs supported by the Swiss National Science Foundation (SNSF) were prematurely discontinued owing to poor recruitment of study participants and 40% were not published in a peer-reviewed scientific journal [12].In the present article we complement our findings with the funding characteristics of these 101 SNSF-supported RCTs.We aimed to describe funding characteristics of published and unpublished RCTs supported by the SNSF, to quantify the amount of money spent for unpublished studies, and to compare our results with a similar study performed in the UK.

Ethics approval
None was necessary because this study examined proposals approved by the Swiss National Science Foundation and corresponding publications.Patients or patient data were not involved.

Study design and data source
We established a retrospective cohort of SNSF-supported RCTs for which recruitment and funding had ended in 2015 or earlier.We systematically searched titles and abstracts provided on the SNSF's p 3 research database (p3.snf.ch), which contains all research projects that the SNSF approved and supported fully or partially.We excluded RCTs that were supported through personal grants and that were carried out exclusively outside Switzerland (i.e., SNSF mobility grants), RCTs that were never started, and RCTs for which recruitment or funding was still ongoing at the time of the cut-off date (30 April 2015).The search strategy included synonyms for "randomisation" and "trial" and was limited to human healthcare categories using the database's "advanced search" option.(appendix 1).For each eligible RCT, two methodologically trained investigators independently searched corresponding publications through information available in the SNSF's p 3 research database, electronic databases (Medline, Embase, Cochrane Central Register of Controlled Trials, Google Scholar), and trial registries, using relevant keywords for study design, population, interventions, and outcome if applicable.Furthermore, we screened personal websites of principal investigators and co-investigators, and reference lists of related articles.In addition, we approached all principal investigators of included RCTs using an online questionnaire.We asked investigators for information about funding characteristics (additional funding sources, estimation of overall cost of their trial, estimation of percentage of SNSF support on the whole trial) and completion/ publication status of their trial, including the reason for discontinuation.If a principal investigator did not respond, we sent several reminders by email, or a paper version of the questionnaire by regular mail, and eventually tried to call the investigator by telephone or approach a co-investigator.Sixty-seven of 101 investigators responded to all of the following questions: "Please estimate the total costs of the trial in CHF," "Please estimate the proportion of total costs covered by the SNSF grant(s) in %," and "If not 100% funding by the SNSF, please specify the sources of additional funding."Consequently, the overall minimum response rate was 67%.

Data extraction
After data extractors signed appropriate confidentiality statements, the SNSF granted access to its proposals.In teams of two, data extractors trained in trial methodology extracted data from trial proposals and, if available, from corresponding publications independently and in duplicate.Disagreements were resolved by discussion and consensus.We used a password-protected web-based data extraction tool (www.squiekero.org) that included a detailed manual with instructions for each variable of interest.

Statistical analysis
We describe categorical variables as frequencies and percentages and continuous variables as medians and interquartile ranges (IQRs).We used complete case analysis.The SNSF's p 3 research database includes the approved grant sum per proposal but not per study.Some proposals combined RCTs with other studies such as pilot phases or observational studies.To account for this, we stratified the descriptive analyses by "SNSF grant proposal included a RCT and other projects" vs "SNSF grant proposal included one RCT only".We were specifically interested in the latter proposals for which we knew that the total grant sum was allocated to the RCT.For SNSF grant proposals that included several projects, we were not able to reliably extract the amount of grant sum specifically used for the RCT.
Comparison with similar data from the UK Our systematic review of the literature found only one comparable study that allowed us to extrapolate our results: a UK study that included RCTs supported by the Medical Research Council (MRC) or the Health Technology Assessment (HTA) programme from 1994-2002 [10].
To compare grant sums we (a) included SNSF-supported RCT proposals that comprised one RCT only and (b) included only SNSF-supported RCTs that started between 1994 and 2002.We used an exchange rate dating back to April 2006 (1 GBP = 2.265 CHF) when the UK study was published.

Trial selection and characteristics of included trials
The search of SNSF's p 3 research database yielded 455 proposals, of which a total of 101 were eligible healthcare RCTs, conducted between 1986 and 2015 (fig.1).Twentyfour proposals included other projects in addition to the RCT.The median study size was 138 (interquartile range [IQR] 76-400), and the median planned duration of recruitment was 14 months (IQR 12-21 months).Most RCTs were single-centre, parallel-group trials testing a medication or behavioural intervention in adults.Only 53% were published in a peer-reviewed scientific journal and 30% were prematurely discontinued owing to poor recruitment of study participants.Of the 52 trials approved after the ICMJE recommendation to prospectively register all RCTs (i.e., in 2005 or later) [13], only 40 (77%) were registered in a trial registry (table 1).

Funding characteristics of included trials
Table 2 summarises the funding characteristics of the included RCTs.Investigator-initiated RCTs received a median total of CHF 222 000 (IQR 166 000-276 000) from the SNSF, with a median of CHF 1600 (IQR 600-3000) per planned participant.According to investigators' statements in the survey, the median total cost of RCTs was CHF 428 000 (IQR CHF 282 000-900 000).The SNSF grant sum covered a median of 67% (IQR 40-80%) of the total trial cost.Seventy-four investigators responded to the specific survey question on whether additional funding was obtained.Fifty-two (70%) mentioned additional funding, mainly from their own institution or private foundations.

SNSF grant sum stratified by publication status
In a previous publication we reported on discontinuation and nonpublication of these 101 SNSF-supported RCTs [12].For the analysis reported here, we limited our sample to the SNSF grant proposals that included one RCT only, resulting in a total of 74 eligible RCTs (fig. 1) and stratified the data by publication status.Table 3 summarises char-acteristics of the subsample of 74 RCTs.We identified all peer-reviewed publications through searching electronic databases with indexed publications.The search of trial registries did not yield any additional publications.Table 4 shows that CHF 6.7 million out of CHF 20 477 189 (33%) of SNSF grant sum spent over the last three decades for grant proposals including one RCT only did not result in a peer-reviewed scientific publication.To enhance the comparability of this exploratory analysis, we restricted the sample to the 14 proposals that included one RCT only and were started between 1994 and 2002 (fig.1).The SNSF grant sum per RCT was similar to that of the MRC and HTA programme in the UK, with a median of CHF 1303 per participant and 3 of the 14 RCTs (21%) receiving more than the amount per participant deemed sufficient in the UK (table 5).

Summary of findings
In the last three decades, SNSF-supported RCTs were granted, on average, CHF 222 000, covering 67% of the estimated total trial costs.Most applicants mentioned additional funding from their own institution or private foundations.CHF 6.7 Mio or 33% of the SNSF funding total granted for RCTs did not result in peer-reviewed publications.

Strengths and limitations
This is the first study assessing funding characteristics of publicly funded RCTs in Switzerland.After signing appropriate confidentiality statements, we had access to proposals for all healthcare RCTs supported by the SNSF without restriction by the funder or the applicants [14].
Methodologically trained reviewers systematically identified subsequent publications and extracted data independently and in duplicate.We combined information from several sources (grant proposals, trial registries, publica-tions and investigator survey) in order to increase the accuracy of the data set.
Limitations of our study include data missing owing to the low reporting quality of some older RCT proposals, inability to extract individual RCT grant sums from combined proposals, or lack of response of investigators to our survey.However, our overall response rate of 67% is still sufficient to approximate average total cost per RCT and in line with other similar studies surveying biomedical researchers [15,16].Our comparison with the UK study had not enough power for inferential statistical comparisons because we restricted our sample to the same time period as the UK study.We were not able to compare our results with more recent data from the UK because more updated funding characteristics were not available [17,18].Apart from the multivariable logistic regression analysis in a previous publication [12], we did not further explore the reasons for nonpublication.

Implications and conclusion
The SNSF is mandated by the Swiss Federal Government and thus funded by the public.The SNSF invested CHF 6.7 million (not inflation-adjusted) in RCTs that were never published in peer-reviewed scientific journals.We were not able to quantify the absolute amount of money spent for these unpublished RCTs, including costs such as those associated with ethical review process.We were not able to elucidate how the money was spent in detail.We acknowledge that even if an RCT were not published, its conduct might have had positive and important consequences, such as capacity building within a research group or development of follow-up studies.However, the absence of results from these SNSF-supported RCTs in peer-reviewed scientific journals is worrisome.Nonpublication is not only a waste of precious resources, but also compromises systematic reviews and meta-analyses, and undermines the public's and in particular the patients' trust in clinical research and motivation to participate.We found no evidence that     [10].The proportion of MRC/HTA-supported RCTs failing to recruit 80% of their target decreased in the period between 2002 and 2008, but still accounted for 22% [17].Results from our study were similar, in that one out of four SNSF-supported RCTs were prematurely discontinued because of slow recruitment of study participants [12].Since adequate funding is a crucial factor to support recruitment, one could hypothesise that in both Switzerland and the UK the grant sums provided by the main public funding bodies for RCTs were not sufficient.The SNSF has acknowledged the need to better fund RCTs and started a new funding track in 2015, called the Investigator Initiated Clinical Trial (IICT) programme, which might lead to better publication outcomes.Apart from sufficient funding, realistic projection of the target sample size and adequate planning of the recruitment process seem to be the main drivers for successful trial completion [20].Other reasons for recruitment problems related to the Swiss healthcare system have recently been compiled by Briel et al. in a qualitative study [21].
In conclusion, publicly funded RCTs in Switzerland that were not published subsequently are common and used about a third of the SNSF funding granted for RCTs over the last three decades.Rigorous measures are necessary to ensure successful recruitment and publication.The new SNSF funding track for investigator-initiated RCTs aims to improve the situation but has yet to be evaluated.

Figure 1 :
Figure 1: Trial inclusion and exclusion

Table 1 :
Baseline characteristics of randomised clinical trials supported by the Swiss National Science Foundation.
IQR = interquartile range; RCT = randomised clinical trial; SNSF = Swiss National Science Foundation * We assumed that the data provided in the grant proposal qre associated with the main RCT and not to the other project included in the same proposal (e.g., pilot/feasibility studies, observational studies).†Includesforexample RCTs testing a treatment algorithm or light therapy, etc. ‡ 1 missing datapoint for planned sample size due to poor quality of the study proposal or publication § 13 missing data for planned duration of recruitment due to poor quality of the study proposal or publication ¶ 8 missing data for planned duration of recruitment due to poor quality of the study proposal or publication ** Follow-up ongoing (n =3 in first stratum and n = 1 in second strauma) or manuscript still in preparation up to 5 years after end of study (n = 6 in first stratum and n = 3 in second stratum)Original articleSwissMed Wkly.2018;148:w14587 Swiss Medical Weekly • PDF of the online version • www.smw.chPublished under the copyright license "Attribution -Non-Commercial -No Derivatives 4.0".No commercial reuse without permission.See http://emh.ch/en/services/permissions.html.

Table 2 :
Funding characteristics of randomised clinical trials supported by the Swiss National Science Foundation.

All (n = 101) Characteristic SNSF grant proposal included one RCT only (n = 74) SNSF grant proposal included a RCT and other pro- jects (n = 27)
[12]ortion of unpublished RCTs has changed over the last 20 years[12].Public funding agencies should have a genuine interest in ensuring that results, data and lessons learned from funded RCTs are made available to the scientific community and that ultimately the public will benefit from them.This can be achieved through dissemination of results or sharing of data in data repositories or trial registries and supporting initiatives such as AllTrials (www.alltrials.net).Unfortunately, even after the ICMJE official recommendation to prospectively register all RCTs, only 77% were registered in a trial registry.Public funding agencies should help endorse publication policies by making adherence to standards of publication and data sharing a condition of IQR = interquartile range; RCT = randomised clinical trial; SNSF = Swiss National Science Foundation * 32 missing data due to missing survey answer (n = 69) † 28 missing data due to missing survey answer (n = 73) ‡ 1 missing datapoint for planned sample size § Several answers possible ¶ Swiss Federal Office of Sports, Swiss Tobacco Prevention Funds, etc. ** Swiss Heart Foundation, Swiss Multiple Sclerosis Society, Swiss Society of Pneumology, Swiss Lung League, etc. Swiss Medical Weekly • PDF of the online version • www.smw.chPublishedunder the copyright license "Attribution -Non-Commercial -No Derivatives 4.0".No commercial reuse without permission.See http://emh.ch/en/services/permissions.html.the

Table 3 :
Characteristics of randomised clinical trials supported by the Swiss National Science Foundation by publication status. *

Table 4 :
Swiss National Science Foundation grant sum by publication status.

Table 5 :
[19]arison with UK trial cohort.UK Health Technology Assessment programme; MRC = UK Medical Research Council; RCT = randomised clinical trial; SNSF = Swiss National Science Foundation * Only SNSF grant proposals between 1994-2002 that included one RCT only † More than GBP 1000 or CHF 2270 per planned participanttransfer of funds to investigators on (for example, adherence to a specific deadline after trial completion, such as 18 months, as suggested by the Institute of Medicine[19]).The average grant sums per RCT spent by SNSF and MRC/HTA programme between 1994 to 2002 were similar.McDonald et al. showed that 45% of the MRC/HTAsupported RCTs failed to recruit 80% of the target sample size and hence were at risk for premature discontinuation