Estimation of treatment allocation in a randomised, double-blinded, placebo-controlled trial

DOI: https://doi.org/10.4414/smw.2019.20114

Introduction

Randomised, placebo-controlled, double-blind studies (RCTs) have emerged as a gold standard in evaluating the effects of new drug treatments, but the internal validity of the process of double blinding has become a target of criticism [1]. Several studies raised the question as to how accurately patients were able to guess their assigned treatment [2–7]. The accuracy of patients’ and physicians’ guesses was striking. Two studies evaluating the effects of two antidepressant drugs compared with placebo showed that the majority of patients and physicians were able to distinguish active drug from placebo [2, 3]. In these studies, the more certain the patients were about their guess, the higher was their probability of a correct guess. This is not only a problem for outcome assessment bias – which could be avoided by using hard or objective endpoints – but it is also suspected to be a source of bias concerning treatment effect. A study by Schnoll et al. evaluating smoking cessation drugs showed that a patient’s belief of being assigned to the intervention group positively influenced outcome [5]. Even more striking is the effect of an antidepressant drug in chronic pain management: Pretreatment expectations of pain relief influenced the extent of pain relief in patients assigned to the intervention group. However, this was not observed in the placebo group, suggesting an effect beyond placebo [6].

Nevertheless, these studies were performed in the setting of psychiatric care [2, 3] or management of chronic disease (such as obesity, smoking cessation, chronic pain) [4–6] that have a longer course of disease and follow-up, more distinguishable treatment effects, and subjective outcomes especially influenced by personal behaviour and expectations. Currently, there are no publications available that examined the blinding in RCTs in acute clinical settings with short-term interventions in acute, life-threatening diseases. Therefore, we evaluated treatment estimation accuracy in a RCT investigating the effects of an adjunctive 7-day treatment with prednisone in patients hospitalised with community-acquired pneumonia and whether there were predictors for guessing prednisone rather than placebo.

Methods

Study procedures and definition of successful blinding

A detailed description of the study procedures has been published elsewhere [8]. Briefly, patients with community-acquired pneumonia admitted to tertiary care hospitals in Switzerland were assigned to receive either 50 mg of prednisone or placebo daily for 7 days in a double-blind fashion. Before discharge, patients and their treating physicians (or the assigned study doctors if the treating physician did not answer) were asked to guess whether they had been allocated to the prednisone or the placebo group. The cut-off for guessing correctly by chance was defined as 50%, according to flipping a coin. An alternative possibility was to answer “uncertain”. Answering “uncertain” was equated with making a wrong guess because not being certain about treatment allocation is the ultimate proof that the patient/physician was blinded. Therefore, successful blinding was achieved if ≤50% of the patients made a correct guess, and the rest either made a wrong guess or were uncertain about treatment allocation. The same principle held true for the physicians’ guesses.

Predictors

The following predictors for guessing prednisone were analysed: (a) Time to clinical stability (TTCS, with longer TTCS leading to the thought of placebo treatment); (b) occurrence of hyperglycaemia (defined as either mean blood glucose >10 mmol/l on at least one day during hospitalisation, or one blood glucose measurement >15 mmol/l, or need of new insulin treatment); (c) serious adverse events during hospitalisation that were possibly related to prednisone treatment; (d) the occurrence of nosocomial infections or empyema during hospitalisation; (e) persisting fever (being a predictor for placebo guess); and (f) relapse.

Statistical analysis

Guesses from patients and physicians according to treatment group are presented as summary statistics including 95% confidence intervals (CIs). Criteria for successful blinding were met if the lower 95% CI of the proportion that made a correct guess was below or equal to 50%. For the evaluation of predictors for guessing prednisone a mixed effects logistic regression model was fitted (GLMM). The outcome variable was guessing prednisone as binary variable (placebo = 0, prednisone = 1). Explanatory variables were the predictors mentioned above with participant identification number as random effect.

R software version 3.4.3 was used for the analyses.

Results

Brief summary of original results

In this study, which was conducted from 1 December 2009 to 21 May 2014, 785 patients who were hospitalised with community-acquired pneumonia were randomly assigned to receive either prednisone (n = 392) or placebo (n = 393). The primary endpoint was time to clinical stability (TTCS). TTCS was shorter in the prednisone group (median 3.0 days, interquartile range [IQR] 2.5–3.4) than in the placebo group (4.4 days, 4.0–5.0; hazard ratio 1.33, 95% CI 1.15–1.50; p <0.0001). There was no difference between groups in pneumonia-associated complications. Patients in the prednisone group had a higher incidence of in-hospital hyperglycaemia requiring insulin treatment (76 [19%] vs 43 [11%]; odds ratio [OR] 1.96, 95% CI 1.31–2.93; p<0.01). Both groups had similar rates of adverse events that were compatible with corticosteroid use [9].

Treatment assignment guesses

Data were available from 449 patients from three centres in Switzerland (University Hospital Basel, Kantonsspital Aarau and Bürgerspital Solothurn). In 334 cases, guesses from both the patients and the physicians were available. Physicians’ and patients’ guesses were missing in 70 and 45 cases, respectively.

In the prednisone group, 28.9% of the patients (55/190) correctly guessed prednisone (95% CI 22.6–36.0%), which was below chance. Another 9.5% in this group (18/190) thought they had received placebo. The remaining 61.6% of patients (117/190) were uncertain about treatment allocation. The physicians in the same treatment group (prednisone) guessed correctly in 48.3% of cases (87/180, 95% CI 40.8–55.9%) which was within the expected range of chance. Thirty percent of the physicians’ guesses were wrong (54/180), and 21.7% (39/180) were “uncertain” (fig. 1, table 1).

Figure 1 Treatment allocation guesses by patients and physicians according to actual treatment.

In the placebo group, 13.0% of the patients (25/192) correctly guessed placebo (95% CI 8.8–18.8%), which was below the chance of 50%; 69.8% (134/192) were uncertain and 17.2% (33/192) falsely thought themselves to have been allocated to the prednisone group. Of the physicians’ guesses for the placebo group, 66.3% (118/178) were correct (95% CI 58.8–73.2%); 18.5% (33/178) of the guesses were wrong and 15.2% (27/178) were “uncertain”.

Predictors for treatment guess

The results for the linear regression model are summarised in table 2. The longer the time to clinical stability, the lower was the probability for the patients and their physicians to guess prednisone (OR 0.95, 95% CI 0.91–0.99; p = 0.02). The higher blood sugar values were during hospitalisation the higher was the probability to guess prednisone (OR 3.77, 95% CI 2.39–5.95; p <0.001). All other potential predictors did not show statistical significance: The ORs for guessing prednisone in patients who either had serious adverse events or experienced nosocomial infections and/or empyema were 2.52 (0.89–7.10; p = 0.08) and 1.63 (0.66–4.03; p = 0.29), respectively. In patients with persisting fever the OR was 0.86 (0.54–1.36, p = 0.51). In patients with relapse there was also no difference in guesses (OR 0.67 (0.22–2.04, p = 0.48).

Discussion

Overall, we confirmed that patient blinding was achieved in this study, as successful blinding was defined as having 50% or less correct guesses [10]. Blinding is especially important in clinical trials as the Hawthorne and/or placebo/nocebo effects could lead to an alteration in a patient’s health behaviour and, therefore, in outcome [11, 12]. An interesting finding in this study was that shorter time to clinical stability correlated with guessing prednisone, indicating patients’ and physicians’ expectations in this trial.

The majority of patients was uncertain or made a wrong guess about their treatment allocation, independently of their actual treatment. Even though physicians more often made a correct guess, this was not as pronounced as expected, even though physicians had the possibility to observe the patient at every stage of the disease and had full knowledge of their clinical and laboratory parameters. Moreover, in the prednisone group, only nearly half of the guesses by the treating physicians were correct. It was only in the placebo group that the treating physicians guessed correctly in two thirds of the cases. These results are surprising, as prednisone is known to dysregulate glycaemic control and to have immunosuppressive effects [13, 14]. Also, these findings stand in contrast to the results published by the authors mentioned in the Introduction section, in which 75–80% of patients or physicians correctly guessed their treatment [2–7]. There could be several reasons for this discrepancy. First, those studies were performed in long-term settings. It was shown that the longer a treatment is given, the clearer the treatment allocation becomes [2]. Our trial tested a short-duration treatment of 7 days. Second, patients with chronic diseases have plenty of experience with the course of their disease, compared with patients who have suffered only once or twice from a certain disease, as is the case with community-acquired pneumonia. Third, antidepressant drugs are potent psychopharmaceuticals showing strong beneficial effects but also major side effects. Side effects are an important predictor for guessing accuracy [1, 4]. In contrast, prednisone given over a short period of 7 days does not have many side effects except for transient hyperglycaemia [9].

Interestingly, if patients in the prednisone group were certain about their guess, the probability of guessing prednisone was higher, which was also true for physicians. This is probably as a result of prednisone-induced hyperglycaemia and/or a psychological steroid effect for the patients. Although hyperglycaemia was more prevalent in the prednisone group, it also had a high prevalence in the placebo group [15]. In conformity with these assumptions, we showed that hyperglycaemia is a predictor for guessing prednisone, independently of actual treatment allocation. Although our predictor analysis showed a tendency for physicians and patients to guess prednisone when corticosteroid-related adverse events such as nosocomial infection, delirium or gastrointestinal bleeding occur, these findings were not significant. However, considering the large confidence intervals, the lack of statistical significance could be due to the small number of patients experiencing serious adverse events.

This study has several limitations. First, this was a subanalysis and no sample size calculation was performed. However, the 95% CIs that do not include 50% provide evidence that the sample size was large enough for the primary research question of this article (treatment estimation accuracy). Furthermore, compared with the other trials cited in the introduction, this trial had the second largest sample size of treatment allocation estimates. Another limitation relates to prediction of the physicians’ treatment estimation accuracy. The n = 358 physician guesses did not come from individual physicians. Therefore, it would have been interesting to know whether making a correct guess was associated with the years of experience and other personal factors. However, physician-specific data are lacking.

In conclusion, we showed that sufficient patient and – to a lesser extent – physician blinding was achieved in a study of short-term prednisone vs placebo. These results may be extrapolated to similar interventional studies where administration time of the study drug is short, and drug-related side effects cannot clearly be distinguished from disease-specific symptoms. The predictor analysis showed that the influence of not only positive but also negative expectations about a study drug need to be prevented by blinding, as side effects may arise owing to not only the study drug but also the disease itself. We therefore propose that assessing potential unblinding through correctly guessing treatment allocation (due to effects or side effects of the verum) in the course of a double-blind RCT should be implemented when one of the following conditions are met: (a) investigation of a chronic disease (such as chronic psychiatric or pain conditions);(b) long treatment duration (for example, >7 days);(c) known drug-specific side effects (such as with antidiabetic, cytostatic, or diuretic agents); and (d) no comparison with an active substance with a similar therapeutic effect (placebo control only).

Acknowledgements

We gratefully thank the staff of the emergency departments and medical wards of all participating hospitals in supporting this study. Furthermore, we thank the many supporters, study and laboratory personnel at all participating centres who have made this trial possible, especially Cemile Bathelt, Katharina Regez, Ursula Schild, Sonja Schwenne, Merih Guglielmetti, Kristina Schumacher, Fausta Chiaverio, Renate Hunziker, Ursina Minder and Nicole Salvisberg.

References

1 White K , Kando J , Park T , Waternaux C , Brown WA . Side effects and the “blindability” of clinical drug trials. Am J Psychiatry. 1992;149(12):1730–1. doi:.https://doi.org/10.1176/ajp.149.12.1730  

2 Margraf J , Ehlers A , Roth WT , Clark DB , Sheikh J , Agras WS , et al. How “blind” are double-blind studies? J Consult Clin Psychol. 1991;59(1):184–7. doi:.https://doi.org/10.1037/0022-006X.59.1.184  

3 Rabkin JG , Markowitz JS , Stewart J , McGrath P , Harrison W , Quitkin FM , et al. How blind is blind? Assessment of patient and doctor medication guesses in a placebo-controlled trial of imipramine and phenelzine. Psychiatry Res. 1986;19(1):75–86. doi:.https://doi.org/10.1016/0165-1781(86)90094-6  

4 Moscucci M , Byrne L , Weintraub M , Cox C . Blinding, unblinding, and the placebo effect: an analysis of patients’ guesses of treatment assignment in a double-blind clinical trial. Clin Pharmacol Ther. 1987;41(3):259–65. doi:.https://doi.org/10.1038/clpt.1987.26  

5 Schnoll RA , Epstein L , Audrain J , Niaura R , Hawk L , Shields PG , et al. Can the blind see? Participant guess about treatment arm assignment may influence outcome in a clinical trial of bupropion for smoking cessation. J Subst Abuse Treat. 2008;34(2):234–41. doi:.https://doi.org/10.1016/j.jsat.2007.04.004  

6 Turner JA , Jensen MP , Warms CA , Cardenas DD . Blinding effectiveness and association of pretreatment expectations with pain improvement in a double-blind randomized controlled trial. Pain. 2002;99(1-2):91–9. doi:.https://doi.org/10.1016/S0304-3959(02)00060-X  

7 Carroll KM , Rounsaville BJ , Nich C . Blind man’s bluff: effectiveness and significance of psychotherapy and pharmacotherapy blinding procedures in a clinical trial. J Consult Clin Psychol. 1994;62(2):276–80. doi:.https://doi.org/10.1037/0022-006X.62.2.276  

8 Blum CA , Nigro N , Winzeler B , Suter-Widmer I , Schuetz P , Briel M , et al. Corticosteroid treatment for community-acquired pneumonia--the STEP trial: study protocol for a randomized controlled trial. Trials. 2014;15(1):257. doi:.https://doi.org/10.1186/1745-6215-15-257  

9 Blum CA , Nigro N , Briel M , Schuetz P , Ullmer E , Suter-Widmer I , et al. Adjunct prednisone therapy for patients with community-acquired pneumonia: a multicentre, double-blind, randomised, placebo-controlled trial. Lancet. 2015;385(9977):1511–8. doi:.https://doi.org/10.1016/S0140-6736(14)62447-8  

10 Desbiens NA . In randomized controlled trials, should subjects in both placebo and drug groups be expected to guess that they are taking drug 50% of the time? Med Hypotheses. 2002;59(3):227–32. doi:.https://doi.org/10.1016/S0306-9877(02)00205-0  

11 Benedetti F , Carlino E , Piedimonte A . Increasing uncertainty in CNS clinical trials: the role of placebo, nocebo, and Hawthorne effects. Lancet Neurol. 2016;15(7):736–47. doi:.https://doi.org/10.1016/S1474-4422(16)00066-1  

12 Jakovljevic M . The placebo-nocebo response: controversies and challenges from clinical and research perspective. Eur Neuropsychopharmacol. 2014;24(3):333–41. doi:.https://doi.org/10.1016/j.euroneuro.2013.11.014  

13 Pagano G , Cavallo-Perin P , Cassader M , Bruno A , Ozzello A , Masciola P , et al. An in vivo and in vitro study of the mechanism of prednisone-induced insulin resistance in healthy subjects. J Clin Invest. 1983;72(5):1814–20. doi:.https://doi.org/10.1172/JCI111141  

14 Busillo JM , Cidlowski JA . The five Rs of glucocorticoid action during inflammation: ready, reinforce, repress, resolve, and restore. Trends Endocrinol Metab. 2013;24(3):109–19. doi:.https://doi.org/10.1016/j.tem.2012.11.005  

15 Popovic M , Blum CA , Nigro N , Mueller B , Schuetz P , Christ-Crain M . Benefit of adjunct corticosteroids for community-acquired pneumonia in diabetic patients. Diabetologia. 2016;59(12):2552–60. doi:.https://doi.org/10.1007/s00125-016-4091-4