Prospective validation and adaptation of the HOSPITAL score to predict high risk of unplanned readmission of medical patients
PRINCIPLES: The HOSPITAL score is a simple prediction model that accurately identifies patients at high risk of readmission and showed good performance in an international multicentre retrospective study. We aimed to demonstrate prospectively its accuracy to predict 30-day unplanned readmission and death.
METHODS: We prospectively screened all consecutive patients aged ≥50 years admitted to the department of general internal medicine of a large community hospital in Switzerland. We excluded patients who refused to give consent, who died during hospitalisation, or who were transferred to another acute care, rehabilitation or palliative care facility. The primary outcome was the first unplanned readmission or death within 30 days after discharge. Some of the predictors of the original score (discharge from an oncology service and length of stay) were adapted according to the setting for practical reasons, before the start of patient inclusion. We also assessed a simplified version of the score, without the variable “any procedure performed during hospitalisation”. The performance of the score was evaluated according to its overall accuracy (Brier score), its discriminatory power (C-statistic), and its calibration (Hosmer-Lemeshow goodness-of-fit test).
RESULTS: Among the 346 included patients, 40 (11.6%) had a 30-day unplanned readmission or death. The HOSPITAL score showed very good accuracy (Brier score 0.10), good discriminatory power (C-statistic 0.70, 95% confidence interval [CI] 0.62–0.79), and an excellent calibration (p = 0.77). Patients were classified into three risk categories for the primary outcome: low (59%), intermediate (20.8%) and high risk (20.2%). The estimated risks of unplanned readmission/death for each category were 8.2%, 11.3% and 21.6%, respectively. The simplified score showed the same performance, with a Brier score of 0.10, a C-statistic of 0.70 (95% CI 0.61–0.79), and a goodness-of-fit statistic of 0.40.
CONCLUSIONS: The HOSPITAL score prospectively identified patients at high risk of 30-day unplanned readmission or death with good performance in medical patients in Switzerland. Its simplicity and good performance make it an easy-to-use tool to target patients who might most benefit from intensive transitional care interventions.
- Donzé J, Aujesky D, Williams D, Schnipper JL. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013;173(8):632–8.
- Donzé J, Lipsitz S, Bates DW, Schnipper JL. Causes and patterns of readmissions in patients with common comorbidities: retrospective cohort study. BMJ. 2013;347:f7171.
- Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418–28.
- Leppin AL, Gionfriddo MR, Kessler M, Brito JP, Mair FS, Gallacher K, et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern Med. 2014;174(7):1095–107.
- Allaudeen N, Schnipper JL, Orav EJ, Wachter RM, Vidyarthi AR. Inability of providers to predict unplanned readmissions. J Gen Intern Med. 2011;26(7):771–6.
- Cooksley T, Nanayakkara PW, Nickel CH, Subbe CP, Kellett J, Kidney R, et al. Readmissions of medical patients: an external validation of two existing prediction scores. QJM. 2015.
- Donze JD, Williams MV, Robinson EJ, Zimlichman E, Aujesky D, Vasilevskis EE, et al. International Validity of the HOSPITAL Score to Predict 30-Day Potentially Avoidable Hospital Readmissions. JAMA Intern Med. 2016;176(4):496–502.
- McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS. Users’ guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA. 2000;284(1):79–84.
- Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277(6):488–94.
- Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130(6):515–24.
- Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med. 2015;162(1):55–63.
- Thommen D, Weissenberger N, Schuetz P, Mueller B, Reemts C, Holler T, et al. Head-to-head comparison of length of stay, patients’ outcome and satisfaction in Switzerland before and after SwissDRG-Implementation in 2012 in 2012: an observational study in two tertiary university centers. Swiss Med Wkly. 2014;144:w13972.
- Agency for Healthcare R, Quality. Healthcare Cost and Utilization Project (HCUP), National Inpatient Sample (NIS) 2004–2013, Trends in Inpatient Stays. 2013.
- van Walraven C, Dhalla IA, Bell C, Etchells E, Stiell IG, Zarnke K, et al. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551–7.
- Khudyakov P, Gorfine M, Zucker D, Spiegelman D. The impact of covariate measurement error on risk prediction. Stat Med. 2015;34(15):2353–67.
- Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.
- Arkes HR, Dawson NV, Speroff T, Harrell FE, Jr., Alzola C, Phillips R, et al. The covariance decomposition of the probability score and its use in evaluating prognostic estimates. SUPPORT Investigators. Med Decis Making. 1995;15(2):120–31.
- Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.
- Pencina MJ, D’Agostino RB. Evaluating Discrimination of Risk Prediction Models: The C Statistic. JAMA. 2015;314(10):1063–4.
- Halfon P, Eggli Y, van Melle G, Chevalier J, Wasserfallen JB, Burnand B. Measuring potentially avoidable hospital readmissions. J Clin Epidemiol. 2002;55(6):573–87.
- Allaudeen N, Vidyarthi A, Maselli J, Auerbach A. Redefining readmission risk factors for general medicine patients. J Hosp Med. 2011;6(2):54–60.
- Phillips RS, Safran C, Cleary PD, Delbanco TL. Predicting emergency readmissions for patients discharged from the medical service of a teaching hospital. J Gen Intern Med. 1987;2(6):400–5.
- Silverstein MD, Qin H, Mercer SQ, Fong J, Haydar Z. Risk factors for 30-day hospital readmission in patients ≥65 years of age. Proc (Bayl Univ Med Cent). 2008;21(4):363–72.
- Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, et al. Risk Prediction Models for Hospital Readmission: A Systematic Review. JAMA. 2011;306(15):1688–98.
- Aujesky D, Mor MK, Geng M, Stone RA, Fine MJ, Ibrahim SA. Predictors of early hospital readmission after acute pulmonary embolism. Arch Intern Med. 2009;169(3):287–93.
- Desai MM, Stauffer BD, Feringa HH, Schreiner GC. Statistical models and patient predictors of readmission for acute myocardial infarction: a systematic review. Circ Cardiovasc Qual Outcomes. 2009;2(5):500–7.
- Hammill BG, Curtis LH, Fonarow GC, Heidenreich PA, Yancy CW, Peterson ED, et al. Incremental value of clinical data beyond claims data in predicting 30-day outcomes after heart failure hospitalization. Circ Cardiovasc Qual Outcomes. 2011;4(1):60–7.
- Ross JS, Mulvey GK, Stauffer B, Patlolla V, Bernheim SM, Keenan PS, et al. Statistical models and patient predictors of readmission for heart failure: a systematic review. Arch Intern Med. 2008;168(13):1371–86.