Skip to main navigation menu Skip to main content Skip to site footer

Original article

Vol. 152 No. 1112 (2022)

Derivation and validation of a prediction model to establish nursing-sensitive quality benchmarks in medical inpatients: a secondary data analysis of a prospective cohort study

  • Daniel Koch
  • Alexander Kutz
  • Thomas Volken
  • Claudia Gregoriano
  • Antoinette Conca
  • Michael Kleinknecht-Dolf
  • Philipp Schuetz
  • Beat Mueller
DOI
https://doi.org/10.4414/SMW.2022.w30152
Cite this as:
Swiss Med Wkly. 2022;152:w30152
Published
18.03.2022

Summary

BACKGROUND: Hospitals are using nursing-sensitive outcomes (NSOs) based on administrative data to measure and benchmark quality of nursing care in acute care wards. In order to facilitate comparisons between different hospitals and wards with heterogeneous patient populations, proper adjustment procedures are required. In this article, we first identify predictors for common NSOs in acute medical care of adult patients based on administrative data. We then develop and cross-validate an NSO-oriented prediction model.

METHODS: We used administrative data from seven hospitals in Switzerland to derive prediction models for each of the following NSO: hospital-acquired pressure ulcer (≥ stage II), hospital-acquired urinary tract infection, non-ventilator hospital-acquired pneumonia and in-hospital mortality. We used a split dataset approach by performing a random 80:20 split of the data into a training set and a test set. We assessed discrimination of the models by area under the receiver operating characteristic curves. Finally, we used the validated models to establish a benchmark between the participating hospitals.

RESULTS: We considered 36,149 hospitalisations, of which 51.9% were male patients with a median age of 73 years (with an interquartile range of 59–82). Age and length of hospital stay were independently associated with all four NSOs. The derivation and validation models showed a good discrimination in the training (AUC range: 0.75–0.84) and in the test dataset (AUC range: 0.77–0.81), respectively. Variation among different hospitals was relevant considering the risk for hospital-acquired pressure ulcer (≥ stage II) (adjusted Odds ratio [aOR] range: 0.51 [95% CI: 0.38–0.69] – 1.65 [95% CI: 1.33–2.04]), the risk for hospital-acquired urinary tract infection (aOR range: 0.46 [95% CI: 0.36–0.58] – 1.45 [95% CI: 1.31–1.62]), the risk for non-ventilator hospital-acquired pneumonia (aOR range: 0.28 [95% CI: 0.09–0.89] – 2.87 [95% CI: 2.27–3.64]), and the risk for in-hospital mortality (aOR range: 0.45 [95% CI: 0.36–0.56] – 1.39 [95% CI: 1.23–1.60]).

CONCLUSION: The application of risk adjustment when comparing nursing care quality is crucial and enables a more objective assessment across hospitals or wards with heterogeneous patient populations. This approach has potential to establish a set of benchmarks that could allow comparison of outcomes and quality of nursing care between different hospitals and wards.

References

  1. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002 May;346(22):1715–22. https://doi.org/10.1056/NEJMsa012247
  2. Kane RL, Shamliyan TA, Mueller C, Duval S, Wilt TJ. The association of registered nurse staffing levels and patient outcomes: systematic review and meta-analysis. Med Care. 2007 Dec;45(12):1195–204. https://doi.org/10.1097/MLR.0b013e3181468ca3
  3. Needleman J, Buerhaus P, Pankratz VS, Leibson CL, Stevens SR, Harris M. Nurse staffing and inpatient hospital mortality. N Engl J Med. 2011 Mar;364(11):1037–45. https://doi.org/10.1056/NEJMsa1001025
  4. Aiken LH, Sermeus W, Van den Heede K, Sloane DM, Busse R, McKee M, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ. 2012 Mar;344 mar20 2:e1717. https://doi.org/10.1136/bmj.e1717
  5. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002 Oct;288(16):1987–93. https://doi.org/10.1001/jama.288.16.1987
  6. Driscoll A, Grant MJ, Carroll D, Dalton S, Deaton C, Jones I, et al. The effect of nurse-to-patient ratios on nurse-sensitive patient outcomes in acute specialist units: a systematic review and meta-analysis. Eur J Cardiovasc Nurs. 2018 Jan;17(1):6–22. https://doi.org/10.1177/1474515117721561
  7. Oner B, Zengul FD, Oner N, Ivankova NV, Karadag A, Patrician PA. Nursing-sensitive indicators for nursing care: A systematic review (1997-2017). Nurs Open. 2021 May;8(3):1005–22. https://doi.org/10.1002/nop2.654
  8. Doran D. Functional Status. Nursing-sensitive Outcomes: state of the science. Londom: Jones and Bartlett Publishers International. 2003.
  9. Doran DM, Harrison MB, Laschinger HS, Hirdes JP, Rukholm E, Sidani S, et al. Nursing-sensitive outcomes data collection in acute care and long-term-care settings. Nurs Res. 2006 Mar-Apr;55(2 Suppl):S75–81. https://doi.org/10.1097/00006199-200603001-00012
  10. Sim J, Crookes P, Walsh K, Halcomb E. Measuring the outcomes of nursing practice: A Delphi study. J Clin Nurs. 2018 Jan;27(1-2):e368–78. https://doi.org/10.1111/jocn.13971
  11. Sim J, Joyce-McCoach J, Gordon R, Kobel C. Development of a data registry to evaluate the quality and safety of nursing practice. J Adv Nurs. 2019 Sep;75(9):1877–88. https://doi.org/10.1111/jan.13967
  12. Dubois CA, D’amour D, Brault I, Dallaire C, Déry J, Duhoux A, et al. Which priority indicators to use to evaluate nursing care performance? A discussion paper. J Adv Nurs. 2017 Dec;73(12):3154–67. https://doi.org/10.1111/jan.13373
  13. Connolly D, Wright F. The nursing quality indicator framework tool. Int J Health Care Qual Assur. 2017 Aug;30(7):603–16. https://doi.org/10.1108/IJHCQA-08-2016-0113
  14. National Quality Forum. National Voluntary Consensus Standards for Nursing-Sensitive Care: An Initial Performance Measure Set2004.
  15. Hannah K, White PA, Kennedy MA, Hammell N, editors. C-HOBIC–standardized information to support clinical practice and quality patient care across Canada. NI 2012: 11th International Congress on Nursing Informatics, June 23-27, 2012, Montreal, Canada; 2012: American Medical Informatics Association.
  16. Dubois C-A, D’Amour D, Pomey M-P, Girard F, Brault I. Conceptualizing performance of nursing care as a prerequisite for better measurement: a systematic and interpretive review. 2013.
  17. Paulsen RA. Taking nurse staffing research to the unit level. Nurs Manage. 2018 Jul;49(7):42–8. https://doi.org/10.1097/01.NUMA.0000538915.53159.b5
  18. Kutz A, Koch D, Conca A, Baechli C, Haubitz S, Regez K, et al. Integrative hospital treatment in older patients to benchmark and improve outcome and length of stay - the In-HospiTOOL study. BMC Health Serv Res. 2019 Apr;19(1):237. https://doi.org/10.1186/s12913-019-4045-x
  19. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. https://doi.org/10.1016/0021-9681(87)90171-8
  20. Twigg DE, Gelder L, Myers H. The impact of understaffed shifts on nurse-sensitive outcomes. J Adv Nurs. 2015 Jul;71(7):1564–72. https://doi.org/10.1111/jan.12616
  21. Twigg DE, Geelhoed EA, Bremner AP, Duffield CM. The economic benefits of increased levels of nursing care in the hospital setting. J Adv Nurs. 2013 Oct;69(10):2253–61. https://doi.org/10.1111/jan.12109
  22. Maas ML, Johnson M, Moorhead S. Classifying nursing-sensitive patient outcomes. Image J Nurs Sch. 1996;28(4):295–301. https://doi.org/10.1111/j.1547-5069.1996.tb00377.x
  23. Griffiths P, Jones S, Maben J, Murrells T. State of the art metrics for nursing: a rapid appraisal. London: King’s College London; 2008.
  24. Recio-Saucedo A, Dall’Ora C, Maruotti A, Ball J, Briggs J, Meredith P, et al. What impact does nursing care left undone have on patient outcomes? Review of the literature. J Clin Nurs. 2018 Jun;27(11-12):2248–59. https://doi.org/10.1111/jocn.14058
  25. Schubert M, Glass TR, Clarke SP, Aiken LH, Schaffert-Witvliet B, Sloane DM, et al. Rationing of nursing care and its relationship to patient outcomes: the Swiss extension of the International Hospital Outcomes Study. Int J Qual Health Care. 2008 Aug;20(4):227–37. https://doi.org/10.1093/intqhc/mzn017
  26. Schubert M, Clarke SP, Glass TR, Schaffert-Witvliet B, De Geest S. Identifying thresholds for relationships between impacts of rationing of nursing care and nurse- and patient-reported outcomes in Swiss hospitals: a correlational study. Int J Nurs Stud. 2009 Jul;46(7):884–93. https://doi.org/10.1016/j.ijnurstu.2008.10.008
  27. Twigg D, Duffield C, Bremner A, Rapley P, Finn J. The impact of the nursing hours per patient day (NHPPD) staffing method on patient outcomes: a retrospective analysis of patient and staffing data. Int J Nurs Stud. 2011 May;48(5):540–8. https://doi.org/10.1016/j.ijnurstu.2010.07.013
  28. Butler M, Collins R, Drennan J, Halligan P, O’Mathúna DP, Schultz TJ, et al. Hospital nurse staffing models and patient and staff-related outcomes. Cochrane Database Syst Rev. 2011 Jul;(7):CD007019. https://doi.org/10.1002/14651858.CD007019.pub2
  29. Soban LM, Hempel S, Munjas BA, Miles J, Rubenstein LV. Preventing pressure ulcers in hospitals: A systematic review of nurse-focused quality improvement interventions. Jt Comm J Qual Patient Saf. 2011 Jun;37(6):245–52. https://doi.org/10.1016/S1553-7250(11)37032-8
  30. Lankshear AJ, Sheldon TA, Maynard A. Nurse staffing and healthcare outcomes: a systematic review of the international research evidence. ANS Adv Nurs Sci. 2005 Apr-Jun;28(2):163–74. https://doi.org/10.1097/00012272-200504000-00008
  31. Fuchs MA, Sexton DJ, Thornlow DK, Champagne MT. Evaluation of an evidence-based, nurse-driven checklist to prevent hospital-acquired catheter-associated urinary tract infections in intensive care units. J Nurs Care Qual. 2011 Apr-Jun;26(2):101–9. https://doi.org/10.1097/NCQ.0b013e3181fb7847
  32. Bernard MS, Hunter KF, Moore KN. A review of strategies to decrease the duration of indwelling urethral catheters and potentially reduce the incidence of catheter-associated urinary tract infections. Urol Nurs. 2012 Jan-Feb;32(1):29–37. https://doi.org/10.7257/1053-816X.2012.32.1.29
  33. Ausserhofer D, Schubert M, Desmedt M, Blegen MA, De Geest S, Schwendimann R. The association of patient safety climate and nurse-related organizational factors with selected patient outcomes: a cross-sectional survey. Int J Nurs Stud. 2013 Feb;50(2):240–52. https://doi.org/10.1016/j.ijnurstu.2012.04.007
  34. Kovner C, Jones C, Zhan C, Gergen PJ, Basu J. Nurse staffing and postsurgical adverse events: an analysis of administrative data from a sample of U.S. hospitals, 1990-1996. Health Serv Res. 2002 Jun;37(3):611–29. https://doi.org/10.1111/1475-6773.00040
  35. Cho SH, Ketefian S, Barkauskas VH, Smith DG. The effects of nurse staffing on adverse events, morbidity, mortality, and medical costs. Nurs Res. 2003 Mar-Apr;52(2):71–9. https://doi.org/10.1097/00006199-200303000-00003
  36. Carryer JB, Diers D, McCloskey B, Wilson D. Effects of health policy reforms on nursing resources and patient outcomes in New Zealand. Policy Polit Nurs Pract. 2010 Nov;11(4):275–85. https://doi.org/10.1177/1527154410393360
  37. Weissman JS, Rothschild JM, Bendavid E, Sprivulis P, Cook EF, Evans RS, et al. Hospital workload and adverse events. Med Care. 2007 May;45(5):448–55. https://doi.org/10.1097/01.mlr.0000257231.86368.09
  38. Mattke S, Needleman J, Buerhaus P, Stewart M, Zelevinsky K. Evaluating the role of patient sample definitions for quality indicators sensitive to nurse staffing patterns. Med Care. 2004 Feb;42(2 Suppl):II21–33. https://doi.org/10.1097/01.mlr.0000109124.90702.8b
  39. Tourangeau AE, Giovannetti P, Tu JV, Wood M. Nursing-related determinants of 30-day mortality for hospitalized patients. Can J Nurs Res. 2002 Mar;33(4):71–88.
  40. Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P. The impact of hospital nursing characteristics on 30-day mortality. Nurs Res. 2005 Mar-Apr;54(2):74–84. https://doi.org/10.1097/00006199-200503000-00002
  41. Tourangeau AE, Doran DM, McGillis Hall L, O’Brien Pallas L, Pringle D, Tu JV, et al. Impact of hospital nursing care on 30-day mortality for acute medical patients. J Adv Nurs. 2007 Jan;57(1):32–44. https://doi.org/10.1111/j.1365-2648.2006.04084.x
  42. Kelly DM, Kutney-Lee A, McHugh MD, Sloane DM, Aiken LH. Impact of critical care nursing on 30-day mortality of mechanically ventilated older adults. Crit Care Med. 2014 May;42(5):1089–95. https://doi.org/10.1097/CCM.0000000000000127
  43. Federal Statistical Office (FSO). Variablen der Medizinischen Statistik, Spezifikationen gültig ab 1.1.2019. 2018.
  44. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse staffing and patient outcomes in hospitals. Final report. Rockville, MD: Health Resources and Services Administration, US Department of Health and Human Services. 2001.
  45. Stagg V. CHARLSON: Stata module to calculate Charlson index of comorbidity. 2017.
  46. Twigg DE, Pugh JD, Gelder L, Myers H. Foundations of a nursing-sensitive outcome indicator suite for monitoring public patient safety in Western Australia. Collegian. 2016;23(2):167–81. https://doi.org/10.1016/j.colegn.2015.03.007
  47. Smith IL, Nixon J, Brown S, Wilson L, Coleman S. Pressure ulcer and wounds reporting in NHS hospitals in England part 1: audit of monitoring systems. J Tissue Viability. 2016 Feb;25(1):3–15. https://doi.org/10.1016/j.jtv.2015.11.001
  48. Swiss National Association for Quality Development in Hospitals and Clinics. Sturz und Dekubitus Akutsomatik Erwachsene - Nationaler Vergleichsbericht2019.
  49. Ho C, Jiang J, Eastwood CA, Wong H, Weaver B, Quan H. Validation of two case definitions to identify pressure ulcers using hospital administrative data. BMJ Open. 2017 Aug;7(8):e016438. https://doi.org/10.1136/bmjopen-2017-016438
  50. Backman C, Vanderloo SE, Miller TB, Freeman L, Forster AJ. Comparing physical assessment with administrative data for detecting pressure ulcers in a large Canadian academic health sciences centre. BMJ Open. 2016 Oct;6(10):e012490. https://doi.org/10.1136/bmjopen-2016-012490
  51. Tubaishat A, Papanikolaou P, Anthony D, Habiballah L. Pressure Ulcers Prevalence in the Acute Care Setting: A Systematic Review, 2000-2015. Clin Nurs Res. 2018 Jul;27(6):643–59. https://doi.org/10.1177/1054773817705541
  52. Tomova-Simitchieva T, Akdeniz M, Blume-Peytavi U, Lahmann N, Kottner J. [The Epidemiology of Pressure Ulcer in Germany: systematic Review]. Gesundheitswesen. 2019 Jun;81(6):505–12.
  53. Öztürk R, Murt A. Epidemiology of urological infections: a global burden. World J Urol. 2020 Nov;38(11):2669–79. https://doi.org/10.1007/s00345-019-03071-4
  54. Kalánková D, Kirwan M, Bartoníčková D, Cubelo F, Žiaková K, Kurucová R. Missed, rationed or unfinished nursing care: A scoping review of patient outcomes. J Nurs Manag. 2020 Nov;28(8):1783–97. https://doi.org/10.1111/jonm.12978
  55. Jhung MA, Banerjee SN. Administrative coding data and health care-associated infections. Clin Infect Dis. 2009 Sep;49(6):949–55. https://doi.org/10.1086/605086
  56. Quan H, Eastwood C, Cunningham CT, Liu M, Flemons W, De Coster C, et al.; IMECCHI investigators. Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study). BMJ Open. 2013 Oct;3(10):e003716. https://doi.org/10.1136/bmjopen-2013-003716
  57. Padula WV, Makic MB, Mishra MK, Campbell JD, Nair KV, Wald HL, et al. Comparative effectiveness of quality improvement interventions for pressure ulcer prevention in academic medical centers in the United States. Jt Comm J Qual Patient Saf. 2015 Jun;41(6):246–56. https://doi.org/10.1016/S1553-7250(15)41034-7
  58. Berlowitz DR, Brandeis GH, Morris JN, Ash AS, Anderson JJ, Kader B, et al. Deriving a risk-adjustment model for pressure ulcer development using the Minimum Data Set. J Am Geriatr Soc. 2001 Jul;49(7):866–71. https://doi.org/10.1046/j.1532-5415.2001.49175.x
  59. Berlowitz DR, Ash AS, Brandeis GH, Brand HK, Halpern JL, Moskowitz MA. Rating long-term care facilities on pressure ulcer development: importance of case-mix adjustment. Ann Intern Med. 1996 Mar;124(6):557–63. https://doi.org/10.7326/0003-4819-124-6-199603150-00003
  60. Letica-Kriegel AS, Salmasian H, Vawdrey DK, Youngerman BE, Green RA, Furuya EY, et al. Identifying the risk factors for catheter-associated urinary tract infections: a large cross-sectional study of six hospitals. BMJ Open. 2019 Feb;9(2):e022137. https://doi.org/10.1136/bmjopen-2018-022137
  61. Li F, Song M, Xu L, Deng B, Zhu S, Li X. Risk factors for catheter-associated urinary tract infection among hospitalized patients: A systematic review and meta-analysis of observational studies. J Adv Nurs. 2019 Mar;75(3):517–27. https://doi.org/10.1111/jan.13863
  62. Di Pasquale M, Aliberti S, Mantero M, Bianchini S, Blasi F. Non-Intensive Care Unit Acquired Pneumonia: A New Clinical Entity? Int J Mol Sci. 2016 Feb;17(3):287. https://doi.org/10.3390/ijms17030287
  63. Sopena N, Heras E, Casas I, Bechini J, Guasch I, Pedro-Botet ML, et al. Risk factors for hospital-acquired pneumonia outside the intensive care unit: a case-control study. Am J Infect Control. 2014 Jan;42(1):38–42. https://doi.org/10.1016/j.ajic.2013.06.021
  64. Federal Statistical Office (FSO). Variablen der Medizinischen Statistik Spezifikationen gültig ab 1.1.2019 2019 14.01.2021. Available from: https://www.bfs.admin.ch/bfs/de/home/statistiken/gesundheit/erhebungen/ms.assetdetail.7066232.html
  65. Sabrià M, Sopena N. Nosocomial Pneumonia Outside the Intensive Care Unit. Clin Pulm Med. 2011;18(5):215–21. https://doi.org/10.1097/CPM.0b013e31822c5f19