Derivation and validation of a prediction model to establish nursing-sensitive quality benchmarks in medical inpatients: a secondary data analysis of a prospective cohort study

DOI: https://doi.org/10.4414/SMW.2022.w30152

Background

The relationship between higher levels of qualified nursing staff (registered nurses vs. non-registered nurses) and patient outcomes has been established.The relationship between higher levels of qualified nursing staff (registered nurses vs. non-registered nurses) and patient outcomes has been established [1–5]. Higher nurse staffing is, for example, associated with a lower incidence of hospital-acquired pressure ulcers, hospital-acquired pneumonia, and in-hospital mortality in medical inpatients [6, 7]. These outcomes, which are influenced by nursing care, are generally understood as nursing-sensitive outcomes (NSO). NSOs have been defined as outcomes “that are relevant, based on nurses’ scope and domain of practice, and for which there is empirical evidence linking nursing inputs and interventions to the outcomes’’ [8, 9].

NSOs may help to establish an outcome-related measure of quality of care [10–13]. In some countries, there are mandatory sets of NSOs already established in clinical routines that help to measure and compare the quality of nursing care [11, 12, 14–16]. However, there is no international consensus on NSO sets for quality of care assessment. Definitions and measurement methods vary considerably within international healthcare systems, and thus do not allow comparison of NSOs. In general, administrative coding data is used to measure NOSs because of its availability and standardized definition within a healthcare system. Yet not all countries have standardized reporting of NSOs. In Switzerland, for example, only the ‘Swiss National Association for Quality Development in Hospitals and Clinics’ (ANQ) reports annually on aspects of quality of acute care by means of a cross-sectional survey. The ANQ methodology has important weaknesses. First, the yearly data acquisition for ANQ analyses requires large staff resources. Second, data are collected on one day per year for point prevalence, which results in low incidence rates. This does not allow trends to be assessed in order to analyse the power of NSO.

Here, the frequency of NSOs from uniformly defined administrative data could be a useful and cost-effective complement. In addition to the availability of data and a clear definition, a suitable adjustment procedure is needed to facilitate comparisons between different hospitals or wards with heterogeneous patient populations.

The primary aim of this study was to identify predictors for four common NSOs of acute medical care of adult patients based on administrative data for the development and cross-validation of an NSO-related prediction model. The secondary aim of this study was to establish a set of benchmarksbetween seven Swiss hospitals using the four pre-specified NSOs. There is broad consensus that analyses and comparisons of NSO should occur at the ward level so that specific quality improvement actions can be taken [17]. Therefore, in addition to the overall hospital view presented here, which provides an impression of the overall comparison between medical departments, the proposed procedure will also be applicable at the ward level. This may allow comparisons within a department.

Methods

Design

This was a secondary data analysis of a prospective cohort study (In-HospiTOOL study) [18]. The “In-HospiTOOL” study was a quasi-experimental investigator-initiated, multicenter comparative effectiveness trial investigating the impact of an interprofessional discharge planning tool on length of hospital stay and other outcomes. The study established a representative benchmarking database to promote the measurement of quality of care across different sized Swiss hospitals.

Study population and setting

We included all consecutively admitted adult (≥18 years) emergency patients from July 2017 to January 2019. Patients had to be hospitalized on a medical ward in one of the following seven secondary and tertiary care hospitals: Cantonal Hospital Aarau, Cantonal Hospital Baden, Cantonal Hospital Muensterlingen, Hospital Muri, Hospital Zofingen, Hospital Interlaken, and University Hospital Basel. We excluded patients with a length of hospital stay shorter than 24 hours or longer than 90 days and patients who have been treated in the intensive care unit (ICU) as part of their hospitalisation because the NSOs under study have not been developed for use in these patient populations.

Covariates of interest

We conducted a literature review to identify covariates that may affect the occurrence of an NSO. Based on previous studies, we used a basic set of adjustment variables for all models: age, gender, Charlson Comorbidity Index [19], length of hospital stay, Major Diagnostic Category (MDC) according to Diagnosis Related Groups (DRG), and type of hospital [20, 21]. Further covariates related to individual NSOs are described in table 1. Data availability was a limiting factor in the selection of covariates.

Outcome of interest

An NSO was defined as "a variable patient or family caregiver state, behavior, or perception responsive to nursing intervention… [that] can be measured and compared to a baseline over time" [22]. They are deemed scientifically acceptable if there is sufficient evidence of the link between process measures and patient outcomes and if they are attributable to nursing [23]. Therefore, we have included NSOs for which the relationship between patient outcomes and nursing staff variables was found to be statistically significant in the literature and for which a data basis is available in the administrative data. Based on these criteria we analysed four NSOs:

• hospital-acquired pressure ulcer (≥ stage II) [12, 20, 24–29]; 

• hospital-acquired urinary tract infection [1, 20, 27, 30–33]; 

• non-ventilator hospital-acquired pneumonia [1, 2, 20, 27, 34–38]; 

• in-hospital mortality [3, 5, 27, 39–42] (table 1).

Data collection

We used administrative data provided by the coding department as well as data from the electronic patient record of each of the participating hospitals between July 1, 2017, and January 31, 2019, as part of the In-HospiTOOL study. The datasets were linked at the case number level and data consistency between the different datasets was checked to ensure data quality. The administrative data comprises a set of uniform, clearly defined variables created by the Swiss Federal Statistical Office, that are therefore comparable among hospitals [43]. The diagnosis coding takes place after the hospitalisation has been completed, based on the discharge reports and the electronic patient record. To enable the coding of an outcome, the prerequisite had to be fulfilled that the outcome had been correctly recorded by a nurse or physician in the electronic patient chart or diagnosis list as part of the routine processes during hospitalisation. A single patient may have more than one hospital admission within the study period. Information on status of readmission to the same hospital according to the definition of Swiss-DRG, i.e. within 18 days after hospital discharge, was available for each hospitalisation. Length of hospital stay was calculated based on Swiss-DRG definition by day of admission and each subsequent day without the day of discharge.

Statistical analysis

We stratified sociodemographic characteristics and covariates by the four NSOs. Discrete variables were expressed as frequency (percentage) and continuous variables as medians and interquartile ranges (IQR). We used International Classification of Diseases (ICD) 10 codes to create variables to indicate whether patients experienced a NSO during their hospital stay using algorithms previously developed by Needleman, Buerhaus [44] and used in similar research projects [20, 27]. For example, a hospital-acquired pressure ulcer (≥ stage II) was identified for any patient who had a secondary diagnosis code of L89 (inclusion criteria – see table 1) unless they had a length of stay <4 days, a major diagnostic category of 9 or a diagnosis code between G80-G83. These exclusion criteria are described in figure 1 (second level). The Charlson Comorbidity Index was calculated using the Stata command "charlson" [45].

To assess associations between predictors, covariates and NSOs, we performed logistic regression models. The area under the receiver operating characteristic curve (AUC) was used as a measure of discrimination. To ensure higher generalisability of the results and to avoid overfitting, we used a split dataset approach by performing a random 80:20 split of the data into a training set and a test set, respectively, while maintaining the proportion of outcomes within each of the two samples. The model fit by decile (estimated and observed probabilities) was plotted for each model (see figures A-1–A-4 in the appendix). We used likelihood-ratio tests (LR) to compare models with all predefined covariates with restricted models. For the benchmark comparison, data from a single hospital were compared with those of the remaining six hospitals. For this purpose, we used logistic regression models and reported the crude and adjusted odds ratios as measures of association. We considered a two-tailed P-value at a 5% alpha level for statistical significance. Statistical analyses were performed using Stata 15.1 (StataCorp, College Station, TX, USA). All results are presented in an anonymous form to avoid identification of an individual hospital.

Ethics approval and consent to participate

All patients were informed by a flyer about their study participation after admission. As a quality improvement and control study, the institutional review board (IRB) of Northwestern Switzerland approved the study and waived the need for individual informed consent by formulating a declaration of no objection (AG/SO 2009/074 and EKNZ BASEC PB_2017–00449).

Results

Study population

Of 45,146 hospitalisations, we excluded 8,997 with a length of hospital stay <24 h or >90 days, age <18 years, ICU stay or due to missing ICD-10 diagnosis resulting in 36,149 hospitalisations for the final analysis (see figure 1). There were no other missing data besides the ones mentioned. The median age of the overall population was 73 years (IQR 59–82); 51.9% were male and 80.3% were Swiss residents. The most common reasons for hospital admission regarding MDC were diseases of the circulatory system (n = 8873, 24.5%) and diseases of the respiratory system (n = 5008, 13.9%). Within this sample, 436 patients experienced a hospital-acquired pressure ulcer (≥ stage 2), 2,412 experienced a hospital-acquired urinary tract infection, 339 had a non-ventilator hospital-acquired pneumonia, and 1,525 died in the hospital. Further baseline characteristics are shown in table 2.

Figure 1 Flow chart of study patients.

Prediction model derivation and validation

Most predefined covariates showed a significant association with the corresponding NSO (Table 3). In-hospital death (OR 3.75 [95% CI: 2.80–5.00]), peripheral vascular disease present (OR 1.59 [95% CI: 1.22–2.07]), and present pressure ulcer stage I (OR 7.00 [95% CI: 4.90–9.90]) were strongest associated with the NSO hospital-acquired pressure ulcer (≥ stage II). Gender, MDC and type of hospital did not show any significant association with hospital-acquired pressure ulcer (≥ stage II) and have been removed from the model, based on an LR test (p = 0.17). The development of a hospital-acquired urinary tract infection was most strongly associated with gender (OR for male gender 0.41 [95% CI: 0.37–0.45]) and paraplegia presence (OR 3.10 [95% CI: 2.08–4.62]). Change in Charlson Comorbidity Index, and type of hospital did not show any significant association with hospital-acquired urinary tract infections and have been removed from the model, based on an LR test (p = 0.09). With regards to the NSO  of hospital-acquired pneumonia (for non-ventilator patients), we found the strongest association with gender (OR for male gender 1.53 [95% CI: 1.22–1.92]), the living situation before hospital admission (OR 0.43 [95% CI: 0.34–0.55]) and rehospitalisation (OR 2.34 [95% CI: 1.44–3.82]). MDC did not show any significant association with the NSO and has been removed from the model, based on an LR test (p = 0.11). Concerning the NSO of in-hospital mortality, we found the strongest association with the MDC "myeloproliferative Disease & Disorders" (OR 3.90 [95% CI: 3.26–4.67]) and the living situation before hospital admission (OR 0.58 [95% CI: 0.51–0.67]). Gender did not show any significant association with the NSO and has been removed from the model, based on an LR test (p = 0.05).

All final models showed a good AUC in both the training set (AUC range: 0.75–0.84) and the test set (AUC range: 0.77–0.81), respectively. We did not observe a decrease in discrimination between the training and test sets (p >0.05) (table 3).

Benchmarking between different hospitals

While the prevalence of hospital-acquired pressure ulcer (≥ stage II) was lower in hospital A and higher in hospital F, there were no differences among the remaining hospitals. Hospitals C and D showed a higher prevalence of hospital-acquired urinary tract infection, while hospitals B, F, and G showed a lower prevalence and two hospitals (A and E) showed no difference compared to the other hospitals included. When comparing with the six remaining hospitals, in-hospital mortality was lower in hospitals B, E, F and G and higher in hospitals A and C, respectively. Prevalence of hospital-acquired pneumonia was lower in hospitals A, E, F and G, but higher in hospital C (figure 2).

Figure 2 Crude and adjusted association of the nursing-sensitive outcomes between the participating hospitals.

Marked green: significantly lower odds compared to all other hospitals 

Marked red: significantly higher odds compared to all other hospitals

Discussion

The key findings of this study are two-fold: first, the derived and validated prediction models showed a good discrimination ability for four well-studied NSOs (hospital-acquired pressure ulcer (≥ stage II), hospital-acquired urinary tract infection, non-ventilator hospital-acquired pneumonia, and in-hospital mortality). Second, we found relevant variation in risk of achieving an adverse NSO, suggesting that the outcome-related quality of nursing care differs among the investigated hospitals.

The indicators presented in this study may help to compare quality of care between NSOs of different hospitals using uniformly defined administrative data. This approach allows a timely evaluation of the results without additional effort for data generation. Internationally, mainly administrative data is used in the development of NSO-sets. A recent example is the nursing-sensitive outcome indicator suite for monitoring public patient safety in Western Australia, which was shown to be methodologically robust [46]. These results are largely consistent with the C-statistics in our study, confirming the external validity of our results.

Regarding hospital-acquired pressure ulcer (≥ stage II), we found little variance between the hospitals. On the one hand, the reason may be missing data due to underreporting [47]. Although, cross-sectional surveys by the ANQ showed similar but even lower prevalence rates (1.8%) compared to our study data (1.92%). It has been reported that administrative data do not provide valid data for this NSO due to several reasons [48–50]. This fact seems to be confirmed in our data, as we would expect prevalence rates of five percent or higher [51, 52]. Therefore, before using administrative data to calculate the frequency of this NSO, we recommend reviewing the guideline-compliant documentation of the NSO in the electronic documentations as well as the coding procedure. On the other hand, it is important that the selected outcomes also have a clinically relevant frequency and variability. Otherwise, they offer no benefit in terms of quality development. Regarding hospital-acquired urinary tract infection the measured prevalence in our cohort (6.78%) is consistent with the expected prevalence (between 5.1 and 9.4% [53]), so we do not expect relevant under- or over-recording in the administrative data used.

With regards to the application of the developed method as a benchmark comparison by means of real-world data, two hospitals (B and E) showed a tendency towards lower risks in all NSOs, with some being statistically significant. Other hospitals, however, had higher risks of all (C) or almost all (D) NSOs, again with some reaching statistical significance. While our results should not be used for judging on a hospital’s quality per se, it may provide an overview nevertheless which is notably more comprehensive than reporting individual outcomes based on cross-sectional surveys. For future studies, evaluations at the ward level are needed so that it can be investigated whether the prevalence and variation of the outcomes show relevant differences between the wards. Results on this level may support decision makers to reevaluate their pathways in providing care and thus to improve quality of nursing care. Nursing-related reasons for the differing frequency of negative outcomes could be, for example, an inadequate skill and grade mix, staffing ratios, or insufficiently planned or standardisednursing processes [54].

The strength of this study is the large sample size. While this study covers hospitals from different regions in Switzerland, results might be generalisedat a larger level. However, the results of this study must be interpreted in the context of the study design. First, the use of administrative data is prone to information bias as hospitalisations will be selected according to the ICD-10 codes with the risk of misclassification and underreporting of diagnoses. Thus, frequency of NSO is usually underestimated, especially due to its low financial relevance [55, 56]. Second, we did not have severity estimates of the hospitalised cases. Third, the non-experimental, observational design of our study limits the ability to draw a firm causal link. Fourth, since we do not have information about clinical parameters, we will be unable to account for unmeasured residual confounding and we were limited in selecting all appropriate covariates for the models. Fifth, we did not have data on nurse staffing in the study period. Sixth, external validation of our models is needed in follow-up studies.

Conclusion

The application of risk adjustment when comparing quality of nursing care enables a more objective assessment across hospitals or wards with heterogeneous patient populations. This approach has potential to establish a set of benchmarks allowing comparison of quality of nursing care between different hospitals or wards with manageable effort.

Data sharing statement

The data that support the findings of this study are available from the corresponding author, [DK], upon reasonable request.

Acknowledgements

Authors' contributions: PS and BM were the initiators and scientific project leaders. DK, AK contributed to the data collection. DK, AK, TV, CG, AC, MK, PS and BM participated in the conception and design of the study, contributed methodical expertise, and performed statistical analysis. DK wrote the manuscript, which was revised and approved by all authors.

References

1. Needleman J , Buerhaus P , Mattke S , Stewart M , Zelevinsky K . Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002 May;346(22):1715–22. https://doi.org/10.1056/NEJMsa012247  

2. Kane RL , Shamliyan TA , Mueller C , Duval S , Wilt TJ . The association of registered nurse staffing levels and patient outcomes: systematic review and meta-analysis. Med Care. 2007 Dec;45(12):1195–204. https://doi.org/10.1097/MLR.0b013e3181468ca3  

3. Needleman J , Buerhaus P , Pankratz VS , Leibson CL , Stevens SR , Harris M . Nurse staffing and inpatient hospital mortality. N Engl J Med. 2011 Mar;364(11):1037–45. https://doi.org/10.1056/NEJMsa1001025  

4. Aiken LH , Sermeus W , Van den Heede K , Sloane DM , Busse R , McKee M , et al.  Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ. 2012 Mar;344 mar20 2:e1717. https://doi.org/10.1136/bmj.e1717  

5. Aiken LH , Clarke SP , Sloane DM , Sochalski J , Silber JH . Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002 Oct;288(16):1987–93. https://doi.org/10.1001/jama.288.16.1987  

6. Driscoll A , Grant MJ , Carroll D , Dalton S , Deaton C , Jones I , et al.  The effect of nurse-to-patient ratios on nurse-sensitive patient outcomes in acute specialist units: a systematic review and meta-analysis. Eur J Cardiovasc Nurs. 2018 Jan;17(1):6–22. https://doi.org/10.1177/1474515117721561  

7. Oner B , Zengul FD , Oner N , Ivankova NV , Karadag A , Patrician PA . Nursing-sensitive indicators for nursing care: A systematic review (1997-2017). Nurs Open. 2021 May;8(3):1005–22. https://doi.org/10.1002/nop2.654  

8. Doran D. Functional Status. Nursing-sensitive Outcomes: state of the science. Londom: Jones and Bartlett Publishers International. 2003. 

9. Doran DM , Harrison MB , Laschinger HS , Hirdes JP , Rukholm E , Sidani S , et al.  Nursing-sensitive outcomes data collection in acute care and long-term-care settings. Nurs Res. 2006 Mar-Apr;55(2 Suppl):S75–81. https://doi.org/10.1097/00006199-200603001-00012  

10. Sim J , Crookes P , Walsh K , Halcomb E . Measuring the outcomes of nursing practice: A Delphi study. J Clin Nurs. 2018 Jan;27(1-2):e368–78. https://doi.org/10.1111/jocn.13971  

11. Sim J , Joyce-McCoach J , Gordon R , Kobel C . Development of a data registry to evaluate the quality and safety of nursing practice. J Adv Nurs. 2019 Sep;75(9):1877–88. https://doi.org/10.1111/jan.13967  

12. Dubois CA , D’amour D , Brault I , Dallaire C , Déry J , Duhoux A , et al.  Which priority indicators to use to evaluate nursing care performance? A discussion paper. J Adv Nurs. 2017 Dec;73(12):3154–67. https://doi.org/10.1111/jan.13373  

13. Connolly D , Wright F . The nursing quality indicator framework tool. Int J Health Care Qual Assur. 2017 Aug;30(7):603–16. https://doi.org/10.1108/IJHCQA-08-2016-0113  

14. National Quality Forum . National Voluntary Consensus Standards for Nursing-Sensitive Care: An Initial Performance Measure Set2004. 

15. Hannah K , White PA , Kennedy MA , Hammell N , editors . C-HOBIC–standardized information to support clinical practice and quality patient care across Canada. NI 2012: 11th International Congress on Nursing Informatics, June 23-27, 2012, Montreal, Canada; 2012: American Medical Informatics Association. 

16. Dubois C-A , D’Amour D , Pomey M-P , Girard F , Brault I . Conceptualizing performance of nursing care as a prerequisite for better measurement: a systematic and interpretive review. 2013. 

17. Paulsen RA . Taking nurse staffing research to the unit level. Nurs Manage. 2018 Jul;49(7):42–8. https://doi.org/10.1097/01.NUMA.0000538915.53159.b5  

18. Kutz A , Koch D , Conca A , Baechli C , Haubitz S , Regez K , et al.  Integrative hospital treatment in older patients to benchmark and improve outcome and length of stay - the In-HospiTOOL study. BMC Health Serv Res. 2019 Apr;19(1):237. https://doi.org/10.1186/s12913-019-4045-x  

19. Charlson ME , Pompei P , Ales KL , MacKenzie CR . A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40(5):373–83. https://doi.org/10.1016/0021-9681(87)90171-8  

20. Twigg DE , Gelder L , Myers H . The impact of understaffed shifts on nurse-sensitive outcomes. J Adv Nurs. 2015 Jul;71(7):1564–72. https://doi.org/10.1111/jan.12616  

21. Twigg DE , Geelhoed EA , Bremner AP , Duffield CM . The economic benefits of increased levels of nursing care in the hospital setting. J Adv Nurs. 2013 Oct;69(10):2253–61. https://doi.org/10.1111/jan.12109  

22. Maas ML , Johnson M , Moorhead S . Classifying nursing-sensitive patient outcomes. Image J Nurs Sch. 1996;28(4):295–301. https://doi.org/10.1111/j.1547-5069.1996.tb00377.x  

23. Griffiths P , Jones S , Maben J , Murrells T . State of the art metrics for nursing: a rapid appraisal. London: King’s College London; 2008. 

24. Recio-Saucedo A , Dall’Ora C , Maruotti A , Ball J , Briggs J , Meredith P , et al.  What impact does nursing care left undone have on patient outcomes? Review of the literature. J Clin Nurs. 2018 Jun;27(11-12):2248–59. https://doi.org/10.1111/jocn.14058  

25. Schubert M , Glass TR , Clarke SP , Aiken LH , Schaffert-Witvliet B , Sloane DM , et al.  Rationing of nursing care and its relationship to patient outcomes: the Swiss extension of the International Hospital Outcomes Study. Int J Qual Health Care. 2008 Aug;20(4):227–37. https://doi.org/10.1093/intqhc/mzn017  

26. Schubert M , Clarke SP , Glass TR , Schaffert-Witvliet B , De Geest S . Identifying thresholds for relationships between impacts of rationing of nursing care and nurse- and patient-reported outcomes in Swiss hospitals: a correlational study. Int J Nurs Stud. 2009 Jul;46(7):884–93. https://doi.org/10.1016/j.ijnurstu.2008.10.008  

27. Twigg D , Duffield C , Bremner A , Rapley P , Finn J . The impact of the nursing hours per patient day (NHPPD) staffing method on patient outcomes: a retrospective analysis of patient and staffing data. Int J Nurs Stud. 2011 May;48(5):540–8. https://doi.org/10.1016/j.ijnurstu.2010.07.013  

28. Butler M , Collins R , Drennan J , Halligan P , O’Mathúna DP , Schultz TJ , et al.  Hospital nurse staffing models and patient and staff-related outcomes. Cochrane Database Syst Rev. 2011 Jul;(7):CD007019. https://doi.org/10.1002/14651858.CD007019.pub2  

29. Soban LM , Hempel S , Munjas BA , Miles J , Rubenstein LV . Preventing pressure ulcers in hospitals: A systematic review of nurse-focused quality improvement interventions. Jt Comm J Qual Patient Saf. 2011 Jun;37(6):245–52. https://doi.org/10.1016/S1553-7250(11)37032-8  

30. Lankshear AJ , Sheldon TA , Maynard A . Nurse staffing and healthcare outcomes: a systematic review of the international research evidence. ANS Adv Nurs Sci. 2005 Apr-Jun;28(2):163–74. https://doi.org/10.1097/00012272-200504000-00008  

31. Fuchs MA , Sexton DJ , Thornlow DK , Champagne MT . Evaluation of an evidence-based, nurse-driven checklist to prevent hospital-acquired catheter-associated urinary tract infections in intensive care units. J Nurs Care Qual. 2011 Apr-Jun;26(2):101–9. https://doi.org/10.1097/NCQ.0b013e3181fb7847  

32. Bernard MS , Hunter KF , Moore KN . A review of strategies to decrease the duration of indwelling urethral catheters and potentially reduce the incidence of catheter-associated urinary tract infections. Urol Nurs. 2012 Jan-Feb;32(1):29–37. https://doi.org/10.7257/1053-816X.2012.32.1.29  

33. Ausserhofer D , Schubert M , Desmedt M , Blegen MA , De Geest S , Schwendimann R . The association of patient safety climate and nurse-related organizational factors with selected patient outcomes: a cross-sectional survey. Int J Nurs Stud. 2013 Feb;50(2):240–52. https://doi.org/10.1016/j.ijnurstu.2012.04.007  

34. Kovner C , Jones C , Zhan C , Gergen PJ , Basu J . Nurse staffing and postsurgical adverse events: an analysis of administrative data from a sample of U.S. hospitals, 1990-1996. Health Serv Res. 2002 Jun;37(3):611–29. https://doi.org/10.1111/1475-6773.00040  

35. Cho SH , Ketefian S , Barkauskas VH , Smith DG . The effects of nurse staffing on adverse events, morbidity, mortality, and medical costs. Nurs Res. 2003 Mar-Apr;52(2):71–9. https://doi.org/10.1097/00006199-200303000-00003  

36. Carryer JB , Diers D , McCloskey B , Wilson D . Effects of health policy reforms on nursing resources and patient outcomes in New Zealand. Policy Polit Nurs Pract. 2010 Nov;11(4):275–85. https://doi.org/10.1177/1527154410393360  

37. Weissman JS , Rothschild JM , Bendavid E , Sprivulis P , Cook EF , Evans RS , et al.  Hospital workload and adverse events. Med Care. 2007 May;45(5):448–55. https://doi.org/10.1097/01.mlr.0000257231.86368.09  

38. Mattke S , Needleman J , Buerhaus P , Stewart M , Zelevinsky K . Evaluating the role of patient sample definitions for quality indicators sensitive to nurse staffing patterns. Med Care. 2004 Feb;42(2 Suppl):II21–33. https://doi.org/10.1097/01.mlr.0000109124.90702.8b  

39. Tourangeau AE , Giovannetti P , Tu JV , Wood M . Nursing-related determinants of 30-day mortality for hospitalized patients. Can J Nurs Res. 2002 Mar;33(4):71–88.  

40. Estabrooks CA , Midodzi WK , Cummings GG , Ricker KL , Giovannetti P . The impact of hospital nursing characteristics on 30-day mortality. Nurs Res. 2005 Mar-Apr;54(2):74–84. https://doi.org/10.1097/00006199-200503000-00002  

41. Tourangeau AE , Doran DM , McGillis Hall L , O’Brien Pallas L , Pringle D , Tu JV , et al.  Impact of hospital nursing care on 30-day mortality for acute medical patients. J Adv Nurs. 2007 Jan;57(1):32–44. https://doi.org/10.1111/j.1365-2648.2006.04084.x  

42. Kelly DM , Kutney-Lee A , McHugh MD , Sloane DM , Aiken LH . Impact of critical care nursing on 30-day mortality of mechanically ventilated older adults. Crit Care Med. 2014 May;42(5):1089–95. https://doi.org/10.1097/CCM.0000000000000127  

43. Federal Statistical Office (FSO) . Variablen der Medizinischen Statistik, Spezifikationen gültig ab 1.1.2019. 2018. 

44. Needleman J , Buerhaus P , Mattke S , Stewart M , Zelevinsky K . Nurse staffing and patient outcomes in hospitals. Final report. Rockville, MD: Health Resources and Services Administration, US Department of Health and Human Services. 2001. 

45. Stagg V. CHARLSON: Stata module to calculate Charlson index of comorbidity. 2017. 

46. Twigg DE , Pugh JD , Gelder L , Myers H . Foundations of a nursing-sensitive outcome indicator suite for monitoring public patient safety in Western Australia. Collegian. 2016;23(2):167–81. https://doi.org/10.1016/j.colegn.2015.03.007  

47. Smith IL , Nixon J , Brown S , Wilson L , Coleman S . Pressure ulcer and wounds reporting in NHS hospitals in England part 1: audit of monitoring systems. J Tissue Viability. 2016 Feb;25(1):3–15. https://doi.org/10.1016/j.jtv.2015.11.001  

48. Swiss National Association for Quality Development in Hospitals and Clinics . Sturz und Dekubitus Akutsomatik Erwachsene - Nationaler Vergleichsbericht2019. 

49. Ho C , Jiang J , Eastwood CA , Wong H , Weaver B , Quan H . Validation of two case definitions to identify pressure ulcers using hospital administrative data. BMJ Open. 2017 Aug;7(8):e016438. https://doi.org/10.1136/bmjopen-2017-016438  

50. Backman C , Vanderloo SE , Miller TB , Freeman L , Forster AJ . Comparing physical assessment with administrative data for detecting pressure ulcers in a large Canadian academic health sciences centre. BMJ Open. 2016 Oct;6(10):e012490. https://doi.org/10.1136/bmjopen-2016-012490  

51. Tubaishat A , Papanikolaou P , Anthony D , Habiballah L . Pressure Ulcers Prevalence in the Acute Care Setting: A Systematic Review, 2000-2015. Clin Nurs Res. 2018 Jul;27(6):643–59. https://doi.org/10.1177/1054773817705541  

52. Tomova-Simitchieva T , Akdeniz M , Blume-Peytavi U , Lahmann N , Kottner J . [The Epidemiology of Pressure Ulcer in Germany: systematic Review]. Gesundheitswesen. 2019 Jun;81(6):505–12.  

53. Öztürk R , Murt A . Epidemiology of urological infections: a global burden. World J Urol. 2020 Nov;38(11):2669–79. https://doi.org/10.1007/s00345-019-03071-4  

54. Kalánková D , Kirwan M , Bartoníčková D , Cubelo F , Žiaková K , Kurucová R . Missed, rationed or unfinished nursing care: A scoping review of patient outcomes. J Nurs Manag. 2020 Nov;28(8):1783–97. https://doi.org/10.1111/jonm.12978  

55. Jhung MA , Banerjee SN . Administrative coding data and health care-associated infections. Clin Infect Dis. 2009 Sep;49(6):949–55. https://doi.org/10.1086/605086  

56. Quan H , Eastwood C , Cunningham CT , Liu M , Flemons W , De Coster C , et al.; IMECCHI investigators . Validity of AHRQ patient safety indicators derived from ICD-10 hospital discharge abstract data (chart review study). BMJ Open. 2013 Oct;3(10):e003716. https://doi.org/10.1136/bmjopen-2013-003716  

57. Padula WV , Makic MB , Mishra MK , Campbell JD , Nair KV , Wald HL , et al.  Comparative effectiveness of quality improvement interventions for pressure ulcer prevention in academic medical centers in the United States. Jt Comm J Qual Patient Saf. 2015 Jun;41(6):246–56. https://doi.org/10.1016/S1553-7250(15)41034-7  

58. Berlowitz DR , Brandeis GH , Morris JN , Ash AS , Anderson JJ , Kader B , et al.  Deriving a risk-adjustment model for pressure ulcer development using the Minimum Data Set. J Am Geriatr Soc. 2001 Jul;49(7):866–71. https://doi.org/10.1046/j.1532-5415.2001.49175.x  

59. Berlowitz DR , Ash AS , Brandeis GH , Brand HK , Halpern JL , Moskowitz MA . Rating long-term care facilities on pressure ulcer development: importance of case-mix adjustment. Ann Intern Med. 1996 Mar;124(6):557–63. https://doi.org/10.7326/0003-4819-124-6-199603150-00003  

60. Letica-Kriegel AS , Salmasian H , Vawdrey DK , Youngerman BE , Green RA , Furuya EY , et al.  Identifying the risk factors for catheter-associated urinary tract infections: a large cross-sectional study of six hospitals. BMJ Open. 2019 Feb;9(2):e022137. https://doi.org/10.1136/bmjopen-2018-022137  

61. Li F , Song M , Xu L , Deng B , Zhu S , Li X . Risk factors for catheter-associated urinary tract infection among hospitalized patients: A systematic review and meta-analysis of observational studies. J Adv Nurs. 2019 Mar;75(3):517–27. https://doi.org/10.1111/jan.13863  

62. Di Pasquale M , Aliberti S , Mantero M , Bianchini S , Blasi F . Non-Intensive Care Unit Acquired Pneumonia: A New Clinical Entity? Int J Mol Sci. 2016 Feb;17(3):287. https://doi.org/10.3390/ijms17030287  

63. Sopena N , Heras E , Casas I , Bechini J , Guasch I , Pedro-Botet ML , et al.  Risk factors for hospital-acquired pneumonia outside the intensive care unit: a case-control study. Am J Infect Control. 2014 Jan;42(1):38–42. https://doi.org/10.1016/j.ajic.2013.06.021  

64. Federal Statistical Office (FSO) . Variablen der Medizinischen Statistik Spezifikationen gültig ab 1.1.2019 2019 14.01.2021. Available from: https://www.bfs.admin.ch/bfs/de/home/statistiken/gesundheit/erhebungen/ms.assetdetail.7066232.html  

65. Sabrià M , Sopena N . Nosocomial Pneumonia Outside the Intensive Care Unit. Clin Pulm Med. 2011;18(5):215–21. https://doi.org/10.1097/CPM.0b013e31822c5f19  

Appendix

Figure S1 Model fit by decile (hospital-acquired pressure ulcer (≥ stage II).

Figure S2 Model fit by decile (hospital-acquired urinary tract infection).

Figure S3 Model fit by decile (in-hospital mortality).

Figure S4 Model fit by decile (non-ventilator hospital-acquired pneumonia).