High accuracy of the nine equivalents of nursing manpower use score assessed by critical care nurses
PRINCIPLES: The nine equivalents of nursing manpower use score (NEMS) is frequently used to quantify, evaluate and allocate nursing workload at intensive care unit level. In Switzerland it has also become a key component in defining the degree of ICU hospital reimbursement. The accuracy of nurse registered NEMS scores in real life was assessed and error-prone variables were identified.
METHODS: In this retrospective multicentre audit three reviewers (1 nurse, 2 intensivists) independently reassessed a total of 529 NEMS scores. Correlation and agreement of the sum-scores and of the different variables among reviewers, as well as between nurses and the reviewers’ reference value, were assessed (ICC, % agreement and kappa). Bland & Altman (reference value – nurses) of sum-scores and regression of the difference were determined and a logistic regression model identifying risk factors for erroneous assessments was calculated.
RESULTS: Agreement for sum-scores among reviewers was almost perfect (mean ICC = 0.99 / significant correlation p <0.0001). The nurse registered NEMS score (mean ± SD) was 24.8 ± 8.6 points versus 24.0 ± 8.6 points (p <0.13 for difference) of the reference value, with a slightly lower ICC (0.83). The lowest agreement was found in intravenous medication (0.85). Bland & Altman was 0.84 ± 10, with a significant regression between the difference and the reference value, indicating overall an overestimation of lower scores (≤29 points) and underestimation of higher scores. Accuracy of scores or variables was not associated with nurses’ characteristics.
CONCLUSIONS: In real life, nurse registered NEMS scores are highly accurate. Lower (≤29 points) NEMS sum-scores are overestimated and higher underestimated. Accuracy of scores or variables was not associated with nurses’ characteristics.
- Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med. 2002;346:1715–22.
- Thorens JB, Kaelin RM, Jolliet P, Chevrolet JC. Influence of the quality of nursing on the duration of weaning from mechanical ventilation in patients with chronic obstructive pulmonary disease. Crit Care Med. 1995;23:1807–15.
- Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction. JAMA. 2002;288:1987–93.
- Noseworthy TW, Konopad E, Shustack A, Johnston R, Grace M. Cost accounting of adult intensive care: methods and human and capital inputs. Crit Care Med. 1996;24:1168–72.
- Cullen DJ, Civetta JM, Briggs BA, Ferrara LC. Therapeutic intervention scoring system: a method for quantitative comparison of patient care. Crit Care Med. 1974;2:57–60.
- Keene AR, Cullen DJ. Therapeutic intervention scoring system: update 1983. Crit Care Med. 1983;11:1–3.
- Miranda DR, de Rijk A, Schaufeli W. Simplified therapeutic intervention scoring system: the TISS-28 items results from a multicenter study. Crit Care Med. 1996;24:64–73.
- Reis Miranda D, Moreno R, Iapichino G. Nine equivalents of nursing manpower use score (NEMS). Intensive Care Med. 1997;23:760–5.
- Le Gall J-R, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European / North American multicenter study. JAMA. 1993;270:2957–63.
- Deutsches Institut für Medizinische Dokumentation und Information. OPS Version 2011, Kapitel 8 Nichtoperative Therapeutische Massnahmen 8-98. Available via http://www.dimdi.de/static/de/klassi/prozeduren/ops301/opshtml2011/block-8-97...8-98.htm Accessed June 23, 2011.
- SwissDRG, Schweizerische Operationsklassifikation (CHOP), Systematisches Verzeichnis, Vers 2011 – 2. November 2010; Publikation komplett: p 280–281. Available from: http://www.bfs.admin.ch/bfs/portal/de/index/news/publikationen.html?publicationID=4096 Accessed June 23, 2011.
- Schuetz P, Albrich WC, Suter I, Hug BL, Christ-Crain M, Holler T, et al. Quality of care delivered by fee-for-service and DRG hospitals in Switzerland in patients with community-acquired pneumonia. Swiss Med Wkly. 2011;141:w13228. doi: 10.4414/smw.2011.13228.
- Frutiger A. Qualitätssicherung in der Intensivmedizin: die Situation in der Schweiz. Schweiz Med Wochenschr. 1999;129:1592–9.
- Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005;85:257–68.
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10.
- Junger A, Brenck F, Hartmann B, Klasen J, Quinzio L, Benson M, et al. Automatic calculation of the nine equivalents of nursing manpower use score (NEMS) using a patient data management system. Intensive Care Med. 2004;30:1487–90.