A qualitative analysis of critical incident reporting in a paediatric hospital: benefits and weaknesses

DOI: https://doi.org/10.4414/SMW.2022.w30051

Introduction

The idea of learning from failure or near misses has been used since the 1940s when critical incident investigations were performed by military pilots [1]. As early as 1978, Cooper et al. introduced these investigations into medicine. They interviewed anaesthesiologists to gather information about preventable incidents, focusing on the “human factor” [2]. In 1990, James Reason added a new dimension to these investigations by differentiating between an individual person’s mistakes and system-based errors [3]. He identified two issues now widely accepted. First of all, he emphasised that human individual errors will always occur but that system-based safety measures can be established to prevent them. His “Swiss cheese model” has been acknowledged with multiple awards [4]. Secondly, he showed that latent but repetitively occurring mistakes create considerable threats to safety as well. These latent errors can be dormant within a system but produce an opportunity for accident if combined with individual mistakes or local triggers [3].

The idea of introducing voluntary critical incident reporting systems (CIRSs) promised to identify mistakes and thus allow system-based changes to avoid them. Additionally, it was hoped to identify latent mistakes. Its value has been shown in a number of publications. Among other things, identifying latent errors has been shown to be possible and CIRSs recognised as a means for quality improvement [5]. The scope broadened from technical errors to teamwork, communications and organisational culture [6].

The CIRS is now well established in many paediatric hospitals. Data from paediatric intensive care units have been published repeatedly, often focusing on medication errors [7–9]. Information from investigation of children’s medicine on a broader scale, however, is scarce.

Additionally, there is evidence that caretakers are growing frustrated with a CIRS, believing little change results from reported incidents [10].

We set out to analyse all reported critical incidents in our children’s hospital, including data from intensive care, general paediatrics and paediatric surgery, the operating room and the emergency department. We aimed to better define how caretakers applied the reporting system. We also aimed to identify benefits as well as possible shortcomings and opportunities for improvement.

Methods

In a qualitative analysis, all incidents reported in 2018 with the online reporting system of the Children's Hospital Lucerne were evaluated. A link to the CIRS tool is located on the front page of the hospital’s intranet page. The intranet can be accessed by all employees with a personal login. Incidents are logged electronically in an easy-to-use, straightforward process. Incidents are entered anonymously; the name can be included voluntarily. No tracing to the login or the location of the computer is possible.

Knowledge of the tool is ensured by demonstrating it at the introduction seminar for new employees. Additionally, regular promotions and reminders are published by the hospital’s quality management.

All organisational units have an interprofessional CIRS team consisting of nurses and physicians. All team members receive an email as soon as a new incident has been reported. The team is responsible for analysing the incident within a week and taking appropriate measures. A qualitative approach was applied to analysis of the data. Two of the authors evaluated the data. Focus was on consequences/problems arising from the reported incident. After different types of consequences were identified, all incidents were coded into the developed categories by the same two authors. All categorisation was done iteratively, discussed when differing and adjusted if necessary. Disagreement about categories was resolved by discussion and consensus with all authors. In a second analysis, the data were additionally grouped into "self-reporting", and "third-party reporting". For third-party reported incidents, we additionally searched for information whether direct feedback had been given to the person responsible.

Results

In 2018, 496 incidents were reported. By analysis of all written information about the incidents, three different categories were defined regarding the problem caused. To better illustrate the categories some examples of incidents are listed.

Medical problems: Patients are directly affected by the incident and harm is (potentially) caused. For example:

• A drug was given twice a day, although it was prescribed once per day.
• Prescription dosage was out of the therapeutic range.

Organisational problems: No harm is or can be caused as a result of the incident. However, the incident leads to extra work or expenditure. Additional waiting time and additional blood taking are grouped in this category. For example:

• Missing mark on the surgical site causing increased waiting time preoperatively.
• Incorrect clotting factors were analysed because the person ordering the laboratory work could not transcribe Arabic to Roman numerals.

No problem: No medical or organisational problems arise or could arise. For example:

• Physician prescribes a higher dose of opioid than usual, but well within the therapeutic range.
• After careful evaluation, a patient was extubated, showed respiratory deficiency afterwards and needed to be re-intubated.

The 496 incidents were analyzed accord to these three categories. Overall, 307 (62%) of the incidents led to medical issues, 82 incidents (16%) led to organisational issues and the remaining 107 reported incidents (22%) did not lead to any problems.

All incidents were then analysed in more detail.

Medical problems (n = 307, 62%): 307 incidents did or could have resulted in medical harm. The majority of these, 224 incidents, were found to be medication errors:

• Administration errors (n = 165)
• Prescription errors (n = 59).

The remaining 83 incidents (potentially) leading to medical problems were grouped as follows.

• Technical issues with intravenous infusion (n = 24); for example, extravasation not being recognised quickly.
• Incorrect patient monitoring (n = 12); for example,  fluid balancing not documented.
• Technical problems with medical devices (n = 11); for example, leaks in new infusion tubes, damaged infusion pump modules.
• Incidents involving tracheal tubes (n = 4); for example, incorrect tube placement
• Laboratory tests (n = 7); for example,. blood sugar not tested as scheduled.
• Wrongly marked laboratory tests (n = 7) blood test not marked “urgent”, delaying results.
• Packaging of drugs (n = 4), e.g. packaging of normal saline solution look similar to another drug.
• Decubitus/positioning in the operation room (n = 8) bruises on heel after surgery.
• Documentation (n =1); missing documentation of Vitamin K administration postnatally.
• Surgery on the wrong side (n = 1); patient with suspected arthritis was scheduled for knee joint puncture. The wrong knee was punctured. The mistake was noticed while the child was still under general anaesthesia and the correct knee was additionally punctured.
• Isolation (n = 5) potentially infectious patients who were later found out to have contagious diseases were not isolated at admission.

Organisational problems (n = 82; 18%): 82 incidents reported were classified as incidents leading to organizational problems.

• Extra work (n = 30) urine sample of outpatient was lost and family had to come in again.
• Appointment problem (n = 13) patient was not scheduled for a follow-up visit.
• Interdisciplinary miscommunication (n = 10) due to a misunderstanding between anaesthesiologist and surgeon an operation was delayed.
• Technical problems (n = 8) the blood glucose meter did not work and test had to be done in a different room.
• Lost items (n = 7) blood sample lost.
• Preoperative preparation (n = 6) patient not told to fast.
• Documentation (n = 7) a physician forgot to document the positive end-expiratory pressure (PEEP) for an intubated patient on the designated sheet, which would have been necessary according to the wards' guidelines. The PEEP was correctly documented on the patient chart.
• Transportation (n = 2), e.g. a dispensable but useful connection device of the ventilator was missing during transportation.
• Misidentification (n = 1) patient scheduled for an orthopaedic procedure was taken to a preoperative ear-nose-throat (ENT) consultation that was planned for another patient.
• Treatment plan (n = 3) chemotherapy for an oncology patient was not on ward at time of admission.

No problem (n = 107; 21%): 107 incidents did not result in any problem. No subcategorisation was performed because of the vast variety of incidents reported. The texts suggested that the CIRS was used to voice personal complaints.

• A patient was discharged after the cleaning staff had already left and the cleaning had to be done by nursing staff.
• An umbilical catheter was well secured, but differently from usual.
• A deck chair collapsed with a father and child in it. No one was hurt.
• A "schnitzel", part of a patient's meal, wasn’t cooked to the patient’s liking.
• An examination took longer than expected because it was performed by a medical student.

In the second analysis, the focus was on the person reporting the incident and knowledge of the consequences. The reporting system does not give information on the filer's profession. However, the profession could be identified in most cases. Since it could not be identified unequivocally, the data were not analysed quantitatively. Nevertheless, approximately 95% of the reports were certainly entered by nursing staff.

Overall, 54 incidents were self-reported. In 46 cases direct feedback to the caregiver responsible was documented. For 396 incidents, there was no evidence that the person responsible was informed. In many cases, the text suggested that no feedback had been given. The data subdivided into the result categories is presented in figure 1.

Figure 1 Feedback.

Discussion

The analysis of all incidents reported in 2018 in the children's hospital showed a frequent use of the system with an average of 1.3 reports per day.

The predominance of medication errors in our analysis is in accordance with the literature; there are a large number of publications that describe large numbers of medication errors in their CIRS data [7, 11–13]. Frey et al. analysed system changes resulting from CIRS data and found that most were based on minor critical incidents, which were detected by a number of similar reports and after a longer period of time [7].

The majority of reports in our analysis seem to have been made by nursing staff; this phenomenon is in accordance with data from the literature. In their analysis, Sharein et al. found most reports to be filed by nurses, with the number increasing during their study period [14]. The low number of reports filed by physicians and therapists, in our analysis (approximately 5%), is conspicuous and somewhat alarming. The usefulness of a CIRS relies on its multi-professionalism and many different healthcare professions should provide input. The reason for physicians' lack of reporting incidents was not part of our analysis, but needs to be addressed in the future. By interpreting and discussing the data from the analysis, the authors consider the following reasons for the discrepancy likely: different views on which incidents are critical and therefore justify reporting; reports only being filed when a system-based error is thought likely to be discovered; and lack of time. Another reason for a reluctance to use the system may be lack of follow-up information. Hubertus et al. examined experience and satisfaction with a CIRS and two thirds of study participants did not identify any improvement following their report [15].

The decision as to which incidents should be reported seems to be a balancing act. Both overuse of the system and increasing reluctance to report incidents threaten to endanger its functionality. By categorising our reported incidents according to problems created, we tried to discriminate between relevant and possibly irrelevant issues. The large number of incidents in the categories "organisational problems" and "no problem", which together comprise 38% of all reported incidents, raises the question whether  the CIRS should focus on medical issues or incorporate organisational issues as well. Thus far, the CIRS literature has focused on types of errors reported and resulting system-based changes, therefore the issue of restricting the system to issues causing, or potentially leading to, medical errors can not be answered by analysing published data.

There are important reasons to oppose a restriction of CIRS systems in any way. As emphasised above, even minor incidents need to be reported to allow identification of latent errors over time. But the growing reporting fatigue, especially among physicians, is certainly fuelled by non-critical incidents being reported. Our data give evidence that in a large number of cases, a clarifying conversation could have done more to prevent future organisational problems and reduce expenditure.

So instead of restricting the CIRS, we propose the creation of an additional tool that enables valuable feedback. This could be another portal to report less dangerous, but nevertheless important, issues, which assists in giving direct, non-blaming feedback. Thus, a CIRS can be restored to its primary task while complaints and annoyances are addressed in a more profitable matter. Both systems need to be permeable.

The focus of CIRS literature has been on identifying system-based errors and causes of mistakes. This is fo,r example, evident in the Yorkshire framework, which regards causes of mistakes more important than individual mistakes, emphasised by the statement "A focus on individual responsibility for errors is likely to be ineffective as an incident reduction strategy" [16].

We acknowledge that analysing system-based errors and causes are essential, but our data suggest that diminishing the importance of the individual error is dangerous. Individual actions must never be reason to blame and shame, but regarded as an opportunity for learning. Reflective observation is a key component of experiential learning and only possible if mistakes are acknowledged. Our suggestion is further supported by a study by Ramirez et al. They found root-cause analysis to be one factor associated with a reduction of near misses or adverse effects [17].

A complementary feedback tool needs to be developed to counteract the filing of personal complaints in the CIRS and to inform the person responsible in order to enable learning and improvement.

Conclusion

The critical incident reporting system of our children’s hospital is an established, regularly used quality improvement tool. It has helped to decrease system-based threats to patient safety. Nonetheless, the system's functionality is threatened by misuse in the form of filing non-critical, personal complaints on the one hand and physicians' reluctance to use it on the other. Additionally, our data suggest that individual feedback is often missing, thus decreasing the potential for individual learning and improvement in order to avoid future mistakes. To counter the misuse and simplify personal feedback, a second online tool needs to be developed and implemented.

References

1. Flanagan JC . The critical incident technique. Psychol Bull. 1954 Jul;51(4):327–58. https://doi.org/10.1037/h0061470  

2. Cooper JB , Newbower RS , Long CD , McPeek B . Preventable anesthesia mishaps: a study of human factors. Anesthesiology. 1978 Dec;49(6):399–406. https://doi.org/10.1097/00000542-197812000-00004  

3. Reason J . Human error: models and management. BMJ. 2000 Mar;320(7237):768–70. https://doi.org/10.1136/bmj.320.7237.768  

4. Reason J . James Reason: patient safety, human error, and Swiss cheese. Interview by Karolina Peltomaa and Duncan Neuhauser. Qual Manag Health Care. 2012 Jan-Mar;21(1):59–63. https://doi.org/10.1097/QMH.0b013e3182418294  

5. Short TG , O’Regan A , Jayasuriya JP , Rowbottom M , Buckley TA , Oh TE . Improvements in anaesthetic care resulting from a critical incident reporting programme. Anaesthesia. 1996 Jul;51(7):615–21. https://doi.org/10.1111/j.1365-2044.1996.tb07841.x  

6. Beydon L , Conreux F , Le Gall R , Safran D , Cazalaa JB ; ‘Sous-commission de Materiovigilance’ for Anaesthesia and Intensive Care . Analysis of the French health ministry’s national register of incidents involving medical devices in anaesthesia and intensive care. Br J Anaesth. 2001 Mar;86(3):382–7. https://doi.org/10.1093/bja/86.3.382  

7. Frey B , Buettiker V , Hug MI , Waldvogel K , Gessler P , Ghelfi D , et al.  Does critical incident reporting contribute to medication error prevention? Eur J Pediatr. 2002 Nov;161(11):594–9. https://doi.org/10.1007/s00431-002-1055-0  

8. Ghaleb MA , Barber N , Franklin BD , Yeung VW , Khaki ZF , Wong IC . Systematic review of medication errors in pediatric patients. Ann Pharmacother. 2006 Oct;40(10):1766–76. https://doi.org/10.1345/aph.1G717  

9. Miller MR , Robinson KA , Lubomski LH , Rinke ML , Pronovost PJ . Medication errors in paediatric care: a systematic review of epidemiology and an evaluation of evidence supporting reduction strategy recommendations. Qual Saf Health Care. 2007 Apr;16(2):116–26. https://doi.org/10.1136/qshc.2006.019950  

10. Shojania KG . The frustrating case of incident-reporting systems. Qual Saf Health Care. 2008 Dec;17(6):400–2. https://doi.org/10.1136/qshc.2008.029496  

11. Bordun LA , Butt W . Drug errors in intensive care. J Paediatr Child Health. 1992 Aug;28(4):309–11. https://doi.org/10.1111/j.1440-1754.1992.tb02674.x  

12. Raju TN , Kecskes S , Thornton JP , Perry M , Feldman S . Medication errors in neonatal and paediatric intensive-care units. Lancet. 1989 Aug;2(8659):374–6. https://doi.org/10.1016/S0140-6736(89)90548-5  

13. Folli HL , Poole RL , Benitz WE , Russo JC . Medication error prevention by clinical pharmacists in two children’s hospitals. Pediatrics. 1987 May;79(5):718–22.  

14. Scharein P , Trendelenburg M . Critical incidents in a tertiary care clinic for internal medicine. BMC Res Notes. 2013 Jul;6(1):276. https://doi.org/10.1186/1756-0500-6-276  

15. Hubertus J , Piehlmeier W , Heinrich M . Communicating the improvements developed from Critical Incident Reports is an essential part of CIRS. Klin Padiatr. 2016 Sep;228(5):270–4. https://doi.org/10.1055/s-0042-113311  

16. Lawton R , McEachan RR , Giles SJ , Sirriyeh R , Watt IS , Wright J . Development of an evidence-based framework of factors contributing to patient safety incidents in hospital settings: a systematic review. BMJ Qual Saf. 2012 May;21(5):369–80. https://doi.org/10.1136/bmjqs-2011-000443  

17. Ramírez E , Martín A , Villán Y , Lorente M , Ojeda J , Moro M , et al.; SINOIRES Working Group . Effectiveness and limitations of an incident-reporting system analyzed by local clinical safety leaders in a tertiary hospital: prospective evaluation through real-time observations of patient safety incidents. Medicine (Baltimore). 2018 Sep;97(38):e12509. https://doi.org/10.1097/MD.0000000000012509