Critical incident reporting over time: a retrospective, descriptive analysis of 5493 cases

DOI: https://doi.org/10.4414/SMW.2021.w30098

Introduction

According to the World Health Organization (WHO), 1 in 10 patients is harmed while receiving hospital care [1], and a recent meta-analysis showed that half of the harms are considered preventable [2]. Reporting of and learning from critical incidents, defined as any event that has the potential to cause damage if no intervention occurs [3], is considered key in preventing future medical errors [1]. Therefore, as recommended by the WHO and many national healthcare authorities, incident reporting and learning systems have been established in the healthcare institutions of most industrialised countries across Europe, North America, Japan and Australia [4]. The purpose of critical incident reporting systems (CIRSs) is to collect and analyse critical events, in order to create awareness of these situations and thereby reduce potential risks by learning from them in order to prevent similar events in the future [5–7].

Critical incidents are reported by employees, but the analysis and rating of severity is carried out by a team of experts who then decide in collaboration with the employees which actions need to be taken. Usually, the finalised cases are published within the institution to be seen by other co-workers, and in some systems even nationwide. Granting anonymity and freedom from sanctions are considered crucial for motivating employees to report incidents [1, 8, 9].

In the past two decades, numerous published studies have addressed critical incident reporting in clinical specialties including traumatology [10], radiology [11], ophthalmology [12], obstetrics [13], emergency medicine [14] and dermatology [15], amongst others. These reports conclusively identified the main factors contributing to critical incidents such as medication errors, lack of communication and organisation failure.

However, many other questions required to obtain a more comprehensive picture regarding incident reporting have rarely been addressed. Therefore, the purpose of this retrospective study was to review trends in critical incident reporting over a 7-year period in a tertiary hospital in Switzerland. To address the study purpose, we aimed (1) to determine the distribution of critical incidents across clinical specialties, (2) to describe CIRS reporter’s professional profiles, (3) to explore types, severity and risk of reoccurrence of critical incidents, and (4) to investigate factors contributing to such incidents and preventive actions that have been taken in response.

Methods

Setting and definitions of “report” and “case”

In this investigator-initiated, single-centre, retrospective, cross-sectional study we analysed reports filed in the last 7 years in the CIRS of our 770-bed tertiary referral centre. The University Hospital of Basel (USB) consists of 44 specialty clinics with approximately 7000 employees and about 38,000 inpatients annually. The hospital’s CIRS is organised in 17 reporting circles across the specialty clinics, in administration and in maintenance.

A “report” is the description of the incident drafted by an employee. A “case” is the finalised version including the report, the assessment, the categorisation and the feedback from the CIRS circle-managers.

Critical incident reporting in our institution

The hospital’s CIRS was first introduced in the department of anaesthesiology [16] in 1995 and subsequently implemented over the years across the specialty clinics with use hospital-wide since 2013. The system is annually updated and refined. All employees are invited to voluntarily and anonymously report incidents that did or could have caused patient harm, unsettled patients, required a revision of procedures and/or could have been avoided. Complaints, allegations, liability issues, haemato- and pharmacovigilance reports are collected and processed elsewhere and are not part of the CIRS.

In principle, our CIRS operates as a cycle of an afferent loop (incident reporting), case processing and an efferent loop (feedback from the CIRS teams to the reporter) to suggest actions aimed to reduce the probability of reoccurrence. More specifically, incident reports are submitted by employees via a data entry screen of the e-portal (CIRSmedical^© by ProtecData AG). The system automatically assigns a number to the report, which only the submitting employee knows, granting anonymisation to the reporting professional and still allowing inquiries. Then, the trained CIRS team of the assigned reporting circle reviews the incoming reports. The teams, which consist of a clinical physician and nurses working on these wards, underwent specific training including personal coaching by CIRS experts, group discussions of filed reports under supervision of CIRS experts and attendance of four 2-hour meetings per year. Furthermore, the employee orientation programme includes a brief CIRS introduction, as well as access to a short instruction video. The CIRS team checks for anonymisation, relevance and type of CIRS reports with the possibility to reject reports based on inadequate content (as stated above) or quality (e.g., too fragmentary information). This check is followed by a risk assessment (severity and likelihood of incident reoccurrence). After this analytic process, feedback to the reporter is given including recommendations for actions to be taken (i.e. suggestions towards structural and/or procedural improvement) if indicated. Finally, all cases are published on the hospital’s intranet for information and learning purposes. In addition, biannual reports from each CIRS reporting circle are communicated to the leadership of the respective specialty clinics and an annual hospital-wide CIRS report is published. Moreover, selected CIRS cases with lessons learned are published as safety alerts and in newsletters to inform hospital employees.

Variables and measurements

Risk assessment

The risk score for patient harm is assessed for each report by grading its severity on a scale from low to extreme risk according to the UK’s National Health Services (NHS) [17]. The likelihood of incident reoccurrence and severity of potential harmful consequences are estimated according to the NHS standardised guidelines [17]. The risk is calculated by multiplying the consequences by the likelihood and depicted in a matrix indicating low, moderate, high and extreme risk, which are depicted using different colours from green (low risk) to red (extreme risk). An example is given in the appendix (figure S1).

Incident report classification

All reports were assigned by the reporting person into type and contributing factors from a drop-down menu with known factors based on the “International Classification on Patient Safety (ICPS)” [18]: An incident type is a category made up of incidents of a common nature and is a “parent” category. A contributing factor is a circumstance that is thought to have played a part in the origin of an incident. Contributing factors may be external (not under control of the hospital, such as product and technology factors), organisational (e.g., lack of resources), staff-related (e.g., poor team work, lack of knowledge) or patient-related (e.g., non-adherence). Attribution of contributing factors to the incidents are based on the assessments of the reporters and the CIRS teams.

Actions following incidents

After reviewing the incident report, the CIRS teams recommend actions to be taken. The CIRS tool allows for differentiation between executive categories (“not required”, “initiated” and “already implemented”) and content categories (“communication, guideline, SOP”, “technical and material”, “education and supervision” and “organisational and process”). Multiple categories per report are possible. Additionally, the CIRS circle teams can give written feedback to the reporting employee via the CIRS tool.

Dataset and data analysis

We extracted all cases from the CIRS database covering a 7-year time period from 1 January 2013 to 31 December 2019 and analysed the following variables:

• Number of reports per department / clinical specialty

• Profession of the reporter

• Type of incident

• Severity of consequences

• Likelihood of reoccurrence

• Contributing factors

•  Actions taken following an incident

Descriptive statistics were calculated using IBM’s SPSS Statistics, Version 26. Because of the voluntary and anonymous nature of reporting, some items have not been completed by the reporter. In these cases, the missing data are indicated as “N/A” in the appendix.

The study was approved by the ethical committee of north-western Switzerland (project-ID: 2020-00522)

Results

During the study period a total of 6276 reports were submitted to the hospital’s CIRS database, of which 783 were excluded for being duplicates or non-CIRS cases. Thus, a total of 5493 critical incident cases were included in our analysis. In 1290 cases (23.5%) no patient was directly affected by the incident. Female patients were affected in 2225 (40.5%) cases, male patients in 1987 (36.0%); the remainder was unspecified. The age group most affected was 61–70 years. The annual number of cases varied between 606 (2019) and 873 (2014), and rates of the critical incidents per 1000 patient days varied between 4.04 (2014) and 2.86 (2019) with a slight decrease over time (fig. S2 in the appendix).

Reported incidents by departments

Most incidents were reported from the reporting circles internal medicine (n = 1400, 25.5%), followed by general surgery (n = 1013, 18.4%), laboratory medicine / clinical microbiology (n = 964, 17.5%), anaesthesiology (n = 367, 6.7%) and obstetrics and reproductive medicine (n = 351, 6.4%). These five clinical areas accounted for 74.5% of all incident reports (fig. 1). The remaining 12 reporting circles contributed less than 5% each to the reported incidents.

Figure 1 Reporting by departments over time.

Except for laboratory medicine / clinical microbiology, reporting of the clinics remained stable or increased. Of note, the reporting by laboratory medicine / clinical microbiology decreased markedly over time. The complete table including the annual reports of all departments is shown in the appendix (table S1).

Reporting by profession

The majority of reports were submitted by nurses (n = 2432, 44.3%), followed by physicians (n = 1186, 21.6%) and biomedical technicians (n = 984, 17.9%). The longitudinal trends of all professional categories are shown in figure 2. Reporting by nurses and midwives increased and those by physicians remained stable, whereas reporting by biomedical technicians decreased significantly with time.

Figure 2 Reporting by a selection of professional groups.

Other professional groups such as therapists, pharmacists and administrative staff each accounted for less than 1% of reported incidents (detailed information in the appendix, table S2).

Types of incidents

During the observation period, reports related to medications (n = 1295, 32.8%) and clinical procedures (e.g., venous puncture or a specific surgical operation, n = 1286, 32.6%) were most common. Other frequently reported types of incidents were related to behaviour of employees or patients (23.3%), documentation errors (21.5%) and lack of resources (19.4%). Incidents related to equipment, nosocomial infections or dietary incidents were each responsible for less than 10% of total reports. The temporal trends of the most common types of incidents are depicted in figure 3; total numbers of all types are shown in the appendix, table S3.

Figure 3 The five most common types of incidents.

Severity and likelihood of reoccurrence of incidents

In 4727 (86%) of the 5493 assessed cases, risk profiles were assigned as follows (figure 4): 15.4% low risk (n = 726, green cells), 33.3% moderate risk (n = 1578, yellow cells), 48% high risk (n = 2268, orange cells) and 3.3% extreme risk (n = 155, red cells). 

By far the most common combination of likelihood of reoccurrence and severity of potential consequences of an incident was “possible reoccurrence” and “moderate severity” (n = 1319, 27.9%), which is considered as high risk.

Figure 4 Risk assessment matrix according to the NHS. The colors indicate the risk score from low risk (green) to extreme risk (red). Percentages are shown in brackets.

Contributing factors

The top five factors (multiple factors per case were possible) assessed as contributing to the incidents were personal factors (e.g., fatigue, distraction, n = 2067, 44%), lack of training or knowledge (n = 2054, 43.7%), communication errors (n = 1698, 36.1%), organisational (e.g., work load, lack of resources, n = 1308, 27.8%) and medication errors (e.g., over-/underdosing, mix-ups, n = 1269, 27%). The number of contributing factors per report increased from 1.5 (2013) to 2.2 (2019). Over time, medication incidents and lack of training and knowledge increased whereas the other factors remained stable. Since 2016, lack of training and knowledge was the most cited factor contributing to more than half of all reported incidents (fig. 5). All contributing factors are listed in the appendix, table S4.

Figure 5 Percentages for the top five categories contributing to the incidents.

Actions taken in response to the filed reports

In 67.7% (3719/5493) of the cases, the CIRS teams recommended or initiated actions following a report. Most of these actions (23.6%) aimed at improving communication between employees and/or within teams involved in the CIRS case (e.g., creating or improving standard operational procedures (SOPs)). The other categories (organisation and process, technical and material) remained at a constant low level, accounting for less than 10% each throughout the observed period (table S5, appendix). From 2017 onwards most commonly, no actions were initiated (labelled as “Not required”). In 21.6% (2019) CIRS teams did not respond to the reports at all (labelled as “No response” in figure 6). An increasing number of reporters received written feedback from the CIRS teams.

Figure 6 Longitudinal changes of actions following incidents.

Discussion

In this descriptive, retrospective study we have analysed all reported critical incidents and their consequent actions (CIRS cases) documented in our CIRS between 2013 and 2019. The analysis of 5493 cases showed that most had been reported by the departments of internal medicine and general surgery, and more than half of the incidents by nurses. The most common types of incident were medication-related, involved a clinical procedure or intervention and were predominantly associated with personal factors, lack of training/knowledge and communication errors. Over half of the reports were classified as “high risk” (48%) or “extreme risk” (3.3%) incidents. In two thirds of the cases actions were taken, mainly aiming at improving communication.

Number of reports per department

In our analysis the reporting circles of the departments of internal medicine and general surgery reported most incidents, presumably as these are the largest clinical specialties in our hospital and probably also because the two major types of incidents (i.e. medication related in internal medicine and clinical/surgical procedures in general surgery) are highly prevalent in these specialties.

In contrast to other recent studies [19, 20], only relatively few incidents were reported by the intensive care unit (ICU). The small numbers of reports in our dataset could be due to the distinction between ICU and anaesthesiology and/or the preference to discuss incidents as part of on-location debriefings instead of using the CIRS tool.

The department obstetrics and reproductive medicine showed a large increase in reports since 2017. One reason for this could be an open and encouraging error and feedback culture in this department, including meetings to discuss the reports and a “CIRS case of the month” in their teams. The usefulness of such feedback mechanisms and communication of improvements has previously been reported [21]. Accordingly, the departments’ CIRS team demonstrates how assuming responsibility for error culture can lead to increased reporting.

The massive drop in reports filed by the department of laboratory medicine / clinical microbiology might have originated in an initial “misuse” of the CIRS by employees. After implementation of the CIRS it became obvious that employees used the system as a continual improvement tool, which is not its intended purpose. As a result, the CIRS team installed an additional continual improvement tool and trained employees in the distinction between these two systems, resulting in the drop of reports in the CIRS. Thus, the number of reports as of 2017 is more accurate.

Profession of the reporter

In line with other studies [22–24], nurses and physicians were the professional groups most commonly reporting incidents. The lower reporting rates of physicians compared with nurses have been discussed already, stating that doctors mistrust the systems, feared how reports would be used or are uncertain about what to report [8]. However, the same might apply to other professional groups including nurses. We should interpret the proportions of filed reports in the context of the relative proportions of the professional workforces in the hospital, such as 22% for physicians, 35% for nurses, 28% for other types of healthcare workers, and 15% for administrative and housekeeping staff.

The increasing number of reports filed by midwives may reflect the “CIRS-friendly” leadership of the department of obstetrics and reproductive medicine mentioned above. An additional reason for this increase might be a new curriculum in midwife training: a few years ago, it was changed to a Bachelor of Science degree with an in-depth focus on communication and error management.

Type of incident

Medication-related incidents became the leading type of incident in the past three years. The high frequency of medication errors compared with other incident types has also been reported in other studies [20, 25]. Nevertheless, we were surprised by its steady increase although this is a widely known issue that usually receives special attention by the hospital’s risk management. Possible reasons could be that patients nowadays receive an increasing number of drugs with more complex protocols, particularly in tertiary referral hospitals as ours. Moreover, turnover of employees is high in university hospitals, possibly contributing to communication errors, which were also consistently among the top three types of incident. The geographic location of our hospital could be an additional factor to explain medication and communication errors: it is situated at the border of three countries (France, Germany and Switzerland); many employees were educated in one of these countries, with slightly different curricula and terminology, and in different languages. The increase in incidents related to behaviour could be due to the implementation of electronic special medication ordering systems within clinical specialties. A lack of interoperability of the patient data management systems used might provoke workarounds to maintain flow of information and documentation.

Likelihood of reoccurrence and severity of potential consequences

Over half of all assessed incidents were considered “high risk” or “extreme risk”. The most commonly rated severity of potential consequence was “moderate” (41.3%), and the most commonly assigned likelihood of reoccurrence was “possible” (63.5%). This indicates that predominantly moderate to high-risk incidents with reasonable likelihood of reoccurrence were reported.

According to the guidelines by the NHS [17], "moderate severity" implies a potential harm to the patient extending the duration of hospitalisation by 4–14 days. If extrapolated to our study population, the moderate-severe cases alone (n = 1955) might in theory have resulted in a total of 17,595 additional hospitalisation-days (if calculated with 9 additional days per case), corresponding to more than 48 patient-years.

Contributing factors

Unsurprisingly, the proportion of medication errors increased considerably when looking at the contributing factors. Remarkably, lack of training and knowledge also increased as a reported factor over time and was even the main contributing factor in the last 4 years. This may be counterintuitive, as medical training and knowledge is steadily increased in our hospital, as is the dissemination of information about the CIRS. However, it appears that these measures still do not compensate for lack of training and knowledge of individual health professionals. Yet, the relative increase of the contributing factor “lack of training and knowledge” may also reflect the growing need for specific knowledge and skills of new and changed treatment procedures given the growing complexity of patient care needs. In addition, this finding might also be partially explained by changes in the trained CIRS team of the assigned reporting circles. Many of our CIRS experts remained in service over the 5-year period, but changes in CIRS team occur(ed) frequently owing to career developments and for other reasons. In contrast, the proportion of communication errors remained roughly constant, organisational factors slightly declined – both of which may possibly reflect a positive effect of the CIRS feedback loop.

Actions taken in response to the filed reports

We were surprised by the fact that education/supervision was a measure only rarely initiated. This could be a further explanation why lack of training and knowledge were considered key factors contributing to many incidents. Moreover, the proportions of “no action required” and “no response given” were comparable to that of “improved communication, establishment of guidelines and/or SOPs” (approximately 25%). Thus, it appears that the feedback (or efferent) loop of our CIRS was not adequately used. This is also supported by the fact that in 2019 about 25% of the filed reports were still not even answered with written feedback including recommendations for improvement. Development of effective measures may be time-consuming, requires several meetings or even special task groups and approval by decision-makers. Most employees working in health care do not have spare capacity to engage in such a process in everyday work. They may hardly have time to file a report and even less to take actions from it – the latter being of paramount importance, as described before [26].

However, it is possible that reporters and the CIRS teams gave direct feedback to the relevant coworkers without waiting for or filing the CIRS feedback, as reported recently [27]. Therefore, the aforementioned amount of feedback including pragmatic actions taken could be higher in reality. It was previously stated that feedback about errors is a predictive dimension for event reporting [21]. Yet, despite the increasing feedback rate, the number of filed reports decreased during our observation period. Hence, we cannot confirm this relationship in our study.

Limitations

Our analysis is limited by several aspects: the biggest issue when analysing a CIRS is the quality of the filed reports. It is in the nature of such a system that the entered information is often not well standardised, with sometimes only fragmented information. Therefore, and based on the voluntary principle of a CIRS, our data only provide a rough approximation of the occurrence of critical incidents in our institution during the observation period – presumably, only “the tip of the iceberg” is represented in this study, limiting the generalisability of the data. Furthermore, the reports and actions taken by different departments are influenced by the number of health professionals, patient populations treated and the readiness to report incidents (“safety culture”), including the stance of the department’s leadership towards our CIRS. Hence, the reporting circles in our study were heterogeneous, which certainly had an impact on the results. Moreover, some CIRS teams prefer to give direct feedback to their employees without using the CIRS. In these cases, even if actions were taken after an incident, they do not show up in the dataset and therefore induce reporting bias. In addition to the CIRS, other specific tools are being used to report critical incidents such as the pharmacovigilance, haemovigilance, and falls registry portals. Furthermore, the generalisability of our data is limited because publicly available data from other CIRS in Switzerland is scarce to nonexistant and we therefore only analysed data from our institution.

Conclusion

Our analysis revealed that the consistent increase in medication- and education-related incidents over time markedly contrasted to the disproportionately low number of actions taken to address those issues. Thus, it remains unclear if we, as an organisation, really learn from critical incident reporting to prevent future incidents.

We conclude that our analysis of incidents via the CIRS was helpful in determining trends over time, differences in reporting circles/departments/professions and types of incidents, amongst other variables. Better insights into these aspects may be of particular importance for leaders of healthcare institutions to (1) optimise their institution`s CIRS policy, (2)) to anticipate CIRS-relevant “hot spots” within their institution, (3) to detect weaknesses within their CIRS and (4)) to react adequately to longitudinal trends.

Furthermore, national and international comparison of institutions will be facilitated if institutions share their longitudinal data with their peers.

According to this analysis, the feedback loop was the weakest part of our CIRS, indicating that unfortunately no learning effect and improvement of patient safety may result from many reports. However, it is possible that reading the reports might have an effect. Still, the discrepancy between the significant increase in lack of training and education as factors contributing to incidents and the minimal number of educational actions taken is unsettling and needs to be tackled.

Acknowledgments

We thank Ivana Cvijetic for administrative support in this project. We also thank Dr Dorothy Huang and Martina Gisin for providing us with further insights into the CIRS management of the obstetrics department. We particularly thank Anna Larbalestier, contract biostatistician, for native speaker proofreading and statistical support as well as Prof. Dr Thierry Girard for his helpful suggestions to improve the manuscript.

References

1. World Health Organization . (2019). Patient safety - Global action on patient safety. Report by the Director-General. Geneva: World Health Organization 2019. https://apps.who.int/gb/ebwha/pdf_files/WHA72/A72_26-en.pdf. (accessed 2020 June 23) 

2. Panagioti M , Khan K , Keers RN , Abuzour A , Phipps D , Kontopantelis E , et al.  Prevalence, severity, and nature of preventable patient harm across medical care settings: systematic review and meta-analysis. BMJ. 2019 Jul;366:l4185. https://doi.org/10.1136/bmj.l4185  

3. de Vries EN , Ramrattan MA , Smorenburg SM , Gouma DJ , Boermeester MA . The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care. 2008 Jun;17(3):216–23. https://doi.org/10.1136/qshc.2007.023622  

4. Glossar Patientensicherheit . Definition und Begriffsbestimmungen. Ärztliches Zentrum für Qualität in der Medizin (äzq). http://www.aezq.de/mdb/edocs/pdf/patientensicherheit/glossar-patientensicherheit.pdf. Published 2005. (Accessed 2020 July 16) 

5. World Health Organization . (‎2005)‎. World alliance for patient safety: WHO draft guidelines for adverse event reporting and learning systems: from information to action. World Health Organization. https://apps.who.int/iris/handle/10665/69797  

6. Petschnig W , Haslinger-Baumann E . Critical Incident Reporting System (CIRS): a fundamental component of risk management in health care systems to enhance patient safety. States Health. 2017;3(9):9. https://doi.org/10.1186/s40886-017-0060-y  

7. Bommersholdt ME . (2014). Wesentliche Erkenntnisse und Empfehlungen zu Melde- und Lernsystemen für Zwischenfälle auf dem Gebiet der Patientensicherheit in Europa. https://studylibde.com/doc/17369186/melde--und-lernsystemen-f%C3%BCr-zwischenf%C3%A4lle-auf-dem-gebiet-der. (accessed 2020 July 17) 

8. Mitchell I , Schuster A , Smith K , Pronovost P , Wu A . Patient safety incident reporting: a qualitative study of thoughts and perceptions of experts 15 years after ‘To Err is Human’. BMJ Qual Saf. 2016 Feb;25(2):92–9. https://doi.org/10.1136/bmjqs-2015-004405  

9. Leape LL . Reporting of adverse events. N Engl J Med. 2002 Nov;347(20):1633–8. https://doi.org/10.1056/NEJMNEJMhpr011493  

10. Niemeier M , Hamsen U , Yilmaz E , et al.  Critical incident reporting systems (CIRS) in trauma patients may identify common quality problems. Eur J Trauma Emerg Surg. 2019;•••: https://doi.org/10.1007/s00068-019-01128-y; Advance online publication.  

11. Findlay Ú , Best H , Ottrey M . Improving patient safety in radiotherapy through error reporting and analysis. Radiography. 2016;22(1):3–11. https://doi.org/10.1016/j.radi.2016.10.009  

12. Luebke J , Lang SJ , Reinhard T . Das Critical Incident Reporting System im Risikomanagement der Klinik für Augenheilkunde in Freiburg [Critical Incident Reporting System in Risk Management at the Eye Center in Freiburg]. Klin Monatsbl Augenheilkd. 2017 Jul;234(7):894–9. https://doi.org/10.1055/s-0043-110570  

13. Forster AJ , Fung I , Caughey S , Oppenheimer L , Beach C , Shojania KG , et al.  Adverse events detected by clinical surveillance on an obstetric service. Obstet Gynecol. 2006 Nov;108(5):1073–83. https://doi.org/10.1097/01.AOG.0000242565.28432.7c  

14. Aaronson EL , Brown D , Benzer T , Natsui S , Mort E . Incident Reporting in Emergency Medicine: A Thematic Analysis of Events. J Patient Saf. 2019 Dec;15(4):e60–3. https://doi.org/10.1097/PTS.0000000000000399  

15. Mueller SM , Mohr G , Navarini AA , Gantenbein L , Goldust M , Karagaiah P , et al.  Critical incidence reporting in dermatology: a cross-sectional study of 94 cases in a tertiary referral center. J Dermatolog Treat. 2020 Oct;  1–2. https://doi.org/10.1080/09546634.2020.1839005  

16. Staender S , Davies J , Helmreich B , Sexton B , Kaufmann M . The anaesthesia critical incident reporting system: an experience based database. Int J Med Inform. 1997 Nov;47(1-2):87–90. https://doi.org/10.1016/S1386-5056(97)00087-7  

17. National Patient Safety Agency NHS . (2008). A risk matrix for risk managers. https://www.neas.nhs.uk/media/118673/foi.16.170_-_risk_matrix_for_risk_managers_v91.pdf. (accessed 2020 June 23 2020) 

18. World Health Organization . (2009). Conceptual framework for the international classification for patient safety (ICPS); Version 1.1 Final Technical Report. http://www.who.int/patientsafety/taxonomy/icps_full_report.pdf. (accessed 2020 June 27) 

19. Neuhaus C , Holzschuh M , Lichtenstern C , St Pierre M . Erkenntnisse aus 10 Jahren CIRS‑AINS : Eine Analyse von Nutzerverhalten und Ausblick auf neue Herausforderungen [Findings from 10 years of CIRS-AINS: An analysis of usepatterns and insights into new challenges] [published online ahead of print, 2020 Aug 17]. Anaesthesist. 2020 Nov;69(11):793–802. https://doi.org/10.1007/s00101-020-00829-z  

20. Marra AR , Algwizani A , Alzunitan M , Brennan TM , Edmond MB . Descriptive Epidemiology of Safety Events at an Academic Medical Center. Int J Environ Res Public Health. 2020 Jan;17(1):353. https://doi.org/10.3390/ijerph17010353  

21. Burlison JD , Quillivan RR , Kath LM , Zhou Y , Courtney SC , Cheng C , et al.  A Multilevel Analysis of U.S. Hospital Patient Safety Culture Relationships With Perceptions of Voluntary Event Reporting. J Patient Saf. 2020 Sep;16(3):187–93. https://doi.org/10.1097/PTS.0000000000000336  

22. Sendlhofer G , Schweppe P , Sprincnik U , Gombotz V , Leitgeb K , Tiefenbacher P , et al.  Deployment of Critical Incident Reporting System (CIRS) in public Styrian hospitals: a five year perspective. BMC Health Serv Res. 2019 Jun;19(1):412. https://doi.org/10.1186/s12913-019-4265-0  

23. Hubertus J , Piehlmeier W , Heinrich M . Communicating the Improvements Developed from Critical Incident Reports is an Essential Part of CIRS. Klin Padiatr. 2016 Sep;228(5):270–4. https://doi.org/10.1055/s-0042-113311  

24. Bodina A , Demarchi A , Castaldi S. A web-based incident reporting system: a two years’ experience in an Italian research and teaching hospital. Annali di igieni 2014;26(3):219-25. 

25. Schwendimann R , Blatter C , Dhaini S , Simon M , Ausserhofer D . The occurrence, types, consequences and preventability of in-hospital adverse events - a scoping review. BMC Health Serv Res. 2018 Jul;18(1):521. https://doi.org/10.1186/s12913-018-3335-z  

26. Macrae C . The problem with incident reporting. BMJ Qual Saf. 2016 Feb;25(2):71–5. https://doi.org/10.1136/bmjqs-2015-004732  

27. Ahlberg EL , Elfström J , Borgstedt MR , Öhrn A , Andersson C , Sjödahl R , et al.  Learning From Incident Reporting? Analysis of Incidents Resulting in Patient Injuries in a Web-Based System in Swedish Health Care. J Patient Saf. 2020 Dec;16(4):264–8. https://doi.org/10.1097/PTS.0000000000000343  

Appendix

Figure S1 Risk assessment by calculating the product of likelihood and consequences.

Figure S2 Total numbers of reports per year and incident rates per 1’000 patient days per year.