Skip to main navigation menu Skip to main content Skip to site footer
##common.pageHeaderLogo.altText##

Original article

Vol. 153 No. 7 (2023)

Tackling alert fatigue with a semi-automated clinical decision support system: quantitative evaluation and end-user survey

  • Hendrike Dahmke
  • Rico Fiumefreddo
  • Philipp Schuetz
  • Remo De Iaco
  • Claudia Zaugg
DOI
https://doi.org/10.57187/smw.2023.40082
Cite this as:
Swiss Med Wkly. 2023;153:40082
Published
07.07.2023

Summary

STUDY AIMS: Clinical decision support systems (CDSS) embedded in hospital electronic health records efficiently reduce medication errors, but there is a risk of low physician adherence due to alert fatigue. At the Cantonal Hospital Aarau, a CDSS is being developed that allows the highly accurate detection and correction of medication errors. The semi-automated CDSS sends its alerts either directly to the physician or to a clinical pharmacist for review first. Our aim was to evaluate the performance of the recently implemented CDSS in terms of acceptance rate and alert burden, as well as physicians’ satisfaction with the CDSS.

METHODS: All alerts generated by the clinical decision support systems between January and December 2021 were included in a retrospective quantitative evaluation. A team of clinical pharmacists performed a follow-up to determine whether the recommendation made by the CDSS was implemented by the physician. The acceptance rate was calculated including all alerts for which it was possible to determine an outcome. A web-based survey was conducted amongst physicians to assess their attitude towards the CDSS. The survey questions included overall satisfaction, helpfulness of individual algorithms, and perceived alert burden.

RESULTS: In 2021, a total of 10,556 alerts were generated, of which 619 triggered a direct notification to the physician and 2,231 notifications were send to the physician after evaluation by a clinical pharmacist. The acceptance rates were 89.8% and 68.4%, respectively, which translates as an overall acceptance rate of 72.4%. On average, clinical pharmacists received 17.2 alerts per day, while all of the hospital physicians together received 7.8 notifications per day. In the survey, 94.5% of physicians reported being satisfied or very satisfied with the CDSS. Algorithms addressing potential medication errors concerning anticoagulants received the highest usefulness ratings.

CONCLUSION: The development of this semi-automated clinical decision support system with context-based algorithms resulted in alerts with a high acceptance rate. Involving clinical pharmacists proved a promising approach to limit the alert burden of physicians and thus tackle alert fatigue. The CDSS is well accepted by our physicians.

References

  1. Aronson JK. Medication errors: definitions and classification. Br J Clin Pharmacol. 2009 Jun;67(6):599–604. 10.1111/j.1365-2125.2009.03415.x DOI: https://doi.org/10.1111/j.1365-2125.2009.03415.x
  2. Krähenbühl-Melcher A, Schlienger R, Lampert M, Haschke M, Drewe J, Krähenbühl S. Drug-related problems in hospitals: a review of the recent literature. Drug Saf. 2007;30(5):379–407. 10.2165/00002018-200730050-00003 DOI: https://doi.org/10.2165/00002018-200730050-00003
  3. Nanji KC, Patel A, Shaikh S, Seger DL, Bates DW. Evaluation of Perioperative Medication Errors and Adverse Drug Events. Anesthesiology. 2016 Jan;124(1):25–34. 10.1097/ALN.0000000000000904 DOI: https://doi.org/10.1097/ALN.0000000000000904
  4. Westbrook JI, Sunderland NS, Woods A, Raban MZ, Gates P, Li L. Changes in medication administration error rates associated with the introduction of electronic medication systems in hospitals: a multisite controlled before and after study. BMJ Health Care Inform. 2020 Aug;27(3):e100170. 10.1136/bmjhci-2020-100170 DOI: https://doi.org/10.1136/bmjhci-2020-100170
  5. Convertino I, Salvadori S, Pecori A, Galiulo MT, Ferraro S, Parrilli M, et al. Potential Direct Costs of Adverse Drug Events and Possible Cost Savings Achievable by their Prevention in Tuscany, Italy: A Model-Based Analysis. Drug Saf. 2019 03;42(3):427-44. DOI: https://doi.org/10.1007/s40264-018-0737-0
  6. Muylle KM, Gentens K, Dupont AG, Cornu P. Evaluation of an optimized context-aware clinical decision support system for drug-drug interaction screening. Int J Med Inform. 2021 Apr;148:104393. 10.1016/j.ijmedinf.2021.104393 DOI: https://doi.org/10.1016/j.ijmedinf.2021.104393
  7. Hardmeier B, Braunschweig S, Cavallaro M, Roos M, Pauli-Magnus C, Giger M, et al. Adverse drug events caused by medication errors in medical inpatients. Swiss Med Wkly. 2004 Nov;134(45-46):664–70. 10.4414/smw.2004.10801 DOI: https://doi.org/10.4414/smw.2004.10801
  8. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003 Jun;163(12):1409–16. 10.1001/archinte.163.12.1409 DOI: https://doi.org/10.1001/archinte.163.12.1409
  9. Sutton RT, Pincock D, Baumgart DC, Sadowski DC, Fedorak RN, Kroeker KI. An overview of clinical decision support systems: benefits, risks, and strategies for success. NPJ Digit Med. 2020 Feb;3(1):17. 10.1038/s41746-020-0221-y DOI: https://doi.org/10.1038/s41746-020-0221-y
  10. Cash JJ. Alert fatigue. Am J Health Syst Pharm. 2009 Dec;66(23):2098–101. 10.2146/ajhp090181 DOI: https://doi.org/10.2146/ajhp090181
  11. Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R, et al. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017 04;17(1):36. 10.1186/s12911-017-0430-8 DOI: https://doi.org/10.1186/s12911-017-0430-8
  12. Poly TN, Islam MM, Yang HC, Li YJ. Appropriateness of Overridden Alerts in Computerized Physician Order Entry: systematic Review. JMIR Med Inform. 2020 Jul;8(7):e15653. 10.2196/15653 DOI: https://doi.org/10.2196/15653
  13. Nanji KC, Seger DL, Slight SP, Amato MG, Beeler PE, Her QL, et al. Medication-related clinical decision support alert overrides in inpatients. J Am Med Inform Assoc. 2018 05 01;25(5):476-81. DOI: https://doi.org/10.1093/jamia/ocx115
  14. Blecker S, Pandya R, Stork S, Mann D, Kuperman G, Shelley D, et al. Interruptive Versus Noninterruptive Clinical Decision Support: usability Study. JMIR Human Factors. 2019 Apr;6(2):e12469. 10.2196/12469 DOI: https://doi.org/10.2196/12469
  15. Carroll AE. Averting Alert Fatigue to Prevent Adverse Drug Reactions. JAMA. 2019 Aug;322(7):601. 10.1001/jama.2019.11710 DOI: https://doi.org/10.1001/jama.2019.11710
  16. Chou E, Boyce RD, Balkan B, Subbian V, Romero A, Hansten PD, et al. Designing and evaluating contextualized drug-drug interaction algorithms. JAMIA Open. 2021 Mar;4(1):ooab023. 10.1093/jamiaopen/ooab023 DOI: https://doi.org/10.1093/jamiaopen/ooab023
  17. Chien SC, Chen YL, Chien CH, Chin YP, Yoon CH, Chen CY, et al. Alerts in Clinical Decision Support Systems (CDSS): A Bibliometric Review and Content Analysis. Healthcare (Basel). 2022 Mar;10(4):601. 10.3390/healthcare10040601 DOI: https://doi.org/10.3390/healthcare10040601
  18. Baysari MT, Zheng WY, Van Dort B, Reid-Anderson H, Gronski M, Kenny E. A Late Attempt to Involve End Users in the Design of Medication-Related Alerts: Survey Study. J Med Internet Res. 2020 03;22(3):e14855. DOI: https://doi.org/10.2196/14855
  19. Van De Sijpe G, Quintens C, Walgraeve K, Van Laer E, Penny J, De Vlieger G, et al. Overall performance of a drug-drug interaction clinical decision support system: quantitative evaluation and end-user survey. BMC Med Inform Decis Mak. 2022 02 22;22(1):48. 10.1186/s12911-022-01783-z DOI: https://doi.org/10.1186/s12911-022-01783-z
  20. Johnson M, Sanchez P, Langdon R, Manias E, Levett-Jones T, Weidemann G, et al. The impact of interruptions on medication errors in hospitals: an observational study of nurses. J Nurs Manag. 2017 Oct;25(7):498–507. 10.1111/jonm.12486 DOI: https://doi.org/10.1111/jonm.12486
  21. Harris CR, Millman KJ, van der Walt SJ, Gommers R, Virtanen P, Cournapeau D, et al. Array programming with NumPy. Nature. 2020 Sep;585(7825):357–62. 10.1038/s41586-020-2649-2 DOI: https://doi.org/10.1038/s41586-020-2649-2
  22. McKinney W. Data structures for statistical computing in python. Proceedings of the 9th Python in Science Conference; 2010: Austin, TX; 2010. p. 51-6. 10.25080/Majora-92bf1922-00a DOI: https://doi.org/10.25080/Majora-92bf1922-00a
  23. Hunter JD. Matplotlib: A 2D Graphics Environment. Comput Sci Eng. 2007;9(3):90–5. 10.1109/MCSE.2007.55 DOI: https://doi.org/10.1109/MCSE.2007.55
  24. Moja L, Polo Friz H, Capobussi M, Kwag K, Banzi R, Ruggiero F, et al. Effectiveness of a Hospital-Based Computerized Decision Support System on Clinician Recommendations and Patient Outcomes: A Randomized Clinical Trial. JAMA Netw Open. 2019 12;2(12):e1917094. 10.1001/jamanetworkopen.2019.17094 DOI: https://doi.org/10.1001/jamanetworkopen.2019.17094
  25. Wright A, Aaron S, Seger DL, Samal L, Schiff GD, Bates DW. Reduced Effectiveness of Interruptive Drug-Drug Interaction Alerts after Conversion to a Commercial Electronic Health Record. J Gen Intern Med. 2018 11;33(11):1868-76. DOI: https://doi.org/10.1007/s11606-018-4415-9
  26. Yoo J, Lee J, Rhee PL, Chang DK, Kang M, Choi JS, et al. Alert Override Patterns With a Medication Clinical Decision Support System in an Academic Emergency Department: Retrospective Descriptive Study. JMIR Med Inform. 2020 Nov;8(11):e23351. 10.2196/23351 DOI: https://doi.org/10.2196/23351
  27. Hussain MI, Reynolds TL, Zheng K. Medication safety alert fatigue may be reduced via interaction design and clinical role tailoring: a systematic review. J Am Med Inform Assoc. 2019 10;26(10):1141-9. DOI: https://doi.org/10.1093/jamia/ocz095
  28. Villa Zapata L, Subbian V, Boyce RD, Hansten PD, Horn JR, Gephart SM, et al. Overriding Drug-Drug Interaction Alerts in Clinical Decision Support Systems: A Scoping Review. Stud Health Technol Inform. 2022 Jun;290:380–4. 10.3233/SHTI220101 DOI: https://doi.org/10.3233/SHTI220101
  29. Skalafouris C, Reny JL, Stirnemann J, Grosgurin O, Eggimann F, Grauser D, et al. Development and assessment of PharmaCheck: an electronic screening tool for the prevention of twenty major adverse drug events. BMC Med Inform Decis Mak. 2022 May;22(1):146. 10.1186/s12911-022-01885-8 DOI: https://doi.org/10.1186/s12911-022-01885-8
  30. Gaskin J, Conyard E. Clinical pharmacist interventions in the emergency department and their impact on preventable adverse drug events and associated cost avoidance. Eur J Hosp Pharm Sci Pract. 2017;24:A81. DOI: https://doi.org/10.1136/ejhpharm-2017-000640.179
  31. Cohen V, Jellinek SP, Hatch A, Motov S. Effect of clinical pharmacists on care in the emergency department: a systematic review. Am J Health Syst Pharm. 2009 Aug;66(15):1353–61. 10.2146/ajhp080304 DOI: https://doi.org/10.2146/ajhp080304
  32. Drovandi A, Robertson K, Tucker M, Robinson N, Perks S, Kairuz T. A systematic review of clinical pharmacist interventions in paediatric hospital patients. Eur J Pediatr. 2018 Aug;177(8):1139–48. 10.1007/s00431-018-3187-x DOI: https://doi.org/10.1007/s00431-018-3187-x
  33. Richardson M, Dwyer CO, Gaskin J, Conyard E, Murphy K. The potential contribution of medicines to falls in older persons and the acceptance of pharmacist intervention. Int J Pharm Pract. 2020;28:57–8.
  34. Gallo T, Curry SC, Padilla-Jones A, Heise CW, Ramos KS, Woosley RL, et al. A computerized scoring system to improve assessment of heparin-induced thrombocytopenia risk. J Thromb Haemost. 2019 02;17(2):383-8. DOI: https://doi.org/10.1111/jth.14359
  35. Gallo T, Heise CW, Woosley RL, Tisdale JE, Antonescu CC, Gephart SM, et al. Clinician Satisfaction With Advanced Clinical Decision Support to Reduce the Risk of Torsades de Pointes. J Patient Saf. 2022 Sep;18(6):e1010–3. 10.1097/PTS.0000000000000996 DOI: https://doi.org/10.1097/PTS.0000000000000996
  36. Carli D, Fahrni G, Bonnabry P, Lovis C. Quality of Decision Support in Computerized Provider Order Entry: Systematic Literature Review. JMIR Med Inform. 2018 Jan;6(1):e3. 10.2196/medinform.7170 DOI: https://doi.org/10.2196/medinform.7170