Skip to main navigation menu Skip to main content Skip to site footer
##common.pageHeaderLogo.altText##

Review article: Biomedical intelligence

Vol. 154 No. 10 (2024)

Chatbots in medicine: certification process and applied use case

DOI
https://doi.org/10.57187/s.3954
Cite this as:
Swiss Med Wkly. 2024;154:3954
Published
25.10.2024

Summary

Chatbots are computer programs designed to engage in natural language conversations in an easy and understandable way. Their use has been accelerated recently with the advent of large language models. However, their application in medicine and healthcare has been limited due to concerns over data privacy, the risk of providing medical diagnoses, and ensuring regulatory and legal compliance. Medicine and healthcare could benefit from chatbots if their scope is carefully defined and if they are used appropriately and monitored long-term.

The confIAnce chatbot, developed at the Geneva University Hospitals and the University of Geneva, is an informational tool aimed at providing simplified information to the general public about primary care and chronic diseases. In this paper, we describe the certification and regulatory aspects applicable to chatbots in healthcare, particularly in primary care medicine. We use the confIAnce chatbot as a case study to explore the definition and classification of a medical device and its application to chatbots, considering the applicable Swiss regulations and the European Union AI Act.

Chatbots can be classified anywhere from non-medical devices (informational tools that do not handle patient data or provide recommendations for treatment or diagnosis) to Class III medical devices (high-risk tools capable of predicting potentially fatal events and enabling a pre-emptive medical intervention). Key considerations in the definition and certification process include defining the chatbot’s scope, ensuring compliance with regulations, maintaining security and safety, and continuously evaluating performance, risks, and utility. A lexicon of relevant terms related to artificial intelligence in healthcare, medical devices, and regulatory frameworks is also presented in this paper.

Chatbots hold potential for both patients and healthcare professionals, provided that their scope of practice is clearly defined, and that they comply with regulatory requirements. This review aims to provide transparency by outlining the steps required for certification and regulatory compliance, making it valuable for healthcare professionals, scientists, developers, and patients.

References

  1. Open AI. Introducing ChatGPT. 2022. https://openai.com/blog/chatgpt
  2. Holmgren AJ, Downing NL, Tang M, Sharp C, Longhurst C, Huckman RS. Assessing the impact of the COVID-19 pandemic on clinician ambulatory electronic health record use [Erratum in: J Am Med Inform Assoc. 2022 Jan 10; PMID: 34888680; PMCID: PMC8689796]. J Am Med Inform Assoc. 2022 Jan;29(3):453–60. 10.1093/jamia/ocab268 DOI: https://doi.org/10.1093/jamia/ocab268
  3. Tai-Seale M, Dillon EC, Yang Y, Nordgren R, Steinberg RL, Nauenberg T, et al. Physicians’ Well-Being Linked To In-Basket Messages Generated By Algorithms In Electronic Health Records. Health Aff (Millwood). 2019 Jul;38(7):1073–8. 10.1377/hlthaff.2018.05509 DOI: https://doi.org/10.1377/hlthaff.2018.05509
  4. Sinsky CA, Shanafelt TD, Ripp JA. The Electronic Health Record Inbox: recommendations for Relief. J Gen Intern Med. 2022 Nov;37(15):4002–3. 10.1007/s11606-022-07766-0 DOI: https://doi.org/10.1007/s11606-022-07766-0
  5. Haque MD, Rubya S. An Overview of Chatbot-Based Mobile Mental Health Apps: Insights From App Description and User Reviews. JMIR Mhealth Uhealth. 2023 May;11:e44838. 10.2196/44838 DOI: https://doi.org/10.2196/44838
  6. Hauser-Ulrich S, Künzli H, Meier-Peterhans D, Kowatsch T. A Smartphone-Based Health Care Chatbot to Promote Self-Management of Chronic Pain (SELMA): Pilot Randomized Controlled Trial. JMIR Mhealth Uhealth. 2020 Apr;8(4):e15806. 10.2196/15806 DOI: https://doi.org/10.2196/15806
  7. Bin Sawad A, Narayan B, Alnefaie A, Maqbool A, Mckie I, Smith J, et al. A Systematic Review on Healthcare Artificial Intelligent Conversational Agents for Chronic Conditions. Sensors (Basel). 2022 Mar;22(7):2625. doi: https://doi.org/10.3390/s22072625 DOI: https://doi.org/10.3390/s22072625
  8. Xu L, Sanders L, Li K, Chow JC. Chatbot for Health Care and Oncology Applications Using Artificial Intelligence and Machine Learning: systematic Review. JMIR Cancer. 2021 Nov;7(4):e27850. 10.2196/27850 DOI: https://doi.org/10.2196/27850
  9. Lee N et al. Developing a Chatbot–Clinician Model for Hypertension Management. NEJM Catal Innov Care Deliv 2022;3(11) DOI: 10.1056/CAT.22.0228 VOL. 3 NO. 11. DOI: https://doi.org/10.1056/CAT.22.0228
  10. Ter Stal S, Sloots J, Ramlal A, Op den Akker H, Lenferink A, Tabak M. An Embodied Conversational Agent in an eHealth Self-management Intervention for Chronic Obstructive Pulmonary Disease and Chronic Heart Failure: Exploratory Study in a Real-life Setting. JMIR Hum Factors. 2021 Nov;8(4):e24110. 10.2196/24110 DOI: https://doi.org/10.2196/24110
  11. Vaddadi G, et al. A Pilot Program Utilising a “Chatbot “ to Support Patients in the First 30 Days Following an Admission to Hospital With Acute Decompensated Heart Failure. Heart Lung and Circulation. July 2023. VOLUME 32, SUPPLEMENT 3, S139. DOI: https://doi.org/10.1016/j.hlc.2023.06.039
  12. Griffin AC, Xing Z, Khairat S, Wang Y, Bailey S, Arguello J, et al. Conversational Agents for Chronic Disease Self-Management: A Systematic Review. AMIA Annu Symp Proc. 2021 Jan;2020:504–13. DOI: https://doi.org/10.2196/preprints.14989
  13. Blasco JM, Díaz-Díaz B, Igual-Camacho C, Pérez-Maletzki J, Hernández-Guilén D, Roig-Casasús S. Effectiveness of using a chatbot to promote adherence to home physiotherapy after total knee replacement, rationale and design of a randomized clinical trial. BMC Musculoskelet Disord. 2023 Jun;24(1):491. 10.1186/s12891-023-06607-3 DOI: https://doi.org/10.1186/s12891-023-06607-3
  14. Schachner T, Keller R, V Wangenheim F. Artificial Intelligence-Based Conversational Agents for Chronic Conditions: Systematic Literature Review. J Med Internet Res. 2020 Sep;22(9):e20701. 10.2196/20701 DOI: https://doi.org/10.2196/20701
  15. Kurniawan MH, Handiyani H, Nuraini T, Hariyati RT, Sutrisno S. A systematic review of artificial intelligence-powered (AI-powered) chatbot intervention for managing chronic illness. Ann Med. 2024 Dec;56(1):2302980. doi: https://doi.org/10.1080/07853890.2024.2302980 DOI: https://doi.org/10.1080/07853890.2024.2302980
  16. Greer S, Ramo D, Chang YJ, Fu M, Moskowitz J, Haritatos J. Use of the Chatbot “Vivibot” to Deliver Positive Psychology Skills and Promote Well-Being Among Young People After Cancer Treatment: Randomized Controlled Feasibility Trial. JMIR Mhealth Uhealth. 2019 Oct;7(10):e15018. doi: https://doi.org/10.2196/15018 DOI: https://doi.org/10.2196/15018
  17. Thirunavukarasu AJ, Ting DS, Elangovan K, Gutierrez L, Tan TF, Ting DS. Large language models in medicine. Nat Med. 2023 Aug;29(8):1930–40. 10.1038/s41591-023-02448-8 DOI: https://doi.org/10.1038/s41591-023-02448-8
  18. Meskó B, Topol EJ. The imperative for regulatory oversight of large language models (or generative AI) in healthcare. NPJ Digit Med. 2023 Jul;6(1):120. 10.1038/s41746-023-00873-0 DOI: https://doi.org/10.1038/s41746-023-00873-0
  19. Webster P. Medical AI chatbots: are they safe to talk to patients? Nat Med. 2023 Nov;29(11):2677–9. 10.1038/s41591-023-02535-w DOI: https://doi.org/10.1038/s41591-023-02535-w
  20. United States Food and Drug Administration. Clinical Decision Support Software. Guidance for Industry and Food and Drug Administration Staff. September 2022. https://www.fda.gov/media/109618/download
  21. Regulation (EU) 2017/745 of the European parliament and of the council https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32017R0745
  22. Swiss Confederation - Federal Office of Public Health - Medical Devices Legislation https://www.bag.admin.ch/bag/en/home/medizin-und-forschung/heilmittel/revision-med-prod-verord-mepv.html#:~:text=The%20EU%20Medical%20Devices%20Regulation,with%20developments%20in%20European%20law
  23. Food and Drug Administration Regulations https://www.fda.gov/regulatory-information/fda-rules-and-regulations
  24. Fink M, Akra B. Comparison of the international regulations for medical devices-USA versus Europe. Injury. 2023 Oct;54 Suppl 5:110908. 10.1016/j.injury.2023.110908 DOI: https://doi.org/10.1016/j.injury.2023.110908
  25. European Commission. Notified Bodies https://health.ec.europa.eu/medical-devices-topics-interest/notified-bodies_en
  26. Nehme M, Schneider F, Perrin A, Sum Yu W, Schmitt S, Violot G, et al. The Development of a Chatbot Technology to Disseminate Post-COVID-19 Information: Descriptive Implementation Study. J Med Internet Res. 2023 Jun;25:e43113. 10.2196/43113 DOI: https://doi.org/10.2196/43113
  27. Division of Primary Care Medicine, Geneva University Hospitals. Strategies for Primary Care. https://www.hug.ch/medecine-premier-recours/strategies-medecine-premier-recours
  28. European Union. Regulation (EU) 2017/746 of the European Parliament and of the Council of 5 April 2017 on in vitro diagnostic medical devices and repealing Directive 98/79/EC and Commission Decision 2010/227/EU.
  29. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). https://eur-lex.europa.eu/eli/reg/2016/679/oj
  30. ISO/IEC 27001:2022 Information security, cybersecurity and privacy protection. Information security management systems Requirements. https://www.iso.org/standard/27001
  31. European Parliament. Artificial Intelligence. https://www.europarl.europa.eu/topics/en/topic/artificial-intelligence
  32. International Business Machines Corporation. IBM. What is Machine Learning (ML)? https://www.ibm.com/topics/machine-learning#:~:text=Machine%20learning%20(ML)%20is%20a,learn%2C%20gradually%20improving%20its%20accuracy. [Last accessed July 19, 2024].
  33. International Monetary Fund. AI Lexicon. December 2023. Finance and Development Magazine. https://www.imf.org/en/Publications/fandd/issues/2023/12/AI-Lexicon [last accessed July 19, 2024].
  34. International Business Machines Corporation. IBM. What is fine tuning? https://www.ibm.com/topics/fine-tuning#:~:text=Fine%2Dtuning%20in%20machine%20learning,models%20used%20for%20generative%20AI. [Last accessed July 19, 2024].
  35. Medical Device Coordination Group Document. MDCG 2019-11 Guidance on Qualification and Classification of Software in Regulation (EU) 2017/745 – MDR and Regulation (EU) 2017/746 – IVDR. October 2019.

Most read articles by the same author(s)

<< < 1 2