ChatGPT in hospital infection prevention and control – assessing knowledge of an AI model based on a validated questionnaire
DOI:
https://doi.org/10.15584/ejcem.2024.2.19Keywords:
artificial intelligence, ChatGPT, infection control, large language model, medical educationAbstract
Introduction and aim. An AI model like ChatGPT is a good source of knowledge. We can explore the potential of AI models to complement the expertise of healthcare professionals by providing real-time, evidence-based information in infection prevention and control (IPC).
Material and methods. This study involved 110 queries related to IPC, validated by subject experts in IPC. The responses from ChatGPT were evaluated using Bloom’s taxonomy by experienced microbiologists. The scores were divided as 4 as a good response. Statistical analysis was done by correlation coefficient and Cohen’s Kappa.
Results. The overall score was 4.33 (95% CI, q1 3.65–q3 4.64) indicating ChatGPT’s substantial IPC knowledge. A good response (i.e.>4 score) was found in 70 (63.6%) questions, while in 10 (9%) questions, it showed a poor response. The poor response was seen in needle stick injury and personal protective equipment (PPE) doffing-related questions. The overall correlations were found to be significant. Cohen’s Kappa confirmed moderate to substantial agreement between evaluators.
Conclusion. ChatGPT demonstrated a commendable understanding of IPC principles in various domains and the study identifies specific instances where the model may require further refinement especially in critical scenarios such as needlestick injuries and PPE doffing.
Downloads
References
Alhumaid S, Al Mutair A, Al Alawi Z, et al. Knowledge of infection prevention and control among healthcare workers and factors influencing compliance: a systematic review. Antimicrob Resist Infect Control. 2021;10(1):86. doi: 10.1186/s13756-021-00957-0
Sallam M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare (Basel). 2023;11(6):887. doi: 10.3390/healthcare11060887
Sinha RK, Deb Roy A, Kumar N, Mondal H. Applicability of ChatGPT in Assisting to Solve Higher Order Problems in Pathology. Cureus. 2023;15(2):35237. doi:10.7759/cureus.35237
Lahat A, Shachar E, Avidan B, Shatz Z, Glicksberg BS, Klang E. Evaluating the use of large language model in identifying top research questions in gastroenterology. Sci Rep. 2023;13 (1):4164. doi: 10.1038/s41598-023-31412-2
Ponsford MJ, Ward TJC, Stoneham SM, et al. A Systematic Review and Meta-Analysis of Inpatient Mortality Associated with Nosocomial and Community COVID-19 Exposes the Vulnerability of Immunosuppressed Adults. Front Immunol. 2021;12:744696. doi: 10.3389/fimmu.2021.744696
Fitzpatrick F, Doherty A, Lacey G. Using Artificial Intelligence in Infection Prevention. Curr Treat Options Infect Dis. 2020;12(2):135-144. doi: 10.1007/s40506-020-00216-7
Ghosh A, Bir A. Evaluating ChatGPT's Ability to Solve Higher-Order Questions on the Competency-Based Medical Education Curriculum in Medical Biochemistry. Cureus. 2023;15(4):37023. doi: 10.7759/cureus.37023
Khosravi P, Kazemi E, Imielinski M, Elemento O, Hajirasouliha I. Deep Convolutional Neural Networks Enable Discrimination of Heterogeneous Digital Pathology Images. EBioMedicine. 2018;27:317-328. doi: 10.1016/j.ebiom.2017.12.026
Abràmoff MD, Lou Y, Erginay A, et al. Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through Integration of Deep Learning. Invest Ophthalmol Vis Sci. 2016;57(13):5200-5206. doi: 10.1167/iovs.16-19964
Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115-118. doi: 10.1038/nature21056
Classen DC, Longhurst C, Thomas EJ. Bending the patient safety curve: how much can AI help? NPJ Digit Med. 2023;6(1):2. doi: 10.1038/s41746-022-00731-5
Reverberi C, Rigon T, Solari A, et al. Experimental evidence of effective human-AI collaboration in medical decision-making. Sci Rep. 2022;12(1):14952. doi: 10.1038/s41598-022-18751-2
Khanna NN, Maindarkar MA, Viswanathan V, et al. Economics of Artificial Intelligence in Healthcare: Diagnosis vs. Treatment. Healthcare (Basel). 2022;10(12):2493. doi: 10.3390/healthcare10122493
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 European Journal of Clinical and Experimental Medicine
This work is licensed under a Creative Commons Attribution 4.0 International License.
Our open access policy is in accordance with the Budapest Open Access Initiative (BOAI) definition: this means that articles have free availability on the public Internet, permitting any users to read, download, copy, distribute, print, search, or link to the full texts of these articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose, without financial, legal, or technical barriers other than those inseparable from having access to the Internet itself.
All articles are published with free open access under the CC-BY Creative Commons attribution license (the current version is CC-BY, version 4.0). If you submit your paper for publication by the Eur J Clin Exp Med, you agree to have the CC-BY license applied to your work. Under this Open Access license, you, as the author, agree that anyone may download and read the paper for free. In addition, the article may be reused and quoted provided that the original published version is cited. This facilitates freedom in re-use and also ensures that Eur J Clin Exp Med content can be mined without barriers for the research needs.
