Factors Influencing Competency Scores of Medical Students in Researching Emergency Care Principles Using ChatGPT
Abstract
Factors Influencing Competency Scores of Medical Students in Researching Emergency Care Principles Using ChatGPT
at Yala Hospital, Medical Education Center
Sathaya Buri MD.
1Department of Emergency medicine, Yala hospital
2Medical Education Center, Yala hospital
Background: Artificial intelligence (AI), particularly ChatGPT, offers significant potential for enhancing medical education and clinical decision-making, especially in high-pressure environments like emergency departments. This study investigates the factors influencing medical students' competency and satisfaction when using ChatGPT to research emergency care principles.
Methods: A descriptive-analytical study was conducted with 55 clinical-year medical students (years 4–6) at Yala Hospital’s Medical Education Center. Participants used ChatGPT to solve clinical scenarios that varied in information detail. Competency was assessed based on the accuracy of their search results. Data were analyzed using descriptive statistics, independent t-tests, and ANOVA.
Results: The study found that demographic factors—including sex, academic year, GPA, and prior AI experience—had no significant impact on students' competency scores. However, the quality and quantity of input data in the clinical scenarios significantly influenced performance; scenarios with sufficient data volume yielded significantly higher scores compared to data-limited scenarios (p < 0.001). Regarding satisfaction, students with higher GPAs (3.50–4.00) reported significantly greater satisfaction with ChatGPT’s utility for clinical decision support and medical education compared to other groups (p < 0.05).
Conclusion: ChatGPT serves as an effective supplementary tool for medical education, democratizing access to information by allowing students with varying backgrounds to achieve comparable search outcomes. However, its effectiveness relies heavily on the sufficiency of input data rather than individual learner traits. Therefore, medical curricula should integrate training on effective prompt engineering to maximize AI's utility in clinical practice.
Key words: ChatGPT, AI, Medical education, Decision making, Problem solving
References
เอกสารอ้างอิง
Kung TH, Cheatham M, Medenilla A, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2(2):e0000198. Available from: https://doi.org/10.1371/journal.pdig.0000198
Gilson A, Safranek CW, Huang T, et al. How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications for Medical Education. JMIR Med Educ. 2023;9:e45312. Available from: https://doi.org/10.2196/45312
Strong E, DiGiammarino A, Weng Y, et al. Chatbot vs Medical Student Performance on Free-Response Clinical Reasoning Examinations. JAMA Intern Med. 2023;183(11):1228-1230. Available from: https://doi.org/10.1001/jamainternmed.2023.2909
Hosseini M, Gao CA, Liebovitz DM, et al. An Exploratory Study on the Performance of ChatGPT in Answering Clinical Questions: The Importance of Prompt Engineering. J Med Syst. 2023;47(1):116. Available from: https://doi.org/10.1007/s10916-023-01968-3
Thawillarp S. Generative AI, ChatGPT and Medicine. PSU Med J. 2023;3(2):63-67. Available from: https://he01.tci-thaijo.org/index.php/PSUMJ/article/view/259047
Rao A, Pang M, Kim J, et al. Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study. J Med Internet Res. 2023;25:e48659. Available from: https://doi.org/10.2196/48659
Sallam M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare (Basel). 2023;11(6):887. Available from: https://doi.org/10.3390/healthcare11060887
Khorsand A, Khosravi B. Diagnostic Accuracy of ChatGPT for Patients' Triage: A Systematic Review and Meta-Analysis. Arch Acad Emerg Med. 2024;12(1):e15. Available from: https://doi.org/10.22037/aaem.v12i1.2384
Dahmen J, Mewes P, Berres M, et al. Artificial Intelligence in Medical Education: The Student Perspective. Med Educ Online. 2023;28(1):2239729. Available from: https://doi.org/10.1080/10872981.2023.2239729
Masters K. Ethical use of Artificial Intelligence in Health Professions Education: AMEE Guide No. 158. Med Teach. 2023;45(6):574-584. Available from: https://doi.org/10.1080/0142159X.2023.2186203
Sit C, Srinivasan R, Amlani A, et al. Attitudes and perceptions of UK medical students towards artificial intelligence and radiology: a multicentre survey. Insights Imaging. 2020;11(1):14. Available from: https://doi.org/10.1186/s13244-020-00830-7
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Medical and Health Professions Education
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Articles published in the Journal of Medical Education for Medical Personnel (Medical and Public Health Professional Education) are copyrighted by the Office of Promotion, Support, Production, and Development of Physicians and Medical Personnel.
