ChatGPT’s potential to support home care for patients in the early period after orthopedic interventions and enhance public health
Dilek Yapar1,2
, Yasemin Demir Avcı2,3, Esra Tokur Sonuvar2, Ömer Faruk Eğerci4, Aliekber Yapar4
1Department of Public Health, Turkish Ministry of Health, Muratpaşa District Health Directorate, Antalya, Türkiye
2Institute of Health Science, Medical Informatics, Akdeniz University, Antalya, Türkiye
3Department of Public Health Nursing, Akdeniz University Faculty of Nursing, Türkiye
4Department of Orthopedics and Traumatology, Antalya Training and Research Hospital, Antalya, Türkiye
Keywords: ChatGPT, expert evaluation, large language models, medical consultation, orthopedic interventions, rubric.
Abstract
Objectives: This study presents the first investigation into the potential of ChatGPT to provide medical consultation for patients undergoing orthopedic interventions, with the primary objective of evaluating ChatGPT’s effectiveness in supporting patient self-management during the essential early recovery phase at home.
Materials and methods: Seven scenarios, representative of common situations in orthopedics and traumatology, were presented to ChatGPT version 4.0 to obtain advice. These scenarios and ChatGPT̓s responses were then evaluated by 68 expert orthopedists (67 males, 1 female; mean age: 37.9±5.9 years; range, 30 to 59 years), 40 of whom had at least four years of orthopedic experience, while 28 were associate or full professors. Expert orthopedists used a rubric on a scale of 1 to 5 to evaluate ChatGPTʼs advice based on accuracy, applicability, comprehensiveness, and clarity. Those who gave ChatGPT a score of 4 or higher considered its performance as above average or excellent.
Results: In all scenarios, the median evaluation scores were at least 4 across accuracy, applicability, comprehensiveness, and communication. As for mean scores, accuracy was the highest-rated dimension at 4.2±0.8, while mean comprehensiveness was slightly lower at 3.9±0.8. Orthopedist characteristics, such as academic title and prior use of ChatGPT, did not influence their evaluation (all p>0.05). Across all scenarios, ChatGPT demonstrated an accuracy of 79.8%, with applicability at 75.2%, comprehensiveness at 70.6%, and a 75.6% rating for communication clarity.
Conclusion: This study emphasizes ChatGPT̓s strengths in accuracy and applicability for home care after orthopedic intervention but underscores a need for improved comprehensiveness. This focused evaluation not only sheds light on ChatGPT̓s potential in specialized medical advice but also suggests its potential to play a broader role in the advancement of public health.
Citation: Yapar D, Demir Avcı Y, Tokur Sonuvar E, Eğerci ÖF, Yapar A. ChatGPT’s potential to support home care for patients in the early period after orthopedic interventions and enhance public health. Jt Dis Relat Surg 2024;35(1):169-176. doi: 10.52312/ jdrs.2023.1402.
The study protocol was approved by the Antalya Training and Research Hospital Clinical Research Ethics Committee (date: 08.06.2023, no: 8/11-2023). The study was conducted in accordance with the principles of the Declaration of Helsinki.
Idea/concept, control/ supervision: D.Y., Y.D.A., E.T.S., A.Y.; Design, data collection and/or processing, literature review, references and fundings, materials: D.Y., Y.D.A., E.T.S., Ö.F.E., A.Y.; Analysis and/or interpretation, writing the article: D.Y., Y.D.A., E.T.S.; Critical review: Ö.F.E., A.Y.
The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.
The authors received no financial support for the research and/or authorship of this article.
The detailed scenarios and ChatGPT responses can be shared with readers upon their request.