Dilek Yapar1,2, Yasemin Demir Avcı2,3, Esra Tokur Sonuvar2, Ömer Faruk Eğerci4, Aliekber Yapar4

1Department of Public Health, Turkish Ministry of Health, Muratpaşa District Health Directorate, Antalya, Türkiye
2Institute of Health Science, Medical Informatics, Akdeniz University, Antalya, Türkiye
3Department of Public Health Nursing, Akdeniz University Faculty of Nursing, Türkiye
4Department of Orthopedics and Traumatology, Antalya Training and Research Hospital, Antalya, Türkiye

Keywords: ChatGPT, expert evaluation, large language models, medical consultation, orthopedic interventions, rubric.


Objectives: This study presents the first investigation into the potential of ChatGPT to provide medical consultation for patients undergoing orthopedic interventions, with the primary objective of evaluating ChatGPT’s effectiveness in supporting patient self-management during the essential early recovery phase at home.

Materials and methods: Seven scenarios, representative of common situations in orthopedics and traumatology, were presented to ChatGPT version 4.0 to obtain advice. These scenarios and ChatGPT̓s responses were then evaluated by 68 expert orthopedists (67 males, 1 female; mean age: 37.9±5.9 years; range, 30 to 59 years), 40 of whom had at least four years of orthopedic experience, while 28 were associate or full professors. Expert orthopedists used a rubric on a scale of 1 to 5 to evaluate ChatGPTʼs advice based on accuracy, applicability, comprehensiveness, and clarity. Those who gave ChatGPT a score of 4 or higher considered its performance as above average or excellent.

Results: In all scenarios, the median evaluation scores were at least 4 across accuracy, applicability, comprehensiveness, and communication. As for mean scores, accuracy was the highest-rated dimension at 4.2±0.8, while mean comprehensiveness was slightly lower at 3.9±0.8. Orthopedist characteristics, such as academic title and prior use of ChatGPT, did not influence their evaluation (all p>0.05). Across all scenarios, ChatGPT demonstrated an accuracy of 79.8%, with applicability at 75.2%, comprehensiveness at 70.6%, and a 75.6% rating for communication clarity.

Conclusion: This study emphasizes ChatGPT̓s strengths in accuracy and applicability for home care after orthopedic intervention but underscores a need for improved comprehensiveness. This focused evaluation not only sheds light on ChatGPT̓s potential in specialized medical advice but also suggests its potential to play a broader role in the advancement of public health.

Citation: Yapar D, Demir Avcı Y, Tokur Sonuvar E, Eğerci ÖF, Yapar A. ChatGPT’s potential to support home care for patients in the early period after orthopedic interventions and enhance public health. Jt Dis Relat Surg 2024;35(1):169-176. doi: 10.52312/ jdrs.2023.1402.

Ethics Committee Approval

The study protocol was approved by the Antalya Training and Research Hospital Clinical Research Ethics Committee (date: 08.06.2023, no: 8/11-2023). The study was conducted in accordance with the principles of the Declaration of Helsinki.

Author Contributions

Idea/concept, control/ supervision: D.Y., Y.D.A., E.T.S., A.Y.; Design, data collection and/or processing, literature review, references and fundings, materials: D.Y., Y.D.A., E.T.S., Ö.F.E., A.Y.; Analysis and/or interpretation, writing the article: D.Y., Y.D.A., E.T.S.; Critical review: Ö.F.E., A.Y.

Conflict of Interest

The authors declared no conflicts of interest with respect to the authorship and/or publication of this article.

Financial Disclosure

The authors received no financial support for the research and/or authorship of this article.

Data Sharing Statement

The detailed scenarios and ChatGPT responses can be shared with readers upon their request.