Evaluation of artificial intelligence-generated patient education content for adolescent idiopathic scoliosis: A comparative study of reliability, educational value and readability
DIGITAL HEALTH, vol.12, 2026 (SCI-Expanded, SSCI, Scopus)
- Publication Type: Article / Article
- Volume: 12
- Publication Date: 2026
- Doi Number: 10.1177/20552076261499680
- Journal Name: DIGITAL HEALTH
- Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Social Sciences Citation Index (SSCI), Scopus, Directory of Open Access Journals, Health Research Premium Collection (ProQuest)
- Istanbul University Affiliated: Yes
Abstract
Objective To compare the reliability, educational value, readability, and exploratory clinical accuracy/safety of patient-education responses generated by four consumer AI chatbots for adolescent idiopathic scoliosis (AIS). Methods Ten AIS questions were submitted once, in separate conversations, to ChatGPT (GPT-5, free version), Gemini (Gemini 2.5 Flash, free version), Claude (Claude Sonnet, free version), and DeepSeek (DeepSeek-V3, free version) between 27 May and 10 June 2026. Three blinded orthopedic spine surgeons evaluated first responses using mDISCERN, PEMAT-Understandability, PEMAT-Actionability, and GQS; readability was assessed with five standard indices. Because all four models answered the same 10 questions, model comparisons were analyzed as question-matched data. An exploratory assessment evaluated factual inaccuracies, clinically important omissions, potentially harmful recommendations, and SRS/SOSORT concordance. Results Forty responses were analyzed. Question-matched comparisons showed significant differences among models for mDISCERN, PEMAT-U, PEMAT-A, and GQS (all Friedman p<0.001), with ChatGPT achieving the highest mean scores (4.03 +/- 0.51, 84.07 +/- 1.90, 83.07 +/- 2.07, and 4.33 +/- 0.35, respectively). Responses generally required a 10th-to 11th-grade reading level. Factual inaccuracies occurred in 5/40 responses (12.5%), clinically important omissions in 2/40 (5.0%), and potentially harmful recommendations in 2/40 (5.0%); matched exploratory comparisons were not statistically significant. All responses were at least partially SRS/SOSORT-concordant by majority classification. Conclusion AI chatbots differed substantially in expert-rated AIS educational quality, while exploratory accuracy/safety differences were not statistically significant. Readability remained above recommended patient-education levels. AI-generated information should supplement, not replace, clinician-guided counseling.