Can artificial intelligence solve school-level physics exam questions? A decade-based evaluation using Turkish university entrance exams
Physics Education, cilt.61, sa.5, 2026 (Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 61 Sayı: 5
- Basım Tarihi: 2026
- Doi Numarası: 10.1088/1361-6552/ae9bf4
- Dergi Adı: Physics Education
- Derginin Tarandığı İndeksler: Scopus, Aerospace Database, Agricultural & Environmental Science Database, EBSCO Education Source, Educational research abstracts (ERA), INSPEC, DIALNET, Academic Search Ultimate (EBSCO), Social Science Premium Collection (ProQuest), Education Collection (ProQuest), Education Source Ultimate (EBSCO), Materials Science & Engineering Collection (ProQuest), Technology Collection (ProQuest)
- Anahtar Kelimeler: artificial intelligence, ChatGPT, DeepSeek, Gemini, large language models, physics education, university entrance examination
- Açık Arşiv Koleksiyonu: AVESİS Açık Erişim Koleksiyonu
- İstanbul Üniversitesi Adresli: Evet
Özet
Background. Large language models LLMs are increasingly used by students as informal learning tools; however, their reliability in solving school-level physics questions remains uncertain. Aim. This study evaluated the performance of three LLMs—ChatGPT-5, Gemini 2.5 Flash, and DeepSeek V3.1—in solving physics questions from Turkish university entrance examinations administered between 2016 and 2025, comprising the Yükseköğretime Geçiş Sınavı (YGS; Transition to Higher Education Examination) in 2016–2017 and the Temel Yeterlilik Testi (TYT; Basic Proficiency Test) in 2018–2025. Methods. A total of 84 publicly available physics questions—28 from the 2016–2017 YGS examinations and 56 from the 2018–2025 TYT examinations—were presented to each model under identical testing conditions using the same neutral student-style prompt, without revealing the examination source. Responses were scored for final-answer correctness using official answer keys. Correct-answer accuracyand net scores were calculated, and model performance was compared using Cochran’s Q test. Results. DeepSeek V3.1 achieved the highest numerical correct-answer accuracy at 78.6% (95%CI 68.7%–86.0%), followed by Gemini 2.5 Flash at 69.0% (95% CI 58.5%–77.9%) and ChatGPT-5 at 65.5% (95% CI 54.8%–74.8%). However, the difference in correct-answer performance among the three models was not statistically significant (Cochran’s Q = 4.732, df = 2, p = 0.094). Conclusion. LLMs may support physics learning and exam preparation, but their responses should be used critically and under teacher guidance rather than as substitutes for structured physics instruction.