Can artificial intelligence solve school-level physics exam questions? A decade-based evaluation using Turkish university entrance exams
Physics Education, vol.61, no.5, 2026 (Scopus)
- Publication Type: Article / Article
- Volume: 61 Issue: 5
- Publication Date: 2026
- Doi Number: 10.1088/1361-6552/ae9bf4
- Journal Name: Physics Education
- Journal Indexes: Scopus, Aerospace Database, Agricultural & Environmental Science Database, EBSCO Education Source, Educational research abstracts (ERA), INSPEC, DIALNET, Academic Search Ultimate (EBSCO), Social Science Premium Collection (ProQuest), Education Collection (ProQuest), Education Source Ultimate (EBSCO), Materials Science & Engineering Collection (ProQuest), Technology Collection (ProQuest)
- Keywords: artificial intelligence, ChatGPT, DeepSeek, Gemini, large language models, physics education, university entrance examination
- Open Archive Collection: AVESIS Open Access Collection
- Istanbul University Affiliated: Yes
Abstract
Background. Large language models LLMs are increasingly used by students as informal learning tools; however, their reliability in solving school-level physics questions remains uncertain. Aim. This study evaluated the performance of three LLMs—ChatGPT-5, Gemini 2.5 Flash, and DeepSeek V3.1—in solving physics questions from Turkish university entrance examinations administered between 2016 and 2025, comprising the Yükseköğretime Geçiş Sınavı (YGS; Transition to Higher Education Examination) in 2016–2017 and the Temel Yeterlilik Testi (TYT; Basic Proficiency Test) in 2018–2025. Methods. A total of 84 publicly available physics questions—28 from the 2016–2017 YGS examinations and 56 from the 2018–2025 TYT examinations—were presented to each model under identical testing conditions using the same neutral student-style prompt, without revealing the examination source. Responses were scored for final-answer correctness using official answer keys. Correct-answer accuracyand net scores were calculated, and model performance was compared using Cochran’s Q test. Results. DeepSeek V3.1 achieved the highest numerical correct-answer accuracy at 78.6% (95%CI 68.7%–86.0%), followed by Gemini 2.5 Flash at 69.0% (95% CI 58.5%–77.9%) and ChatGPT-5 at 65.5% (95% CI 54.8%–74.8%). However, the difference in correct-answer performance among the three models was not statistically significant (Cochran’s Q = 4.732, df = 2, p = 0.094). Conclusion. LLMs may support physics learning and exam preparation, but their responses should be used critically and under teacher guidance rather than as substitutes for structured physics instruction.