Journal archive

Abstract: The rapid advancement of generative artificial intelligence has created an urgent need to reform both the traditional assessment methods and the assignments themselves. In literature Olympiads, test assignments have traditionally served as a “zone of objectivity” – that of forced-choice answers. However, if participants gain unauthorized but technologically feasible access to the Internet, these tests lose their validity. Modern neural networks can instantaneously retrieve answers to any factual questions, placing honest participants at a distinct disadvantage. The main aim of this study is to evaluate the robustness of literature Olympiad assignments against AI algorithms. The author seeks to identify specific types of tasks that remain “invulnerable” to neural networks and can, therefore, maintain the competitive integrity of intellectual contests. The article details a series of experiments conducted using popular and accessible neural networks, including ChatGPT, Gemini, Grok, DeepSeek, and Claude (utilizing free versions only). The practical research material comprised test items from the regional stage and oral round assignments from the final stages of the All-Russian Olympiad for School Students of recent years. Various formats were tested: identifying a literary work based on three facts, analyzing fictional instant-messenger dialogues between characters, recognizing literary situations (e.g., a chess match), and solving problems based on visual metaphors. The results indicate that assignments containing unique markers – such as proper nouns, rare terms, toponyms, or occasionalisms – are solved by neural networks without error. Conversely, tasks devoid of verbal components proved to be the most resilient. The use of visual sequences constructed on metaphorical or associative principles (for instance, linking the images of a billiard table, galoshes, and an axe to The Cherry Orchard) makes the task extremely difficult for artificial intelligence. Nevertheless, in certain instances, the Gemini neural network demonstrated an unexpected capacity for deep metaphorical analysis of complex images (e.g., connecting Picasso’s Guernica to The Tale of Igor’s Campaign), showing a level of comprehension comparable to the highest-performing students. Conclusions: incorporating images, schemes, charts, and diagrams into literature Olympiad assignments as a basis for analytical work with a fiction text allows for the actualization of complex intra-textual connections and the interaction of structural elements. Such an approach renders even short-answer test items “opaque” to artificial intelligence. A one- or two-word answer should be the result of rigorous cognitive processing, requiring not merely the recall of facts, but the ability to correlate, evaluate, and interpret them.

About the author(s) :

Tatiana G. Kuchina

Yaroslavl State Pedagogical University
named after K. D. Ushinsky (Yaroslavl, Russian Federation)

ORCID ID: http://orcid.org/0000-0002-1837-8429

Publication Timeline:

Date of receipt: 11.05.2026; date of publication: 31.10.2026

References:

Anderson, L. W., Krathwohl, D. R. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Boston, MA: Pearson Education Group, 352 p. ISBN 9780801319037.

Arkhipov, P. A., Desyaeva, N. D. (2024). Iskusstvennyy intellekt v obrazovanii – konkurent ili pomoshchnik uchitelya? = Artificial intelligence in education – a competitor or an assistant to a teacher? Hominum, 1(13), 16–33. EDN TJXQJL.

Bhayana, R., Krishna, S., Bleakney, R. R. (2023). Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations. Radiology, 307(5). DOI: 10.1148/radiol.230582. EDN ATPNVQ.

Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals. New York: Longmans.

Huang, D., Hash, N., Cummings, Ja. J., Prena, K. (2025). Academic cheating with generative AI: Exploring a moral extension of the theory of planned behavior. Computers and Education: Artificial Intelligence, 8, 100424. DOI: 10.1016/j.caeai.2025.100424. EDN NZIOUZ.

Kasneci, E., Sessler, K., Küchemann, S. et al. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, 102274. DOI: 10.1016/j.lindif.2023.102274. EDN DTFXQN.

Kofinas, A. K., Tsay, C. H. H., Pike, D. (2025). The impact of generative AI on academic integrity of authentic assessments within a higher education context. British Journal of Educational Technology, 56(6), 2522–2549. DOI: 10.1111/bjet.13585. EDN MAXZUP.

Luong, T., Lockhart, E. Advanced version of Gemini with Deep Think officially achieves gold-medal standard at the International Mathematical Olympiad. Available at May 12, 2026 from https://deepmind.google/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/.

Scarfe, P., Watcham, K., Clarke, A., Roesch, E. (2024). A real-world test of artificial intelligence infiltration of a university examinations system: A “Turing Test” case study. PLoS ONE, 19(6), e0305354. DOI: 10.1371/journal.pone.0305354. EDN ADZATF.

Surahman, E., Wang, T. H. (2022). Academic dishonesty and trustworthy assessment in online learning: A systematic literature review. Journal of Computer Assisted Learning, 38(6), 1535–1553. DOI: 10.1111/jcal.12708. EDN MYYYZU.

Susnjak, T., Mcintosh, T. (2024). ChatGPT: The End of Online Exam Integrity? Education Sciences, 14(6), 656. DOI: 10.3390/educsci14060656. EDN RKEZPA.