RU
Philology. Theory & Practice
  • Original research article
  • August 7, 2026
  • Open access

Evaluation of automatic simplification of German-language literary texts using large language models

Full
text RU

Abstract

The aim of this study is to investigate the applicability of large language models for the simplification of German-language literary texts. The paper provides a brief overview of existing approaches to this task, as well as statistical metrics used to assess text complexity and readability. Two types of statistical analysis (analysis of variance and correlation analysis) were applied to the values of a range of quantitative indicators obtained from original and simplified texts. These methods were used to test the hypothesis concerning the existence of differences between different groups of texts, as well as the relationships between the analyzed parameters. The scientific novelty of the study lies in the first comprehensive evaluation of the generated simplified texts based on thirteen quantitative indicators. The study also demonstrates that, in the process of simplifying German-language literary texts with large language models, changes in lexical density do not exhibit statistically significant differences. The results of the analysis show that texts simplified using the Llama 3 and MarianMT models are characterized by vocabulary corresponding to lower levels of the CEFR scale. Furthermore, the Llama 3 model significantly reduces text length during the simplification process.

Research materials

  1. Duden. Das Wörterbuch der Synonyme: 100.000 Synonyme für Alltag und Beruf / Dudenredaktion. 3. Aufl. Berlin: Dudenverlag, 2016. https://www.duden.de/woerterbuch/synonyme
  2. lexicalrichness: A small Python module to compute textual lexical richness // PyPI: сайт. https://pypi.org/project/lexicalrichness/
  3. spaCy: Industrial-strength Natural Language Processing in Python: официальный сайт. https://spacy.io/

References

  1. Большакова С. А. Система автоматической адаптации русскоязычных текстов и ее практическая значимость // Проблемы искусственного интеллекта. 2024. № 3. https://doi.org/10.24412/2413-7383-2024-3-45-54
  2. Захарова Е. Ю., Савина О. Ю. Лексическое разнообразие текста и способы его измерения // Вестник Тюменского государственного университета. Гуманитарные исследования. Humanitates. 2020. Т. 6. № 1 (21). https://doi.org/10.21684/2411-197X-2020-6-1-20-34
  3. Оборнева И. В. Автоматизация оценки качества восприятия текста // Вестник Московского городского педагогического университета. Серия: Информатика и информатизация образования. 2005. № 5.
  4. Солнышкина М. И., Андреева М. И. Оценка сложности учебного текста по русскому как иностранному: перспективы и вызовы использования искусственного интеллекта // Русистика. 2026. Т. 24. № 1. https://doi.org/10.22363/2618-8163-2026-24-1-120-137
  5. Солнышкина М. И., Андреева М. И., Гайнутдинова Д. З. Оценка сложности текстов по русскому как иностранному: инструментарий и алгоритм на основе больших языковых моделей // Актуальные проблемы филологии и педагогической лингвистики. 2026. № 2. https://doi.org/10.29025/2079-6021-2026-2-53-66
  6. Дмитриева А. А., Лапошина А. Н., Лебедева М. Ю. Количественное исследование стратегий упрощения в адаптированных текстах для изучающих русский язык на уровне L2 // Компьютерная лингвистика и интеллектуальные технологии: по материалам ежегодной международной конференции «Диалог» (2021), Москва, 16-19 июня 2021 года. М.: Российский государственный гуманитарный университет, 2021. Вып. 20.
  7. Alva-Manchego F., Martin L., Bordes A., Scarton C. ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. http://doi.org/10.18653/v1/2020.acl-main.424
  8. Alva-Manchego F., Scarton C., Specia L. The (Un)Suitability of Automatic Evaluation Metrics for Text Simplification // Computational Linguistics. 2021. Vol. 47. № 4. https://doi.org/10.1162/coli_a_00418
  9. Amstad T. Wie verständlich sind unsere Zeitungen?: Diss. Zürich: Universität Zürich, 1978.
  10. Björnsson C. H. Läsbarhet. Stockholm: Liber, 1968.
  11. Carroll J., Minnen G., Canning Y., Devlin S., Tait J. Practical simplification of English newspaper text to assist aphasic readers // Proceedings of the AAAI-98 Workshop on Integrating Artificial Intelligence and Assistive Technology. Madison, 1998.
  12. Chandrasekar R., Doran C., Srinivas B. Motivations and methods for text simplification // COLING 1996: Proceedings of the 16th International Conference on Computational Linguistics. 1996. Vol. 2. https://doi.org/10.3115/993268.993361
  13. Ebling S., Battisti A., Kostrzewa M., Pfütze D., Rios A., Säuberli A., Spring N. Automatic Text Simplification for German // Frontiers in Communication. 2022. Vol. 7. Art. 706718. https://doi.org/10.3389/fcomm.2022.706718
  14. Flesch R. A new readability yardstick // Journal of Applied Psychology. 1948. Vol. 32. № 3. https://doi.org/10.1037/h0057532
  15. Hewett F., Asghari H., Stede M. Elaborative Simplification for German-Language Texts // Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Kyoto: Association for Computational Linguistics, 2024.
  16. Papineni K., Roukos S., Ward T., Zhu W. Bleu: A method for automatic evaluation of machine translation // Proceedings of the 40th Annual Meeting on Association for Computational Linguistics (ACL '02), Philadelphia, Pennsylvania. Philadelphia: ACL, 2002. https://doi.org/10.3115/1073083.1073135
  17. Qiang J., Li Y., Zhu Y., Yuan Y., Wu X. LSBert: A Simple Framework for Lexical Simplification. 2020. arXiv:2006.14939 [cs.CL]. https://doi.org/10.48550/arXiv.2006.14939
  18. Seiffe L., Kallel F., Möller S., Naderi B., Roller R. Subjective Text Complexity Assessment for German // Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marseille, France. Marseille: European Language Resources Association, 2022.
  19. Specia L. Translating from Complex to Simplified Sentences // Computational Processing of the Portuguese Language. PROPOR 2010. Lecture Notes in Computer Science. Berlin – Heidelberg: Springer, 2010. Vol. 6001. https://doi.org/10.1007/978-3-642-12320-7_5
  20. Spring N., Rios A., Ebling S. Exploring German Multi-Level Text Simplification // Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021). 2021. http://doi.org/10.26615/978-954-452-072-4_150
  21. Sulem E., Abend O., Rappoport A. BLEU is Not Suitable for the Evaluation of Text Simplification. 2018. arXiv:1810.05995 [cs.CL]. https://doi.org/10.48550/arXiv.1810.05995
  22. Truică C., Stan A., Apostol E. SimpLex: A lexical text simplification architecture // Neural Computing and Applications. 2023. Vol. 35. https://doi.org/10.1007/s00521-022-07905-y
  23. Vaněček D., Kursch M., Şen E., Rehman I. U., Yorulmaz Y. I. Exploring how textual complexity affects cognitive load during reading: an eye-tracking study // Scientific Reports. 2026. Vol. 16. Art. 22446. https://doi.org/10.1038/s41598-026-51704-7
  24. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser L., Polosukhin I. Attention Is All You Need. 2017. arXiv:1706.03762 [cs.RL]. https://arxiv.org/abs/1706.03762
  25. Xu W., Napoles C., Pavlick E., Chen Q., Callison-Burch C. Optimizing statistical machine translation for text simplification // Transactions of the Association for Computational Linguistics. 2016. Vol. 4. https://doi.org/10.1162/tacl_a_00107
  26. Zhang X., Lu X. Aligning linguistic complexity with the difficulty of English texts for L2 learners based on CEFR levels // Studies in Second Language Acquisition. 2025. Vol. 47. № 5. https://doi.org/10.1017/S0272263125101125

Author information

Maria Vladimirovna Khokhlova

PhD

St. Petersburg State University

Anna Petrovna Sher

St. Petersburg State University

About this article

Publication history

  • Received: July 8, 2026.
  • Published: August 7, 2026.

Keywords

  • большие языковые модели
  • автоматическое упрощение текстов
  • сложность текстов
  • удобочитаемость текстов
  • немецкий язык
  • large language models
  • automatic text simplification
  • text complexity
  • readability
  • German language

Copyright

© 2026 The Author(s)
© 2026 Gramota Publishing, LLC

User license

Creative Commons Attribution 4.0 International (CC BY 4.0)