Method of Pre-Assessment of Students' Answers Based on the Vector Model of Documents
Main Article Content
Abstract
This article discusses the application of vector models for the preliminary analysis of students' free-form answers. Vector representations of words and documents were obtained using word2vec, doc2vec, and BERT models. The similarity between the answer given by the student and the correct answer was determined using the cosine measure. It was found that vector models allow identifying obviously incorrect answers with sufficient accuracy. For answers that are close in wording, an additional verification step is proposed. Using word2vec, binary classification of answers to certain questions was performed, and accuracy, precision, recall and F1-measure estimates were given.
Keywords:
Article Details
References
2. Миннегалиева Ч.Б., Мухамедшин Д.Р., Русецкий К.В., Паркалов А.В. Некоторые вопросы автоматизации контроля знаний // Компьютерные инструменты в образовании. 2014. № 6. С. 52–59.
3. Харламенко И.В., Воног В.В. Обратная связь как форма контроля в техногенной образовательной среде // Информатика и образование. 2020. № 5(314). С. 44–49.
4. Веремчук С.Э., Гурин Н.И. Система тестирования знаний на естественном языке на основе семантической сети обучающей системы // Труды БГТУ. Серия 3: Физико-математические науки и информатика. 2019. № 1(218). С. 51–56.
5. Xu S., Xu G., Jia P., Ding W., Wu Z., Liu, Z. Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension // In: Roll I., McNamara D., Sosnovsky S., Luckin R., Dimitrova V. (Eds.) Artificial Intelligence in Education. AIED 2021. Utrecht, The Netherlands, June 14–18, 2021. Lecture Notes in Computer Science, Vol. 12748. Springer, Cham, 2021. P.446–458.
6. Милов В.Р., Дубов М.С., Калинина Н.А., Салтыкова А.А. Интеллектуализация тестирования с открытыми вопросами на основе определения семантической близости фраз // Интеллектуальные системы в науке и технике. Искусственный интеллект в решении актуальных социальных и экономических проблем ХХI века: сборник статей по материалам Международной конференции «Интеллектуальные системы в науке и технике» и Шестой всероссийской научно-практической конференции «Искусственный интеллект в решении актуальных социальных и экономических проблем ХХI века» (г. Пермь, 12–18 октября 2020 г.). 2020. С. 112–115.
7. Wulff P., Mientus L., Nowak A. et al. Utilizing a Pretrained Language Model (BERT) to Classify Preservice Physics Teachers’ Written Reflections // International Journal of Artificial Intelligence in Education. 2022.
8. Белов С.Д., Зрелова Д.П., Зрелов П.В., Кореньков В.В. Обзор методов автоматической обработки текстов на естественном языке // Системный анализ в науке и образовании. 2020. № 3. С. 8–22.
9. Mikolov T., Sutskever I., Chen K., Corrado G.S., Dean J. Distributed representations of words and phrases and their compositionality // Advances in Neural Information Processing Systems. 2013. P. 3111–3119.
10. Кузнецов С.А., Вильнин А.Д. Сравнение методов оценки семантического сходства на основе doc2vec и tf-idf // Электронные средства и системы управления. Материалы докладов Международной научно-практической конференции. 2021. № 1–2. С. 166–168.
11. Ярушкина Н.Г., Мошкин В.С., Константинов А.А. Применение языковых моделей word2vec и bert в задаче сентимент-анализа текстовых сообщений социальных сетей // Автоматизация процессов управления. 2020. № 3(61). С. 60–69.
12. Kutuzov A., Kuzmenko E. WebVectors: A Toolkit for Building Web Interfaces for Vector Semantic Models // In: Ignatov D. et al. (Eds.) Analysis of Images, Social Networks and Texts. AIST 2016. Communications in Computer and Information Science, Springer, Cham. 2017. V. 661. P. 155–161.
This work is licensed under a Creative Commons Attribution 4.0 International License.
Presenting an article for publication in the Russian Digital Libraries Journal (RDLJ), the authors automatically give consent to grant a limited license to use the materials of the Kazan (Volga) Federal University (KFU) (of course, only if the article is accepted for publication). This means that KFU has the right to publish an article in the next issue of the journal (on the website or in printed form), as well as to reprint this article in the archives of RDLJ CDs or to include in a particular information system or database, produced by KFU.
All copyrighted materials are placed in RDLJ with the consent of the authors. In the event that any of the authors have objected to its publication of materials on this site, the material can be removed, subject to notification to the Editor in writing.
Documents published in RDLJ are protected by copyright and all rights are reserved by the authors. Authors independently monitor compliance with their rights to reproduce or translate their papers published in the journal. If the material is published in RDLJ, reprinted with permission by another publisher or translated into another language, a reference to the original publication.
By submitting an article for publication in RDLJ, authors should take into account that the publication on the Internet, on the one hand, provide unique opportunities for access to their content, but on the other hand, are a new form of information exchange in the global information society where authors and publishers is not always provided with protection against unauthorized copying or other use of materials protected by copyright.
RDLJ is copyrighted. When using materials from the log must indicate the URL: index.phtml page = elbib / rus / journal?. Any change, addition or editing of the author's text are not allowed. Copying individual fragments of articles from the journal is allowed for distribute, remix, adapt, and build upon article, even commercially, as long as they credit that article for the original creation.
Request for the right to reproduce or use any of the materials published in RDLJ should be addressed to the Editor-in-Chief A.M. Elizarov at the following address: amelizarov@gmail.com.
The publishers of RDLJ is not responsible for the view, set out in the published opinion articles.
We suggest the authors of articles downloaded from this page, sign it and send it to the journal publisher's address by e-mail scan copyright agreements on the transfer of non-exclusive rights to use the work.