• Main Navigation
  • Main Content
  • Sidebar

Russian Digital Libraries Journal

  • Home
  • About
    • About the Journal
    • Aims and Scopes
    • Themes
    • Editor-in-Chief
    • Editorial Team
    • Submissions
    • Open Access Statement
    • Privacy Statement
    • Contact
  • Current
  • Archives
  • Register
  • Login
  • Search
Published since 1998
ISSN 1562-5419
16+
Language
  • Русский
  • English

Search

Advanced filters

Search Results

RusHallu-RAG: a Comprehensive Approach to Hallucination Detection in Russian-Language RAG Systems

Fedor Alekseevich Sadkovskii, Regina Ruslanovna Nasyrova, Alexey Andreevich Sorokin
2188-2214
Abstract:

This paper introduces RusHallu-RAG, the first comprehensive benchmark for detecting hallucinations in Russian-language RAG systems. The dataset of 1,000 query-answer pairs covers general knowledge and scientific domains with controlled context relevance balancing. A fine-grained taxonomy of six hallucination types is proposed and used for annotation at both response and span levels with human experts and LLMs.


For training a hallucination detector, a balanced dataset of 3,000 examples is collected using synthetic generation methods. A detector based on YandexGPT-5-Lite-8B is trained via SFT. We evaluate 15 open-weight and proprietary models alongside the trained detector. Results show that model size does not guarantee performance: medium-sized models (20–33B) sometimes outperform larger ones. The trained detector achieves an average accuracy of 63.7%, ranking second after proprietary Gemini-2.5-Pro, with a gain of 47.2 p.p. over the baseline. The gap between proprietary and open-weight models is reduced from 20–35 to 8.4 p.p., confirming the effectiveness of the approach. The benchmark, code, and trained model are publicly released.

Keywords: hallucination detection, retrieval-augmented generation (RAG), large language models (LLMs), benchmarking, russian language, natural language processing (NLP), question answering (QA), model evaluation, supervised fine-tuning (SFT), hallucination taxonomy.

Normalization of Text Recognized by Optical Character Recognition using Lightweight LLMS

Vladislav Konstantinovich Vershinin, Ivan Vladimirovich Khodnenko, Sergey Vladimirovich Ivanov
1036-1056
Abstract:

Despite recent progress, Optical Character Recognition (OCR) on historical newspapers still leaves 5–10% character errors. We present a fully automated post-OCR normalization pipeline that combines lightweight 7–8B instruction-tuned LLMs quantized to 4-bit (INT4) with a small set of regex rules. On the BLN600 benchmark (600 pages of 19th-century British newspapers), our best model YandexGPT-5-Instruct Q4 reduces Character Error Rate (CER) from 8.4% to 4.0% (–52.5%) and Word Error Rate (WER) from 20.2% to 6.5% (–67.8%), while raising semantic similarity to 0.962. The system runs on consumer hardware (RTX-4060 Ti, 8 GB VRAM) at about 35 seconds per page and requires no fine-tuning or parallel training data. These results indicate that compact INT4 LLMs are a practical alternative to large checkpoints for post-OCR cleanup of historical documents.

Keywords: optical character recognition, post-OCR correction, historical newspapers, large language models, quantization, INT4, normalization pipeline, character error rate, semantic similarity, regex rules, YandexGPT-5, lightweight models, natural language processing, digital humanities, document digitization.

Taxonomy Exploration with Large Language Model Reasoning and Tool Calling

Fedor Alekseevich Sadkovskii, Mikhail Mikhailovich Tikhomirov, Natalia Valentinovna Loukachevitch
1212-1234
Abstract:

The paper addresses the task of taxonomy expansion – hierarchical structures for organizing concepts. We propose an architecture based on the ReAct (Reasoning + Acting) approach that enables taxonomy expansion in a zero-shot setting without fine-tuning large language models. The system is implemented in two scenarios: autonomous navigation from root nodes and verification of hypotheses generated by other models. Experiments on the diachronic RuWordNet dataset show that direct navigation from the root faces limitations due to graph complexity (MAP@3 = 24.6%). However, using the system as a verifier improves the performance of baseline models: MAP@3 gains of 9.5 pp for FastText and 1.1 pp for TaxoYandexGPT-5-Lite. The key advantages of the approach are its universality, the absence of fine-tuning requirements, and interpretability through explicit reasoning chains.

Keywords: taxonomy, taxonomy completion, hypernym extraction, LLM, chain-of-thought, function calling, RuWordNet, reranking, computational semantics.
1 - 3 of 3 items
Information
  • For Readers
  • For Authors
  • For Librarians
Make a Submission
Current Issue
  • Atom logo
  • RSS2 logo
  • RSS1 logo

Russian Digital Libraries Journal

ISSN 1562-5419

Information

  • About the Journal
  • Aims and Scopes
  • Themes
  • Author Guidelines
  • Submissions
  • Privacy Statement
  • Contact
  • eLIBRARY.RU
  • dblp computer science bibliography

Send a manuscript

Authors need to register with the journal prior to submitting or, if already registered, can simply log in and begin the five-step process.

Make a Submission
About this Publishing System

© 2015-2026 Kazan Federal University; Institute of the Information Society