Abstract:
This paper examines the challenges of assessing the reliability of information extracted from popular science texts. Popular science is known to be characterized by a free style of presentation, loose references to primary sources and expert opinions, and the presence of controversial statements and conclusions. Clarifying these components of the author's reasoning can significantly increase the credibility among critical readers or, conversely, refute the authors' assertions.
This paper proposes a new approach to assessing the reliability of information contained in popular science publications. This approach utilizes D. Walton's argumentation theory to identify potentially unreliable claims and logical fallacies. Searching scientific databases and calculating trust metrics for scientific publications based on their metadata and scientific ratings will enable the verification of controversial claims and an assessment of their reliability. The use of large language models will enable the generation of explanations for decisions made.
To support the development and validation of this approach, two data collection methods are proposed. The first uses existing corpora of Russian-language popular science texts and commentaries with annotated argumentation. The annotation includes, among other things, polemical argumentation schemes that challenge the author's thesis or argument. The second method utilizes LLM for generating synthetic theses and popular science articles, based on actual scientific publications.