Abstract:

Cognitive abilities cover all higher human intellectual functions, including oral and written speech. The use of ellipses in writing is associated with the ability to clearly formulate a thought, avoiding the redundancy of its expression. Testing the ability to use ellipses can be the basis, on the one hand, for assessing the cognitive operation of generating ellipses in writing and detecting violations of this operation. This can serve as the basis for the early detection of dementia in medicine. On the other hand, such testing can be used to teach speech technologies, including in foreign languages. From a scientific point of view, it is important to determine the influence degree of training on the performance of text tasks. The article describes the development of a computer system for remote on-line testing the operation of converting a complete sentence into a sentence with an ellipsis of a certain type, the principles of evaluating the results of testing the subjects. The concept of the system is discussed. The structure of system and software development environment are described. A feature of the system is the creation of a new type of tasks related to natural language processing. The complexity of the development is determined by the need for automated formation of a bank of sentences’ pairs in natural language without ellipses and with ellipses of a certain type, which entails the inclusion, in the software environment, of text parsing tools and algorithms for automated formation of ellipses and / or restoration of complete sentences from elliptical ones. In conclusion, the prospects for the development and application of the proposed system are discussed.

Abstract:

This paper presents the experience of verifying server implementations of the TLS cryptographic protocol version 1.3. TLS is a widely used cryptographic protocol designed to create secure data transmission channels and provides the necessary functionality for this: confidentiality of the transmitted data, data integrity, and authentication of the parties. The new version 1.3 of the TLS protocol was introduced in August 2018 and has a number of significant differences compared to the previous version 1.2. A number of TLS developers have already included support for the latest version in their implementations. These circumstances make it relevant to do research in the field of verification and security of the new TLS protocol implementations. We used a new test suite for verifying implementations of the TLS 1.3 for compliance with Internet specifications, developed on the basis of the RFC8446, using UniTESK technology and mutation testing methods. The current work is part of the TLS 1.3 protocol verification project and covers some of the additional functionality and optional protocol extensions. To test implementations for compliance with formal specifications, UniTESK technology is used, which provides testing automation tools based on the use of finite state machines. The states of the system under test define the states of the state machine, and the test effects are the transitions of this machine. When performing a transition, the specified impact is passed to the implementation under test, after which the implementation's reactions are recorded and a verdict is automatically made on the compliance of the observed behavior with the specification. Mutational testing methods are used to detect non-standard behavior of the system under test by transmitting incorrect data. Some changes are made to the protocol exchange flow created in accordance with the specification: either the values of the message fields formed on the basis of the developed protocol model are changed, or the order of messages in the exchange flow is changed. The protocol model allows one to make changes to the data flow at any stage of the network exchange, which allows the test scenario to pass through all the significant states of the protocol and in each such state to test the implementation in accordance with the specified program. So far, several implementations have been found to deviate from the specification. The presented approach has proven effective in several of our projects when testing network protocols, providing detection of various deviations from the specification and other errors.

Abstract:

This paper presents the experience of verifying client implementations of the TLS cryptographic protocol version 1.3. TLS is a widely used cryptographic protocol today, designed to create secure data transmission channels. The protocol provides the necessary functionality for its tasks: confidentiality of transmitted data, data integrity, and authentication of the parties. In the new version 1.3 of the TLS architecture was significantly redesigned, eliminating a number of shortcomings of previous versions that were identified both during the development of implementations and during their operation. We used a new test suite for verifying client implementations of the TLS 1.3 for compliance with Internet specifications, developed on the basis of the RFC8446, using UniTESK technology and mutation testing methods. To test implementations for compliance with formal specifications, UniTESK technology is used, which provides testing automation tools based on the use of finite state machines. The states of the system under test define the states of the state machine, and the test effects are the transitions of this machine. When performing a transition, the specified impact is passed to the implementation under test, after which the implementation's reactions are recorded and a verdict is automatically made on the compliance of the observed behavior with the specification. Mutational testing methods are used to detect non-standard behavior of the system under test by transmitting incorrect data. Some changes are made to the protocol exchange flow created in accordance with the specification: either the values of the message fields formed on the basis of the developed protocol model are changed, or the order of messages in the exchange flow is changed. The protocol model allows one to make changes to the data flow at any stage of the network exchange, which allows the test scenario to pass through all the significant states of the protocol and in each such state to test the implementation in accordance with the specified program. The presented approach has proven effective in several of our projects when testing network protocols, providing detection of various deviations from the specification and other errors. The current work is part of the TLS 1.3 protocol verification project and covers TLS client implementations.

Abstract:

The paper is devoted to the study of the cognitive function associated with the generation of elliptical sentences in the Russian language. The study is conducted by testing this cognitive ability using a computer system specially developed by the authors for this purpose. Testing of this cognitive ability is proposed and implemented for the first time. The system is an extension of Moodle and is openly hosted in the github repository. Elliptical constructions are limited to verbal and nominal ellipses, which are theoretically possible to be completely reconstructed based on the context of the sentence. The study is conducted with the participation of SPbPU students as respondents. The texts of planimetric tasks are chosen as the subject area. As a result of the analysis of testing data, the following results are obtained: the influence of the respondent’s knowledge of the subject area (planimetry) on the test results is established; a tendency towards self-study of respondents was discovered, which is manifested in a reduction in time and an increase in scores as they pass tests; it is shown that respondents are poorly motivated if they do not see feedback on the answer to the completed task. The paper discusses the problems of further development of the testing system and its use in adapting questionnaires (tasks) to assess the knowledge of SPbPU students in the field of automation of bug detection in programs, as well as for diagnosing the functional state of operator specialists and express diagnosis of dementia. It also seems promising to use the system to improve the processes of syntactic parsing of elliptic sentences and automate the restoration of ellipses in the subject area of planimetry.

Abstract:

An approach for designing Microlearning Applications is presented. The dependency of reliability of knowledge evaluation from visual representation of testing questions. And architecture of system and working principle of developed instrument for designing adaptive tests and testing scenarios is shown.

Abstract:

The article is dedicated to the experience of the Russian State Library (RSL) in testing research papers for incorrect borrowings. Relevant digital collection of research papers, specialized software and expert analysis are regarded as necessary components for such tests being carried out. The article describes their main results, as well as the most common flaws found in the research papers tested.

Abstract:

The work is devoted to the study and improvement of the design, development, and testing of video game storytelling. The existing practices of writing and keeping up-to-date scripts for interactive works have been studied. The definition of a scenario prototype and requirements for its form are formulated. An idea was put forward about the efficiency of automating the creation of a scenario prototype in the form of a generator tool. A vision of such a tool has been drawn up. The impact of such a tool on development order is presented. Implemented a tool component and conducted an experiment that proves its effectiveness with an example such as generating storyboards from the text. Plans for future development have been formulated.

Abstract:

The article describes the experience of digital transformation of mathematical disciplines based on author's digital educational platform Mirera. The Mirera DEP is optimized for the Russian system of organization of higher education, focused on the development and delivery of courses that combine online and offline technologies for conducting the educational process. The Mirera DEP provides course authors with tools for developing computerized courses with automated verification of the correctness and independence of current and control tasks performed by students using artificial intelligence methods. Various original types of tests are built into the platform, supporting both in the description of tasks and in answer options, content in various formats, including TeX, sequences of elements (for automated testing of student knowledge of the structure of proof of course theorems or schemes for solving typical problems), semantic analysis of text responses, etc.

Abstract:

This paper focuses on the use of a linguistics-based method for automatic object-oriented sentiment analyses. The study was conducted as part of SentiRuEval automatic sentiment analysis system testing cycle. The original task was to extract users’ opinions (positive, negative, neutral) about telecom companies, expressed in tweets and news. In this study news was excluded from the dataset because, being formal texts, news significantly differs from informal ones in its structure and vocabulary and therefore demands a different approach. Only linguistic approach based on syntactic and semantic analysis was used. In this approach, a sentiment-bearing word or expression is linked to its target object at either of two stages, which perform successively. The first stage includes usage of semantic templates matching the dependence tree, and the second stage involves heuristics for linking sentiment expressions and their target objects when syntactic relations between them do not exist. No machine learning was used. The method showed a very high quality, which roughly coincides with the best results of machine learning methods and hybrid approaches.

Abstract:

This study focuses on the application of machine learning methods to improve the quality of test items. The research includes a review of the subject area and the implementation of two enhancement methods: similar question retrieval and distractor quality assessment. The first method involves testing five transformer-based models for generating text embeddings and six clustering algorithms. The second method uses the same transformer models in combination with three classification algorithms. Experimental results demonstrated the high effectiveness of the proposed approaches in solving both tasks.

Abstract: When deriving tests for checking functional and nonfunctional requirements for components of control systems, the notion of separablity becomes very important that is used for distinguishing the fault-free component from a faulty one. In order to do this, proper separating sequences are utilized. Such sequences are well studied for complete and deterministic Finite State Machines but components of control systems can be only partially described and their behavior can be nondeterministic. In this paper, we consider the formal model of Input/Output automata, introduce the notion of a separating sequence for two such automata and propose an approach for deriving such a separating sequence.

Abstract:

To disclose the concept of game balance and to develop an approach to automate the routine when working with game economics, methods of data analysis and experimentation were applied. According to the results of the analysis of existing definitions, a special case of mathematical balance and a generalized design definition of game balance were singled out. By parsing the existing approaches for balancing and testing games, a vision of our own solution was developed. Based on the findings, an approach for automating balance within a game prototype generator has been proposed. As an intermediate result, an updated structure and operation procedure of the game prototype generation tool were presented. The prospects for further development in this direction are given.

Abstract: We consider features of object domain “Substances and materials properties” treated as a testing area for development of ap-proaches to the integrity of science DB. Specific exam-ples illustrate capabilities of the ontological modeling for the semantic integration of numerical data. A gen-eral sketch of the ontology is proposed for thermody-namic data of pure compounds.

Abstract:

The developed algorithm of the visual perception system for game agents, implemented in the Unity game engine, is presented. The proposed method is based on the comparison of images from two cameras, taking into account complex visual effects (lighting, shadows, camouflage), and supplemented with line-of-sight verification, taking into account the speed of the object, and the mechanics of gradual detection. Testing of the system has shown a significant increase in realistic detection compared to traditional methods, while maintaining performance within a small additional load on the processor. The algorithm was optimized using Unity Job System and dynamic camera activation. The scientific literature on similar solutions has also been analyzed and their strengths and weaknesses have been identified. The results can be applied in video game development to create realistic behavior of non-player characters, especially in games with stealth elements.

Abstract: The article is devoted to dynamic adaptive testing of educational activity of high school students in Krasnoyarsk to solve mathematical problems of transformation of the graph of a quadratic function in electronic problem environments. Dynamic adaptive test simulator allowed to conduct: pre-test, diagnosing the level of residual knowledge of students; dynamic assessment of educational activities in the process of self-learning problem solving; post-test, diagnosing the level of training of students.

Abstract:

The results on the creation of methods for managing mathematical knowledge in the context of digital mathematical libraries are presented. The software tools developed on the basis of these methods are part of the OntoMath digital ecosystem, within which they interact. A brief description of the architecture of the OntoMath ecosystem is given, the levels of subject ontologies and external ontologies are highlighted, as well as the level of software tools and services. Semantic services are separated into a separate category. This term denotes software tools, in the functionality of which queries to subject ontologies are used to ensure the management of knowledge objects. General descriptions of developed subject ontologies are given: educational mathematical ontology OntoMath^Edu and ontology of professional mathematics OntoMath^PRO. The development of educational ontology is reflected in the direction of including educational prerequisite links between classes. Among the software tools of the digital ecosystem, search services for mathematical electronic collections, a service for semantic annotation of mathematical documents, tools for semantic marking of educational mathematical documents, as well as a system for automatically generating testing tests in mathematical educational disciplines are highlighted. As part of the OntoMath digital ecosystem, special-purpose recommender systems are being developed. The current version of the ecosystem includes a recommender system for generating a list of related articles based on the OntoMath^PRO ontology, a recommender system for appointing experts to support the scientific review process, and recommender systems for selecting subject classifiers UDC and Mathematics Subject Classification codes for mathematical documents. The results are also presented in the direction of creating a digital library metadata factory, which includes services and tools for extracting, refining, replenishing and normalizing the metadata of electronic mathematical collections. Note that the OntoMath ecosystem is being developed as the technological basis for the Lobachevskii Digital Mathematical Library.

Abstract:

The video editing process involves numerous labor-intensive operations for sorting and preparing footages, requiring significant time investment. This article describes the development of a software solution that uses machine learning technology to automate these processes.

The primary focus is on creating a system capable of classifying and sorting media files according to the screenplay text, thereby increasing the efficiency of material preparation for editing. The system includes modules for speech recognition, audio and video classification, and algorithms for determining screenplay compliance.

Testing showed that the proposed system correctly classifies media files in most cases, significantly reducing rough-cut editing time.

Abstract: All the time scientists need to publish the results of their work in order to remain relevant, meet the time, criteria, and not be outside the scientific community. The well-known principle of “publish or perish” often forces scientists to strive for quantity, not quality [1]. Along with the problems of authorship, paid research, the fabrication of the results, plagiarism and self-plagiarism are among the most common violations. Their impact is more subtle, but no less disruptive for the scientific community.
The article provides an overview of the existing tools for identifying borrowing in the scientific articles of the authors. Decisions’ analysis is performed by com-paring systems for a number of characteristics. The tools are tested on real data to investigate their performance and efficiency.

Abstract:

The task of automating the routine work of computer game writers and narrative designers, set forth in earlier works, has been continued in the presented work. The issues of visualization of branching narrative structures of computer games are considered, the analysis of various approaches to visualization of the plot and other important components of a video game is performed, a technological stack is selected and specific solutions for storing in the form of a structured script, allowing the generation of continuing narrative branches and testing of the narrative prototyping stage using the automatically generated text novelette are given.

Abstract:

This article is devoted to the study of the possibilities of optimizing the operability and improving the efficiency of complex multifunctional software systems using the example of free automated library and information systems (hereinafter - ALIS).

By 2023, the world has accumulated valuable experience in the creation and operation of integrated ALIS of various scales and purposes, but the issues of improving their design solutions remain relevant. First of all, this concerns the need to optimize the structure of the source code in order to increase its readability and maintainability, reduce the execution time of individual functional modules, and reduce the amount of RAM used.

As part of the study, a comparative analysis of the source codes of several existing open source databases implemented in various programming languages was carried out. The main approaches to the design of the code structure were studied, the most frequently used algorithms and patterns were identified. To assess the degree of optimization of the source code, a set of indicators was developed, including an assessment of the structure, readability, modularity and other characteristics. On this basis, individual code fragments were compared before and after the use of well-known refactoring techniques.

As a result of the work carried out, it was possible to identify the most common errors and shortcomings in the structuring of the source codes of the ALIS, to determine the main directions of their optimization. Data has been obtained on the possible reduction of testing and technical support costs by improving the quality of source codes.

Abstract:

This work addresses some of the legal issues emerging in connection with the necessary to implement mandatory testing of final qualifying works for borrowings at realization final attestation on educational programs of higher education, posting these works in electronic library systems educational organizations, but also their use. In particular, the need to comply with legislation on personal data when processing personal information in the course of audit work identifying plagiarism, publishing works in library systems. Paid attention to the enforcement of intellectual property rights of authors of final qualifying works during educational organizations assigned to them defined responsibilities. Analysis of legal issues carried out taking into account introduced by the Government of the Russian Federation the draft law on education the organization of higher education it is proposed to impose a duty on public access on the official website of the educational organization in the Internet the full texts of final qualification works of master's programs and specialist programs.