A Framework for Safety Analysis of LLM-Generated Code in Multi-Agent Systems
Main Article Content
Abstract
Large language models (LLMs) are increasingly being used for code generation. Yet, both the generated code and the underlying LLM-based systems demand rigorous security evaluation. A common approach to enhancing code generation quality involves multi-agent systems composed of multiple models. This paper evaluates the performance of GPT-OSS 20B, GPT-OSS 120B, and Qwen3-Coder 480B models in single-agent and multi-agent configurations, using two code security benchmarks, SecurityEval and CyberSecEval. The key contribution is SafeAICoder, an extensible and scalable framework for testing LLMs enabling distributed server-side generation of multi-module programs and tests, independent of client-side code.
Article Details
References
2. Ma L., Liu S., Li Y. et al. SpecGen: Automated Generation of Formal Program Specifications via Large Language Models // arXiv preprint. 2024.
arXiv:2401.08807.
3. Mandal S. et al. Large Language Models Based Automatic Synthesis of Software Specifications // arXiv preprint. 2023. arXiv:2304.09181.
4. Cassano F., Gouwar J., Nguyen D. et al. MultiPL-E: A Scalable and Polyglot Approach to Benchmarking Neural Code Generation // IEEE Transactions on Software Engineering. 2023. Vol. 49, No. 7. P. 3675–3691.
5. Chen F., Fard F. H., Lo D., Bryksin T. On the Transferability of Pre-trained Language Models for Low-resource Programming Languages // IEEE/ACM 30th International Conference on Program Comprehension (ICPC). 2022. P. 401–412.
6. Fan G., Chen S., Gao C. et al. Rapid: Zero-shot Domain Adaptation for Code Search with Pre-trained Models // ACM Transactions on Software Engineering and Methodology. 2024. Vol. 33, No. 5. P. 1–35.
7. Schäfer M., Nadi S., Eghbali A., Tip F. An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation // IEEE Transactions on Software Engineering. 2023. Vol. 50, No. 1. P. 85–105.
8. Nappa A., Johnson R., Bilge L. et al. The Attack of the Clones: A Study of the Impact of Shared Code on Vulnerability Patching // IEEE symposium on security and privacy. 2015. P. 692–708.
9. Ardito L., Coppola R., Barbato L., Verga D. A Tool-Based Perspective on Software Code Maintainability Metrics: A Systematic Literature Review // Scientific Programming 2020. P. 1–26.
10. Buse R. P. L., Weimer W. R. Learning a Metric for Code Reliability // IEEE Transactions on Software Engineering. 2010. Vol. 36, No. 4. P. 546–558.
11. Peitek N., Apel S., Parmin C. et al. Program Comprehension and Code Complexity Metrics: An fMRI Study // IEEE/ACM 43rd International Conference on Software Engineering (ICSE). 2021. P. 524–536.
12. Markovtsev V., Long W., Mougard H. et al. STYLE-ANALYZER: fixing code style inconsistencies with interpretable unsupervised algorithms // IEEE/ACM 16th International Conference on Mining Software Repositories (MSR). 2019. P. 468–478.
13. Raemaekers S., Van Deursen A., Visser J. Measuring software library stability through historical version analysis // 28th IEEE International Conference on Software Maintenance (ICSM). 2012. P. 378–387.
14. Common Weakness Enumeration. URL: https://cwe.mitre.org (accessed: 30.03.2026).
15. Open Web Application Security Project. URL: https://owasp.org (accessed: 30.03.2026).
16. SAST Bandit documentation. URL: https://bandit.readthedocs.io/en/latest (accessed: 30.03.2026).
17. SAST Semgrep documentation. URL: https://semgrep.dev/docs (accessed: 30.03.2026).
18. SAST CodeQL documentation. URL: https://codeql.github.com/docs (accessed: 30.03.2026).
19. SAST Bearer documentation. URL: https://docs.bearer.com (accessed: 30.03.2026).
20. SAST Snyk documentation. URL: https://docs.snyk.io (accessed: 30.03.2026).
21. Chen M., Tworek J., Jun H. et al. Evaluating Large Language Models Trained on Code // arXiv preprint. 2021. arXiv:2107.03374.
22. Inala J.P., Wang C., Yang M. et al. Fault-Aware Neural Code Rankers // arXiv preprint. 2022. arXiv:2206.03865.
23. Lu S. et al. CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation // arXiv preprint. 2021. arXiv:2102.04664.
24. Valentin T., Madadi A., Sapia G., Böhme M. Incoherence as Oracle-less Measure of Error in LLM-Based Code Generation // arXiv preprint. 2025. arXiv:2507.00057.
25. Spiess C., Gros D., Pai K. S. et al. Calibration and Correctness of Language Models for Code // arXiv preprint. 2024. arXiv:2402.02047.
26. Siddiq M.L., da Silva Santos J.C., Devareddy S., Muller A. Sallm: Security assessment of generated code // Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering Workshops. 2024. P. 54–65.
27. Siddiq M.L., da Silva Santos J.C. SecurityEval Dataset: Mining Vulnerability Examples to Evaluate Machine Learning-Based Code Generation Techniques // Proceedings of the 1st International Workshop on Mining Software Repositories Applications for Privacy and Security (MSR4P&S22). 2022.
28. SAST SonarQube documentation. URL: https://docs.sonarsource.com (accessed: 30.03.2026).
29. Austin J., Odena A., Nye M. et al. Program Synthesis with Large Language Models // arXiv preprint. 2021. arXiv:2108.07732.
30. Bhatt M. et al. Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models // arXiv preprint. 2023. arXiv:2312.04724.
31. MITRE ATT&CK knowledge base. URL: https://attack.mitre.org (дата обращения: 30.03.2026).
32. Abdelaziz I. et al. Granite-Function Calling Model: Introducing Function Calling Abilities via Multi-task Learning of Granular Tasks // Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track. 2024. P. 1131–1139.
33. Agarwal S. et al. gpt-oss-120b & gpt-oss-20b Model Card // arXiv preprint. 2025. arXiv:2508.10925.
34. Yang An et al. Qwen3 Technical Report // arXiv preprint. 2025.
arXiv:2505.09388.
35. Huynh N., Lin B. Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applications // arXiv preprint. 2025. arXiv:2503.01245.
36. Fakhoury S. et al. LLM-based Test-driven Interactive Code Generation: User Study and Empirical Evaluation // IEEE Transactions on Software Engineering. 2024. Vol. 50, No. 9. P. 2254–2268.
37. He X. et al. CoCoST: Automatic Complex Code Generation with Online Searching and Correctness Testing // Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024. P. 19433–19451.
38. Liu M. et al. An Empirical Study of the Code Generation of Safety-Critical Software Using LLMs // Applied Sciences. 2024. Vol. 14, No. 3. P. 1046.
39. Wang C., Zhang J., Feng Y., Li T. Teaching Code LLMs to Use Autocompletion Tools in Repository-Level Code Generation // ACM Transactions on Software Engineering and Methodology. 2025. Vol. 34, No. 7. P. 1–27.
40. Dong Y. et al. A Survey on Code Generation with LLM-based Agents // arXiv preprint. 2025. arXiv:2508.00083.
41. Nunez A. et al. AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing // arXiv preprint. 2024. arXiv:2409.10737.
42. Islam N.T. et al. Enhancing Source Code Security with LLMs: Demystifying The Challenges and Generating Reliable Repairs // arXiv preprint. 2024. 2409.00571.
43. Ishibashi Y., Nishimura Y. Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization // arXiv preprint. 2024. 2404.02183.
44. Pan R., Zhang H., Liu C. CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation // arXiv preprint. 2025. arXiv:2501.07811.
45. Holt S., Luyten M.R., van der Schaar M. L2MAC: Large Language Model Automatic Computer for Extensive Code Generation // arXiv preprint. 2023. 2310.02003.
46. GitHub repository for SafeAICoder framework. URL: https://github.com/KarineAyrs/saicoder (accessed: 30.03.2026).
47. Ollama documentation. URL: https://docs.ollama.com (accessed: 30.03.2026).
48. Merkel D. Docker: lightweight linux containers for consistent development and deployment // Linux journal. 2014. No. 239. P. 2.
49. Docker Compose tool documentation. URL: https://docs.docker.com/reference/cli/docker/compose (accessed: 30.03.2026).
50. Official Ollama client Docker image on Docker Hub. URL: https://hub.docker.com/r/ollama/ollama (accessed: 30.03.2026).
51. Blakeman A. et al. Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning // arXiv preprint. 2025. arXiv:2512.20848.
52. Devstral 2 123B Instruct 2512 model card on HuggingFace. URL: https://huggingface.co/mistralai/Devstral-2-123B-Instruct-2512 (accessed: 30.03.2026).
This work is licensed under a Creative Commons Attribution 4.0 International License.
Presenting an article for publication in the Russian Digital Libraries Journal (RDLJ), the authors automatically give consent to grant a limited license to use the materials of the Kazan (Volga) Federal University (KFU) (of course, only if the article is accepted for publication). This means that KFU has the right to publish an article in the next issue of the journal (on the website or in printed form), as well as to reprint this article in the archives of RDLJ CDs or to include in a particular information system or database, produced by KFU.
All copyrighted materials are placed in RDLJ with the consent of the authors. In the event that any of the authors have objected to its publication of materials on this site, the material can be removed, subject to notification to the Editor in writing.
Documents published in RDLJ are protected by copyright and all rights are reserved by the authors. Authors independently monitor compliance with their rights to reproduce or translate their papers published in the journal. If the material is published in RDLJ, reprinted with permission by another publisher or translated into another language, a reference to the original publication.
By submitting an article for publication in RDLJ, authors should take into account that the publication on the Internet, on the one hand, provide unique opportunities for access to their content, but on the other hand, are a new form of information exchange in the global information society where authors and publishers is not always provided with protection against unauthorized copying or other use of materials protected by copyright.
RDLJ is copyrighted. When using materials from the log must indicate the URL: index.phtml page = elbib / rus / journal?. Any change, addition or editing of the author's text are not allowed. Copying individual fragments of articles from the journal is allowed for distribute, remix, adapt, and build upon article, even commercially, as long as they credit that article for the original creation.
Request for the right to reproduce or use any of the materials published in RDLJ should be addressed to the Editor-in-Chief A.M. Elizarov at the following address: amelizarov@gmail.com.
The publishers of RDLJ is not responsible for the view, set out in the published opinion articles.
We suggest the authors of articles downloaded from this page, sign it and send it to the journal publisher's address by e-mail scan copyright agreements on the transfer of non-exclusive rights to use the work.