An Empirical Study on Using Large Language Models to Analyze Software Supply Chain Security Failures
Auteurs : Tanmay Singla, Dharun Anandayuvaraj, Kelechi G. Kalu, Taylor R. Schorlemmer, James C. Davis
Résumé scientifique
As we increasingly depend on software systems, the consequences of breaches in the software supply chain become more severe. High-profile cyber attacks like those on SolarWinds and ShadowHammer have resulted in significant financial and data losses, underlining the need for stronger cybersecurity. One way to prevent future breaches is by studying past failures. However, traditional methods of analyzing these failures require manually reading and summarizing reports about them. Automated support could reduce costs and allow analysis of more failures. Natural Language Processing (NLP) techniques such as Large Language Models (LLMs) could be leveraged to assist the analysis of failures. In this study, we assessed the ability of Large Language Models (LLMs) to analyze historical software supply chain breaches. We used LLMs to replicate the manual analysis of 69 software supply chain security failures performed by members of the Cloud Native Computing Foundation (CNCF). We developed prompts for LLMs to categorize these by four dimensions: type of compromise, intent, nature, and impact. GPT 3.5s categorizations had an average accuracy of 68% and Bard had an accuracy of 58% over these dimensions. We report that LLMs effectively characterize software supply chain failures when the source articles are detailed enough for consensus among manual analysts, but cannot yet replace human analysts. Future work can improve LLM performance in this context, and study a broader range of articles and failures.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
Unveiling Large Language Model Supply Chain: Structure, Domain, and VulnerabilitiesYan Hu, Shen-Ao Wang, Tianyuan Nie et al. · 2025 · cité 3 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) have revolutionized artificial intelligence (AI), driving breakthroughs in natural language understanding, text generation, and autonomous systems. However, the rapid growth of LLMs presents significant challenges in the security and reliability of the Large Language Model Supply Chain (LLMSC), a complex network of open-source components, libraries, and tools essential for LLM development and deployment. Despite its critical importance, the LLMSC remains underexplored, particularly regarding its structural characteristics, domain composition, and security vulnerabilities. To address this gap, we conduct the first empirical study of the LLMSC, analyzing a curated …
-
Understanding the Supply Chain and Risks of Large Language Model ApplicationsYu-Jie Ma, Li-Li Quan, Xiao-Fei Xie et al. · 2025 · arXiv.org · preprint, non relu par les pairs · cité 5 fois · Pas encore résumé
Résumé scientifique
The rise of Large Language Models (LLMs) has led to the widespread deployment of LLM-based systems across diverse domains. As these systems proliferate, understanding the risks associated with their complex supply chains is increasingly important. LLM-based systems are not standalone as they rely on interconnected supply chains involving pretrained models, third-party libraries, datasets, and infrastructure. Yet, most risk assessments narrowly focus on model or data level, overlooking broader supply chain vulnerabilities. While recent studies have begun to address LLM supply chain risks, there remains a lack of benchmarks for systematic research. To address this gap, we introduce the first c…
-
Large Language Models in Software Security AnalysisDylan Wolff, Martin Mirchev, Abhik Roychoudhury · 2026 · Communications of the ACM · cité 1 fois · Pas encore résumé
Résumé scientifique
Cybersecurity plays a pivotal role in safeguarding the integrity and reliability of critical systems—spanning healthcare, commerce, transportation, and power infrastructure. Yet software security teams are often critically underfunded and understaffed, given the breadth and complexity of our software infrastructure. Recently, large language models (LLMs) have proven to be a transformative technology that is reshaping how we interact with software. In this article, we explore how LLMs can be leveraged to holistically address longstanding challenges in cybersecurity. We start with a discussion on the structure of a cyber reasoning system (CRS) that can both detect and repair vulnerabilities in…
-
Integrating Artificial Open Generative Artificial Intelligence into Software Supply Chain SecurityVasileios Alevizos, G. Papakostas, Akebu Simasiku et al. · 2024 · 2024 5th International Conference on Data Analytics for Business and Industry (ICDABI) · cité 3 fois · Pas encore résumé
Résumé scientifique
While new technologies emerge, human errors always looming. Software supply chain is increasingly complex and intertwined, the security of a service has become paramount to ensuring the integrity of products, safeguarding data privacy, and maintaining operational continuity. In this work, we conducted experiments on the promising open Large Language Models (LLMs) into two main software security challenges: source code language errors and deprecated code, with a focus on their potential to replace conventional static and dynamic security scanners that rely on predefined rules and patterns. Our findings suggest that while LLMs present some unexpected results, they also encounter significant li…
-
Large Language Models in Cybersecurity: Applications, Vulnerabilities, and Defense TechniquesNiveen O. Jaffal, Mohammed Alkhanafseh, David Mohaisen · 2025 · Applied Informatics · cité 8 fois · Pas encore résumé
Résumé scientifique
Large Language Models (LLMs) are transforming cybersecurity by enabling intelligent, adaptive, and automated approaches to threat detection, vulnerability assessment, and incident response. With their advanced language understanding and contextual reasoning, LLMs surpass traditional methods in tackling challenges across domains such as the Internet of Things (IoT), blockchain, and hardware security. This survey provides a comprehensive overview of LLM applications in cybersecurity, focusing on two core areas: (1) the integration of LLMs into key cybersecurity domains, and (2) the vulnerabilities of LLMs themselves, along with mitigation strategies. By synthesizing recent advancements and ide…
-
Large Language Models for Cybersecurity Policy Compliance and Risk MitigationEmmanuel Cadet, Edima David Etim, Iboro Akpan Essien et al. · 2024 · International Journal of Scientific Research in Humanities and Social Sciences · cité 4 fois · Pas encore résumé
Résumé scientifique
The rapid digitization of critical business processes has heightened the importance of effective cybersecurity policy compliance and proactive risk mitigation. Large Language Models (LLMs), with their advanced natural language processing and reasoning capabilities, present a transformative opportunity to enhance compliance management, regulatory interpretation, and security decision-making. This study explores the application of LLMs in automating policy analysis, monitoring adherence to industry-specific standards, and facilitating real-time risk assessment. Leveraging extensive training on diverse text corpora, LLMs can interpret complex regulatory frameworks such as GDPR, HIPAA, NIST, and…
-
Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse EngineeringHamed Jelodar, Samita Bai, Parisa Hamedi et al. · 2025 · Cryptography and Security · preprint, non relu par les pairs · cité 42 fois · Résumé IA disponible
Résumé scientifique
Large Language Models (LLMs) have recently emerged as powerful tools in cybersecurity, offering advanced capabilities in malware detection, generation, and real-time monitoring. Numerous studies have explored their application in cybersecurity, demonstrating their effectiveness in identifying novel malware variants, analyzing malicious code structures, and enhancing automated threat analysis. Several transformer-based architectures and LLM-driven models have been proposed to improve malware analysis, leveraging semantic and structural insights to recognize malicious intent more accurately. This study presents a comprehensive review of LLM-based approaches in malware code analysis, summarizin…
-
Leveraging Large Language Models for Cybersecurity Risk Assessment - A Case from Forestry Cyber-Physical SystemsFikret Mert Gϋltekin, Oscar Lilja, Ranim Khojah et al. · 2025 · 2025 40th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) · cité 1 fois · Pas encore résumé
Résumé scientifique
In safety-critical software systems, cybersecurity activities become essential, with risk assessment being one of the most critical. In many software teams, cybersecurity experts are either entirely absent or represented by only a small number of specialists. As a result, the workload for these experts becomes high, and software engineers would need to conduct cybersecurity activities themselves. This creates a need for a tool to support cybersecurity experts and engineers in evaluating vulnerabilities and threats during the risk assessment process. This paper explores the potential of leveraging locally hosted large language models (LLMs) with retrieval-augmented generation to support cyber…
Les travaux antérieurs sur lesquels il s’appuie
-
Supply-Chain Vulnerability Elimination via Active Learning and RegenerationN. Vasilakis, Achilles Benetopoulos, Shivam Handa et al. · 2021 · Conference on Computer and Communications Security · cité 35 fois · Pas encore résumé
Résumé scientifique
Software supply-chain attacks target components that are integrated into client applications. Such attacks often target widely-used components, with the attack taking place via operations (for example, file system or network accesses) that do not affect those aspects of component behavior that the client observes. We propose new active library learning and regeneration (ALR) techniques for inferring and regenerating the client-observable behavior of software components. Using increasingly sophisticated rounds of exploration, ALR generates inputs, provides these inputs to the component, and observes the resulting outputs to infer a model of the component's behavior as a program in a domain-sp…
-
Software supply chain: review of attacks, risk assessment strategies and security controlsBetul Gokkaya, Leonardo Aniello, Basel Halak · 2023 · arXiv.org · preprint, non relu par les pairs · cité 10 fois · Pas encore résumé
Résumé scientifique
The software product is a source of cyber-attacks that target organizations by using their software supply chain as a distribution vector. As the reliance of software projects on open-source or proprietary modules is increasing drastically, SSC is becoming more and more critical and, therefore, has attracted the interest of cyber attackers. While existing studies primarily focus on software supply chain attacks' prevention and detection methods, there is a need for a broad overview of attacks and comprehensive risk assessment for software supply chain security. This study conducts a systematic literature review to fill this gap. We analyze the most common software supply chain attacks by pro…
-
A Systemic Approach for Assessing Software Supply-Chain RiskC. Alberts, Audrey J. Dorofee, Rita C. Creel et al. · 2011 · Hawaii International Conference on System Sciences · cité 34 fois · Pas encore résumé
-
NLP-assisted software testing: A systematic mapping of the literatureV. Garousi, S. Bauer, M. Felderer · 2018 · Information and Software Technology · cité 74 fois · Pas encore résumé
Résumé scientifique
Context: To reduce manual effort of extracting test cases from natural-language requirements, many approaches based on Natural Language Processing (NLP) have been proposed in the literature. Given the large amount of approaches in this area, and since many practitioners are eager to utilize such techniques, it is important to synthesize and provide an overview of the state-of-the-art in this area. Objective: Our objective is to summarize the state-of-the-art in NLP-assisted software testing which could benefit practitioners to potentially utilize those NLP-based techniques. Moreover, this can benefit researchers in providing an overview of the research landscape. Method: To address the above…
-
SoK: Analysis of Software Supply Chain Security by Establishing Secure Design PropertiesC. Okafor, Taylor R. Schorlemmer, Santiago Torres-Arias et al. · 2022 · SCORED@CCS · cité 109 fois · Pas encore résumé
Résumé scientifique
This paper systematizes knowledge about secure software supply chain patterns. It identifies four stages of a software supply chain attack and proposes three security properties crucial for a secured supply chain: transparency, validity, and separation. The paper describes current security approaches and maps them to the proposed security properties, including research ideas and case studies of supply chains in practice. It discusses the strengths and weaknesses of current approaches relative to known attacks and details the various security frameworks put out to ensure the security of the software supply chain. Finally, the paper highlights potential gaps in actor and operation-centered sup…
Les articles qui le citent
-
Large Language Models for Security Operations Centers: A Comprehensive SurveyAli Habibzadeh, Farid Feyzi, Reza Ebrahimi Atani · 2025 · Journal of Electrical and Computer Engineering · cité 43 fois · Pas encore résumé
Résumé scientifique
Security operations centers (SOCs) face escalating challenges from alert fatigue and a critical skills gap, leading to delayed incident response times. Addressing these persistent issues calls for innovative automation and decision‐support approaches. Large language models (LLMs) present a transformative opportunity to automate complex workflows and augment the capabilities of human analysts. This survey provides the first comprehensive and structured analysis of LLM integration into SOC operations. We systematically map LLM applications to the functions of the NIST Cybersecurity Framework (CSF) and use the MITRE ATT&CK framework as an analytical lens to evaluate their granular threat detect…
-
The AI-Cybersecurity Nexus: How Large Language Models are Reshaping Threat Intelligence and Digital DefenseRecep Özbay, Merve Çelebi, U. Yavanoğlu · 2026 · IEEE Access · cité 2 fois · Pas encore résumé
Résumé scientifique
As cybersecurity threats become more sophisticated, the integration of Large Language Models (LLMs) into defensive and analytical systems is transforming the field. This paper presents a PRISMA-guided bibliometric and thematic review of 149 studies published between 2015 and 2025, including 117 peer-reviewed journal and conference articles, examining publication trends and dominant research themes in LLM-enabled cybersecurity, organized around five research questions: (i) secure incorporation of LLMs into cyber threat intelligence workflows; (ii) hybrid architectures for privacy-preserving and real-time threat detection; (iii) LLM-enabled secure code remediation; (iv) adversarial misuse and …
-
Leveraging Large Language Models to Detect NPM Malicious PackagesNusrat Zahan, Philipp Burckhardt, Mikola Lysenko et al. · 2024 · International Conference on Software Engineering · cité 32 fois · Pas encore résumé
Résumé scientifique
Existing malicious code detection techniques demand the integration of multiple tools to detect different malware patterns, often suffering from high misclassification rates. Therefore, malicious code detection techniques could be enhanced by adopting advanced, more automated approaches to achieve high accuracy and a low misclassification rate. The goal of this study is to aid security analysts in detecting malicious packages by empirically studying the effectiveness of Large Language Models (LLMs) in detecting malicious code. We present SocketAI, a malicious code review workflow to detect malicious code. To evaluate the effectiveness SocketAI, we leverage a benchmark dataset of $5,115 \text…
-
When LLMs meet cybersecurity: a systematic literature reviewJie Zhang, Haoyu Bu, Hui Wen et al. · 2024 · Cybersecurity · cité 343 fois · Pas encore résumé
Résumé scientifique
The rapid development of large language models (LLMs) has opened new avenues across various fields, including cybersecurity, which faces an evolving threat landscape and demand for innovative technologies. Despite initial explorations into the application of LLMs in cybersecurity, there is a lack of a comprehensive overview of this research area. This paper addresses this gap by providing a systematic literature review, covering the analysis of over 300 works, encompassing 25 LLMs and more than 10 downstream scenarios. Our comprehensive overview addresses three key research questions: the construction of cybersecurity-oriented LLMs, the application of LLMs to various cybersecurity tasks, the…
-
FAIL: Analyzing Software Failures from the News Using LLMsDharun Anandayuvaraj, Matthew Campbell, Arav Tewari et al. · 2024 · International Conference on Automated Software Engineering · cité 15 fois · Pas encore résumé
Résumé scientifique
Software failures inform engineering work, standards, regulations. For example, the Log4J vulnerability brought government and industry attention to evaluating and securing software supply chains. Retrospective failure analysis is thus a valuable line of software engineering research. Accessing private engineering records is difficult, so such analyses tend to use information reported by the news media. However, prior works in this direction have relied on manual analysis. That has limited the scale of their analyses. The community lacks automated support to enable such analyses to consider a wide range of news sources and incidents.To fill this gap, we propose the Failure Analysis Investiga…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 4 novembre 2026.