LogBERT: Log Anomaly Detection via BERT

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Cites this paper Not relevant

   LogContrast: Log-based Anomaly Detection Using BERT and Contrastive Learning

   Wei Yuan, Hongyu Sun, Mo Pang et al. · 2024 · International Conference on Trust, Security and Privacy in Computing and Communications · cited 5 times · Not summarized yet

   Abstract

   Log data plays a vital role in the maintenance and fault diagnosis of computer systems. However, with the increase of system size and complexity, manual log analysis will take a lot of effort, which makes automated log anomaly detection an important research area. Early rule-based methods can effectively analyze software failures in logs, but require a lot of expert knowledge support. In recent years, with the development of deep learning (DL), automatic detection methods based on learning, such as DeepLog and LogBERT, have improved the efficiency of anomaly detection to a certain extent, but there are still problems such as semantic loss and data imbalance when dealing with massive and dive…

2. Very close Not relevant

   SemLog: A Semantics-based Approach for Anomaly Detection in Big Data System Logs

   Xiaoman Tan, Ningning Han, Si-Yang Lu et al. · 2023 · International Conference on Parallel and Distributed Systems · cited 2 times · Not summarized yet

   Abstract

   Syslog-based anomaly detection is crucial for protecting the systems from malicious attacks or malfunctions. System logs are semi-structured text messages printed by logging statements to record the system’s run-time status, involving rich semantic information. However, the existing BERT-based log anomaly detection method is based on the log key sequence, does not consider the semantics of the log data, and discards the variable part, resulting in a high rate of missed detection. In this paper, we propose SemLog, a self-supervised framework for log anomaly detection based on BERT. By incorporating log semantics and variables and employing multi-feature fusion, we mitigate the independent ass…

3. Very close* Not relevant

   Log anomaly detection based on BERT

   Pan Tang, Ye-Peng Guan · 2024 · Signal, Image and Video Processing · cited 9 times · Not summarized yet
4. Very close Not relevant

   DualBERT: Fusing Symbolic and Temporal Dynamics for High-Precision Log Anomaly Detection

   Ahmed A. Alzamil, Ramzi Guesmi, Amer Alghadhban et al. · 2026 · IEEE Access · cited 3 times · Not summarized yet

   Abstract

   Anomaly detection in system logs has attracted considerable interest with the development of deep learning and big data. Many methods and approaches have achieved good results in log anomaly detection, but exploiting the rich dependencies hidden in log data remains an extremely challenging task. Recently, Transformer-based models, such as LogBERT, represent a strong and promising foundation thanks to their ability to capture rich contextual information within log sequences. The limitation of these models is that they mainly treat logs as a sequence of discrete tokens, neglecting the crucial and continuous information embedded in the temporal and numerical dynamics between events. This limita…

5. Very close Cites this paper Not relevant

   Deep Anomaly Detection in Unstructured System Log Files with Ensemble-Based Learning

   Mikhail E. Zuev, Artyom D. Semidolin · 2026 · 2026 ElCon Conference of Young Researchers in Computing & Processing, and Information Security (ElCon-CP) · Not summarized yet

   Abstract

   Anomaly detection in unstructured system logs remains challenging due to high variance of deep learning models across training subsamples, leading to unstable performance. While BERT-based approaches like LogBERT achieve state-of-the-art results, their sensitivity to data shifts limits reliability. To address this, we propose an ensemble of LogBERT models using bootstrap aggregating (bagging) to reduce prediction variance and improve robustness. The implemented method trains multiple LogBERT instances on bootstrapped samples of normal log sequences and combines their outputs via majority voting. Evaluated on standard LogHub datasets — HDFS, BGL, Thunderbird (first 20M log lines), and Spirit …

6. Very close Cites this paper Not relevant

   A Comprehensive Evaluation of Unsupervised Log Anomaly Detection on Logs from Real Network Systems

   Yoshie Morita, Kengo Tajiri, Yoichi Matsuo · 2026 · 2026 IEEE International Conference on Communications Workshops (ICC Workshops) · Not summarized yet

   Abstract

   Log anomaly detection is an essential component in maintaining the reliability of industrial and production network systems. While many unsupervised LSTM-based methods have been evaluated on open datasets, their behavior on logs collected from actual network systems remains unclear. In this paper, we conduct a comparative evaluation of five unsupervised log anomaly detection models—DeepLog, DeepLog with Word2Vec embeddings, DeepLog with BERT embeddings, LogAnomaly, and LogAnomaly with BERT embeddings—using both public datasets (HDFS and OpenStack) and three proprietary datasets collected from actual network systems. The results show that detection performance depends more strongly on dataset…

7. Very close Cites this paper Not relevant

   LogAL-Prefix: Parameter-Efficient Log Anomaly Detection Based on Active Learning and Prefix

   Shi-Ming He, Yanling Wu, Ying Lei et al. · 2025 · International Symposium on Image and Signal Processing and Analysis · Not summarized yet

   Abstract

   Logs continuously record both normal and anomalous system operational states, serving as critical data for system runtime monitoring. Log anomaly detection facilitates system debugging and root cause analysis for issues such as system failures, shutdown faults, null pointer exceptions, illegal argument exceptions, and class cast exceptions. Deep learning has been widely adopted in log anomaly detection to enhance detection accuracy. However, deep learning models require substantial labeled log data, which demands significant human effort and time investment. To address this labeling requirement, pretraining models like BERT have been introduced. However, pretraining models introduce new chal…

8. Very close Cites this paper Not relevant

   Semantic-Aware Log Anomaly Detection and Prediction

   Bengisu Öztürk, Burak Eroğlu, Can Deniz Yetkin et al. · 2026 · 2026 10th International Artificial Intelligence and Data Processing Symposium (IDAP) · Not summarized yet

   Abstract

   Modern IT infrastructures generate vast volumes of system logs essential for monitoring and failure diagnosis. While conventional index-based log anomaly detection methods achieve strong performance in controlled settings, they produce high false positive rates when log messages change due to software updates, a problem known as log template evolution. Generic semantic encoders miss domain-specific meaning and execution order, while LLM baselines are too slow for production. We propose a hybrid LogBERT-LSTM framework that combines a domain-adapted BERT encoder, refined through SimCSE contrastive training, with an LSTM that predicts the next log event in continuous semantic embedding space. E…

Prior work it builds on

1. Close Used in depth by this paper Not relevant

   LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs

   Wei-Bin Meng, Ying Liu, Yichen Zhu et al. · 2019 · International Joint Conference on Artificial Intelligence · cited 810 times · Not summarized yet

   Abstract

   Recording runtime status via logs is common for almost every computer system, and detecting anomalies in logs is crucial for timely identifying malfunctions of systems. However, manually detecting anomalies for logs is time-consuming, error-prone, and infeasible. Existing automatic log anomaly detection approaches, using indexes rather than semantics of log templates, tend to cause false alarms. In this work, we propose LogAnomaly, a framework to model unstructured a log stream as a natural language sequence. Empowered by template2vec, a novel, simple yet effective method to extract the semantic information hidden in log templates, LogAnomaly can detect both sequential and quantitive log ano…

2. Close* Used in depth by this paper Not relevant

   DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning

   Min Du, Fei-Fei Li, Gui-Neng Zheng et al. · 2017 · Conference on Computer and Communications Security · cited 2,152 times · Not summarized yet
3. Close* Used in depth by this paper Not relevant

   Experience Report: System Log Analysis for Anomaly Detection

   Shilin He, Jie-Ming Zhu, Pin-Jia He et al. · 2016 · IEEE International Symposium on Software Reliability Engineering · cited 643 times · Not summarized yet
4. Close Not relevant

   Robust log-based anomaly detection on unstable log data

   Xu Zhang, Yong Xu, Qingwei Lin et al. · 2019 · ESEC/SIGSOFT FSE · cited 840 times · Not summarized yet

   Abstract

   Logs are widely used by large and complex software-intensive systems for troubleshooting. There have been a lot of studies on log-based anomaly detection. To detect the anomalies, the existing methods mainly construct a detection model using log event data extracted from historical logs. However, we find that the existing methods do not work well in practice. These methods have the close-world assumption, which assumes that the log data is stable over time and the set of distinct log events is known. However, our empirical study shows that in practice, log data often contains previously unseen log events or log sequences. The instability of log data comes from two sources: 1) the evolution o…

5. Close Not relevant

   A Survey on Log Anomaly Detection using Deep Learning

   R. Yadav, P. Kumar, S. Dhavale · 2020 · 2020 8th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) ( · cited 65 times · Not summarized yet

   Abstract

   Logs generated from the security systems, network devices, servers, and various software applications are one of the ways to record the operational happening of the equipment or software. These logs are assets for extracting meaningful information related to system behavior. Increasing usage of computer devices and the evolution of software systems can be considered as one of triggering acts for the concentration on the analysis of logs. Also, considering the massive volume of unstructured data, it raises the requirement for automatic analysis of these logs. The log analysis is helpful for understanding system behavior, malfunctioning detection, security scanning, and failure prediction. Mac…

Papers that cite it

1. Close Uses this paper in depth Not relevant

   LogLLM: Log-based Anomaly Detection Using Large Language Models

   Wei Guan, Jian Cao, Shi-You Qian et al. · 2024 · IEEE Transactions on Software Engineering · cited 85 times · Not summarized yet

   Abstract

   Software systems often record important runtime information in logs to help with troubleshooting. Log-based anomaly detection has become a key research area that aims to identify system issues through log data, ultimately enhancing the reliability of software systems. Existing methods often fall short in capturing the semantic information, typically expressed in natural language, or the sequential dependencies inherent in log sequences. In this paper, we propose LogLLM, a framework that enables collaboration between heterogeneous LLMs for log-based anomaly detection. LogLLM exploits the complementary capabilities of different LLM architectures: a Transformer encoder-based LLM is employed to …

2. Close Uses this paper in depth Not relevant

   Unsupervised Anomaly Detection in OpenStack Logs via Fine-Tuned RoBERTa Embeddings

   Janit Rajkarnikar, N. Poudel, Nick Rahimi · 2025 · Journal of Cybersecurity, Digital Forensics and Jurisprudence · cited 2 times · Not summarized yet

   Abstract

   As cloud computing infrastructures increasingly depend on reliable operation, proactively detecting anomalies in system logs becomes indispensable. Conventional log analysis techniques often produce high false-alarm rates and exhibit limited semantic understanding. To address these limitations, we developed an unsupervised anomaly detection framework that leverages fine-tuned RoBERTa-base model embeddings to capture contextual patterns within OpenStack log event sequences. We apply a crucial filtering step to remove high-frequency, non-discriminatory events, ensuring our models learn from nuanced contextual signals rather than simple indicators. From these refined sequences, we construct a c…

3. Close Uses this paper in depth Not relevant

   ADALog: Adaptive Unsupervised Anomaly detection in Logs with Self-attention Masked Language Model

   Przemek Pospieszny, Wojciech Mormul, Karolina Szyndler et al. · 2025 · International Conference on Machine Learning Technologies · cited 7 times · Not summarized yet

   Abstract

   Modern software systems generate extensive heterogeneous log data with dynamic formats, fragmented event sequences, and varying temporal patterns, making anomaly detection both crucial and challenging. To address these complexities, we propose ADALog, an adaptive, unsupervised anomaly detection framework designed for practical applicability across diverse real-world environments. Unlike traditional methods reliant on log parsing, strict sequence dependencies, or labeled data, ADALog operates on individual unstructured logs, extracts intra-log contextual relationships, and performs adaptive thresholding on normal data. The proposed approach utilizes transformer-based, pretrained bidirectional…

4. Close Not relevant

   Log Sequence Anomaly Detection Based on Template and Parameter Parsing via BERT

   Xiaoli Chai, Hang-Yu Zhang, Jue Zhang et al. · 2025 · IEEE Transactions on Dependable and Secure Computing · cited 20 times · Not summarized yet

   Abstract

   Logs record various operations and events during system running in text format, which is an essential basis for detecting and identifying potential security threats or system failures, and is widely used in system management to ensure security and reliability. Existing log sequence anomaly detection is limited by log parsing and does not consider all key features of logs, which may cause false or missed detection. In this article, we propose a fast and accurate log parsing method and feed the entire log content into the deep learning network for analysis. To avoid semantic loss during parsing, we replace some variables with tokens containing semantic information and divide logs with appropri…

5. Close Uses this paper in depth Not relevant

   LogLLaMA: Transformer-based log anomaly detection with LLaMA

   Zhuo-Yi Yang, Ian G. Harris · 2025 · IEEE International Joint Conference on Neural Network · cited 15 times · Not summarized yet

   Abstract

   Log anomaly detection refers to the task that distinguishes the anomalous log messages from normal log messages. Transformer-based large language models (LLMs) are becoming popular for log anomaly detection because of their superb ability to understand complex and long language patterns. In this paper, we propose LogLLaMA, a novel framework that leverages LLaMA2. LogLLaMA is first finetuned on normal log messages from three large-scale datasets to learn their patterns. After finetuning, the model is capable of generating successive log messages given previous log messages. Our generative model is further trained to identify anomalous log messages using reinforcement learning (RL). The experi…