In this paper, the authors explore the use of Cyber Threat Intelligence (CTI) from hacker forums to detect potential security threats. They collect data from a real hacker forum and construct two datasets: a binary dataset and a multi-class dataset. To analyze the data, they employ supervised learning techniques such as classification algorithms and deep neural network-based classifiers. They also utilize unsupervised techniques, specifically topic modeling algorithms like Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), to further explore the datasets. Topic modeling is an unsupervised learning approach that helps identify and determine topics in a set of documents. It can be a powerful tool for finding latent topics in large unlabeled datasets. The authors use LDA and NMF algorithms for topic modeling in this study. They apply frequency weights as features to LDA, while TF-IDF weights are used for the NMF algorithm. The experimental setup involves processing the data using regular expressions and lemmatization with the spaCy library. Open-source Python libraries such as scikit-learn, Keras, gensim, and Word2Vec are utilized for various tasks including classification development, training models, and topic modeling. The authors also discuss feature engineering techniques used in their study. They employ standard techniques like bag-of-words (BOW) and term frequency-inverse document frequency (TF-IDF)-based weights. Additionally, they experiment with more advanced feature engineering techniques based on deep learning models like Word2Vec and Doc2Vec. In terms of supervised methods, the authors perform classification tasks to accurately separate cybersecurity-relevant posts from non-security posts. They compare the performances of different classifiers on their datasets, including deep neural network-based classifiers. Overall, this paper focuses on utilizing CTI from hacker forums to detect potential security threats through both supervised learning (classification) and unsupervised learning (topic modeling) approaches. The experimental results and methodologies provide insights into the effectiveness of different techniques in analyzing and predicting cyber threats.
- Authors explore the use of Cyber Threat Intelligence (CTI) from hacker forums to detect security threats
- Two datasets constructed: binary dataset and multi-class dataset
- Supervised learning techniques used, including classification algorithms and deep neural network-based classifiers
- Unsupervised techniques employed, specifically topic modeling algorithms like Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF)
- LDA and NMF algorithms used for topic modeling with frequency weights for LDA and TF-IDF weights for NMF
- Data processed using regular expressions and lemmatization with spaCy library
- Open-source Python libraries utilized, including scikit-learn, Keras, gensim, and Word2Vec
- Feature engineering techniques discussed, including bag-of-words (BOW), TF-IDF-based weights, Word2Vec, and Doc2Vec
- Classification tasks performed to separate cybersecurity-relevant posts from non-security posts
- Comparison of different classifiers' performances on datasets, including deep neural network-based classifiers
- Focus on utilizing CTI from hacker forums for threat detection through supervised learning (classification) and unsupervised learning (topic modeling)
- Experimental results provide insights into effectiveness of different techniques in analyzing and predicting cyber threats.
Authors are studying how to use information from hacker forums to find security problems. They made two sets of data: one with only two options and one with many options. They used different ways of teaching computers, like telling them what category something belongs in or using a special kind of computer called a deep neural network. They also used other methods that don't need as much guidance, like finding patterns in the words people use. They processed the data using special tools and used different computer programs to help them. They looked at how well different methods worked and learned more about how to find and predict cyber threats."
Definitions
- Cyber Threat Intelligence (CTI): Information about potential security threats online.
- Hacker forums: Online communities where hackers share information and discuss topics related to hacking.
- Datasets: Collections of organized data for analysis.
- Supervised learning techniques: Teaching computers by providing labeled examples for them to learn from.
- Classification algorithms: Computer programs that assign categories or labels to data based on patterns they recognize.
- Deep neural network-based classifiers: A type of computer program that uses artificial intelligence algorithms inspired by the human brain to classify data.
- Unsupervised techniques: Methods that allow computers to find patterns in data without being given specific instructions or labels.
- Topic modeling algorithms: Algorithms that identify common themes or topics within a set of documents or text.
- Latent Dirichlet Allocation (LDA): A specific topic modeling algorithm that finds hidden topics within a collection of documents.
- Non
Using Cyber Threat Intelligence to Detect Potential Security Threats
In recent years, cyber threats have become increasingly prevalent and sophisticated. As a result, organizations are turning to Cyber Threat Intelligence (CTI) as a means of detecting potential security threats. In this paper, the authors explore the use of CTI from hacker forums to detect such threats. They collect data from a real hacker forum and construct two datasets: a binary dataset and a multi-class dataset. To analyze the data, they employ supervised learning techniques such as classification algorithms and deep neural network-based classifiers. They also utilize unsupervised techniques, specifically topic modeling algorithms like Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), to further explore the datasets.
Data Collection
The authors collect data from an online hacker forum for their study. The collected data is then processed using regular expressions and lemmatization with the spaCy library in order to prepare it for analysis.
Supervised Learning Techniques
To accurately separate cybersecurity-relevant posts from non-security posts, the authors perform classification tasks using various supervised learning methods including deep neural network-based classifiers developed with open source Python libraries such as scikit-learn, Keras, gensim, and Word2Vec.
Feature Engineering
The authors discuss feature engineering techniques used in their study which include standard techniques like bag-of-words (BOW) and term frequency–inverse document frequency (TF–IDF)-based weights as well as more advanced feature engineering techniques based on deep learning models like Word2Vec and Doc2Vec.
Unsupervised Learning Techniques
The authors also utilize unsupervised learning approaches such as topic modeling algorithms like Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF). Topic modeling is an unsupervised learning approach that helps identify latent topics in large unlabeled datasets by assigning weights to words or phrases within documents according to their relevance or importance relative to other words or phrases in that document or corpus of documents respectively. Frequency weights are applied when using LDA while TF–IDF weights are used for NMF algorithm implementations in this study .
Experimental Results & Methodologies
The experimental results demonstrate how different supervised & unsupervised machine learning methods can be utilized effectively for analyzing & predicting cyber threats through CTI gathered from hacker forums . Additionally , insights into various methodologies employed during this research process are discussed , providing readers with valuable information on how best to apply these approaches when dealing with similar problems .
Conclusion
This paper focuses on utilizing CTI from hacker forums to detect potential security threats through both supervised learning & unsupervised learning approaches . Through experiments conducted by employing various machine learning methods , it was found that these approaches can be effective tools when attempting to predict cyber threats . Furthermore , insights into feature engineering techniques used throughout this research provide readers with useful information on how best implement them when dealing with similar problems .
