AI summary · arXiv: 1607.01759v3 · cs.CL
Bag of Tricks for Efficient Text Classification
The essentials
- The paper explores a simple and efficient baseline for text classification
- FastText, their fast text classifier, is often on par with deep learning classifiers in terms of accuracy while being many orders of magnitude faster for training and evaluation
- Using a standard multicore CPU, they were able to train FastText on over one billion words in less than ten minutes and classify half a million sentences among 312K classes in less than a minute
- This approach offers significant advantages over traditional deep learning methods that require large amounts of data and computing power
- The authors suggest that their method could be useful for applications where speed is critical such as real-time sentiment analysis or spam detection
- FastText can be used to quickly process large volumes of text data with minimal resources required.
Key points written by AI: they may contain errors, check them against the paper.
Ask questions about this paper to our AI assistant
How to chat with this paper? Illustrated guide
You can also chat with multiple papers at once here.
Keep this conversation — a free account saves your questions and answers, your collections of papers and your alerts.
Continue with Google Create a free accountTools for this paper
Advanced tools Premium
For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium
Among the 31 papers around this one, the name that comes back most: Xiang Zhang (2 papers)
4 more people sign several of them. The full map adds their institutions and approaches, from the papers of the last years.
Papers around this one
See them on a mapRanked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
Analysis and Optimization of fastText Linear Text ClassifierVladimir Zolotov, David Kung · 2017 · Computation and Language · preprint, not peer reviewed · cited 15 times · AI summary available
Abstract
The paper [1] shows that simple linear classifier can compete with complex deep learning algorithms in text classification applications. Combining bag of words (BoW) and linear classification techniques, fastText [1] attains same or only slightly lower accuracy than deep learning algorithms [2-9] that are orders of magnitude slower. We proved formally that fastText can be transformed into a simpler equivalent classifier, which unlike fastText does not have any hidden layer. We also proved that the necessary and sufficient dimensionality of the word vector embedding space is exactly the number of document classes. These results help constructing more optimal linear text classifiers with guara…
-
A Deep Investigation into fastTextJincheng Xu, Qing-Feng Du · 2019 · 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Confer · cited 18 times · Not summarized yet
Abstract
The researches on text classification range from traditional machine learning methods which depend on handcrafted features to recent deep learning models which learn complex and non-linear relationships automatically. While neural models have been proven to be effective for text classification, they operate like a black box and offer no interpretability for humans. In this paper, we study the state-of-the-art text classification model fastText and propose a mathematical approach to interpret the behavior of fastText into a score matrix. By decomposing the output of fastText into the sum of word-level importance scores, we can observe how each word contributes to the final prediction and unde…
-
A Scalable Attention Mechanism Based Neural Network for Text ClassificationJian-Yun Zheng, J. Pang, Xiaochuan Zhang et al. · 2020 · Journal of Physics, Conference Series · Not summarized yet
Abstract
In general, deep learning based text classification methods are considered to be effective but tend to be relatively slow especially for model training. In this work, we present a powerful, so-called “scalable attention mechanism”, which performs better than conventional attention mechanism in terms of both effectiveness and the speed of model training. Based on the scalable attention mechanism, we propose a neural network for text classification. The experimental results on eight representative datasets show that our method can obtain similar accuracy to state-of-the-art methods with training in less than 4 minutes on an NVIDIA GTX 1080Ti GPU. To the best of our knowledge, our method is at …
-
FastText.zip: Compressing text classification modelsArmand Joulin, Édouard Grave, Piotr Bojanowski et al. · 2016 · arXiv (Cornell University) preprint · preprint, not peer reviewed · cited 1,333 times · Not summarized yet
Abstract
We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory. After considering different solutions inspired by the hashing literature, we propose a method built upon product quantization to store word embeddings. While the original technique leads to a loss in accuracy, we adapt this method to circumvent quantization artefacts. Our experiments carried out on several benchmarks show that our approach typically requires two orders of magnitude less memory than fastText while being only slightly inferior with respect to accuracy. As a result, it outperforms the state of the art by a good margin in terms of the c…
-
Less is More: Parameter-Free Text Classification with GzipZhi-Ying Jiang, Matthew Y. R. Yang, Mikhail Tsirlin et al. · 2022 · arXiv.org · preprint, not peer reviewed · cited 12 times · Not summarized yet
Abstract
Deep neural networks (DNNs) are often used for text classification tasks as they usually achieve high levels of accuracy. However, DNNs can be computationally intensive with billions of parameters and large amounts of labeled data, which can make them expensive to use, to optimize and to transfer to out-of-distribution (OOD) cases in practice. In this paper, we propose a non-parametric alternative to DNNs that's easy, light-weight and universal in text classification: a combination of a simple compressor like gzip with a $k$-nearest-neighbor classifier. Without any training, pre-training or fine-tuning, our method achieves results that are competitive with non-pretrained deep learning method…
-
Linear Classifier: An Often-Forgotten Baseline for Text ClassificationYu-Chen Lin, Si-An Chen, Jie-Jyun Liu et al. · 2023 · Computation and Language · cited 26 times · AI summary available
Abstract
Large-scale pre-trained language models such as BERT are popular solutions for text classification. Due to the superior performance of these advanced methods, nowadays, people often directly train them for a few epochs and deploy the obtained model. In this opinion paper, we point out that this way may only sometimes get satisfactory results. We argue the importance of running a simple baseline like linear classifiers on bag-of-words features along with advanced methods. First, for many text data, linear methods show competitive performance, high efficiency, and robustness. Second, advanced models such as BERT may only achieve the best results if properly applied. Simple baselines help to co…
-
MDLText: An efficient and lightweight text classifierRenato Moraes Silva, Tiago A. Almeida, A. Yamakami · 2017 · Knowledge-Based Systems · cited 22 times · Not summarized yet
-
An Optimized Preprocessing Algorithm That Enhances Efficiency and Effectiveness of Text ClassificationH. Ramprasanth, W. S. Vinisha · 2026 · 2026 International Conference on Connected Intelligence for Industrial Applications (CI2A) · Not summarized yet
Abstract
In Natural Language Processing (NLP) the efficiency of the preprocessing methods have an important part in defining the sky with text classification. This study introduces advanced preprocessing algorithm which combine Orthogonal Matching Pursuit algorithm based on 3 different techniques for text classification. The basic idea behind this new type of preprocessing is usage of stopword removal and/or regular filtering in combination with tokenization and lowercasing. This is a holistic integration which not only provides a significant reducing in the feature dimensions of the data, but improves the overall quality of the matrix. Extensive simulation experiments on the well recognized 30 newsg…
Prior work it builds on
-
Character-level Convolutional Networks for Text ClassificationXiang Zhang, J. Zhao, Yann LeCun · 2015 · Neural Information Processing Systems · cited 7,558 times · Not summarized yet
Abstract
This article offers an empirical exploration on the use of character-level convolutional networks (ConvNets) for text classification. We constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results. Comparisons are offered against traditional models such as bag of words, n-grams and their TFIDF variants, and deep learning models such as word-based ConvNets and recurrent neural networks.
-
Classes for fast maximum entropy trainingJoshua Goodman · 2001 · 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221) · cited 266 times · Not summarized yet
Abstract
Maximum entropy models are considered by many to be one of the most promising avenues of language modeling research. Unfortunately, long training times make maximum entropy research difficult. We present a speedup technique: we change the form of the model to use classes. Our speedup works by creating two maximum entropy models, the first of which predicts the class of each word, and the second of which predicts the word itself. This factoring of the model leads to fewer nonzero indicator functions, and faster normalization, achieving speedups of up to a factor of 35 over one of the best previous techniques. It also results in typically slightly lower perplexities. The same trick can be used…
-
Efficient Character-level Document Classification by Combining Convolution and Recurrent LayersYijun Xiao, Kyunghyun Cho · 2016 · arXiv.org · preprint, not peer reviewed · cited 223 times · Not summarized yet
Abstract
Document classification tasks were primarily tackled at word level. Recent research that works with character-level inputs shows several benefits over word-level approaches such as natural incorporation of morphemes and better handling of rare words. We propose a neural network architecture that utilizes both convolution and recurrent layers to efficiently encode character inputs. We validate the proposed model on eight large scale document classification tasks and compare with character-level convolution-only models. It achieves comparable performances with much less parameters.
-
Very Deep Convolutional Networks for Text ClassificationAlexis Conneau, Holger Schwenk, Loïc Barrault et al. · 2016 · Conference of the European Chapter of the Association for Computational Linguistics · cited 1,040 times · Not summarized yet
Abstract
The dominant approach for many NLP tasks are recurrent neural networks, in particular LSTMs, and convolutional neural networks. However, these architectures are rather shallow in comparison to the deep convolutional networks which have pushed the state-of-the-art in computer vision. We present a new architecture (VDCNN) for text processing which operates directly at the character level and uses only small convolutions and pooling operations. We are able to show that the performance of this model increases with the depth: using up to 29 convolutional layers, we report improvements over the state-of-the-art on several public text classification tasks. To the best of our knowledge, this is the …
-
Text Understanding from ScratchXiang Zhang, Yann LeCun · 2015 · arXiv.org · preprint, not peer reviewed · cited 580 times · Not summarized yet
Abstract
This article demontrates that we can apply deep learning to text understanding from character-level inputs all the way up to abstract text concepts, using temporal convolutional networks (ConvNets). We apply ConvNets to various large-scale datasets, including ontology classification, sentiment analysis, and text categorization. We show that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language. Evidence shows that our models can work for both English and Chinese.
Papers that cite it
-
Luxical: High-Speed Lexical-Dense Text EmbeddingsLuke Merrick, Alex Fang, A. Carranza et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet
Abstract
Frontier language model quality increasingly hinges on our ability to organize web-scale text corpora for training. Today's dominant tools trade off speed and flexibility: lexical classifiers (e.g., FastText) are fast but limited to producing classification output scores, while the vector-valued outputs of transformer text embedding models flexibly support numerous workflows (e.g., clustering, classification, and retrieval) but are computationally expensive to produce. We introduce Luxical, a library for high-speed"lexical-dense"text embeddings that aims to recover the best properties of both approaches for web-scale text organization. Luxical combines sparse TF--IDF features, a small ReLU n…
-
Lightweight AST-Fused TextCNNZhiqiang Lyu · 2026 · 2026 8th International Conference on Artificial Intelligence Technologies and Applications (ICAITA) · Not summarized yet
Abstract
To meet the low-cost educational AI requirement of ICAITA 2026 Track 1, this paper proposes CodeAware-TextCNN, a zero-GPU lightweight model for closed-set programming error classification on hybrid text-code student queries, clearly distinguished from intent classification, knowledge retrieval and generative LLM tutoring. Existing LLMs and transformer-based code pre-training models demand GPU resources and incur heavy $\mathbf{O}\left(\mathbf{N}^{\mathbf{2}}\right)$ self-attention overhead, while vanilla CNNs ignore hierarchical AST syntax and misclassify token-identical queries with distinct syntax bugs. Our model extracts lexical GloVe embeddings, dual-branch AST structural features and TF…
-
Graph Convolutional Networks for Text ClassificationLiang Yao, Chengsheng Mao, Yuan Luo · 2019 · Proceedings of the AAAI Conference on Artificial Intelligence · cited 2,171 times · Not summarized yet
Abstract
Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with on…
-
Deep Pyramid Convolutional Neural Networks for Text CategorizationRie Johnson, Tong Zhang · 2017 · Proceedings of the conference - Association for Computational Linguistics. Meeting · cited 862 times · Not summarized yet
Abstract
This paper proposes a low-complexity word-level deep convolutional neural network (CNN) architecture for text categorization that can efficiently represent longrange associations in text.In the literature, several deep and complex neural networks have been proposed for this task, assuming availability of relatively large amounts of training data.However, the associated computational complexity increases as the networks go deeper, which poses serious challenges in practical applications.Moreover, it was shown recently that shallow word-level CNNs are more accurate and much faster than the state-of-the-art very deep nets such as character-level CNNs even in the setting of large training data.M…
-
High-Dimensional Text Classification Using TF-IDF Features and Deep Neural NetworksSudeep S S, C. H, B. Gurubasavaraja et al. · 2026 · 2026 World Conference on Computational Science and Technology (WcCST) · Not summarized yet
Abstract
The rapid rise of digital text data in news articles, forums, and online communication has created the need to develop more accurate text classifying systems. Term-weighting techniques, such as Term Frequency-Inverse Document Frequency (TF-IDF), are the most commonly used text classifying techniques due to their simplicity and effectiveness; however, they are often faced with the challenge of high-dimensional noise, class imbalance, and class overlap in text data. To assess the effectiveness of the term weighting technique in modern text classifying systems, we propose the use of a regularized Multi-Layer Perceptron (MLP) classifier, which is trained using the TF-IDF feature representation o…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
Updated on 6 November 2026.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.