AI summary · arXiv: 1607.01759v3 · cs.CL

Bag of Tricks for Efficient Text Classification

Armand Joulin, Edouard Grave, Piotr Bojanowski, Tomas Mikolov
Original paper · DOI 10.18653/v1/e17-2068 Submitted to arXiv on 06 Jul. 2016 Published in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers (2017) License: NONEXCLUSIVE-DISTRIB 1.0

How to read this summary? Illustrated guide

The essentials

  • The paper explores a simple and efficient baseline for text classification
  • FastText, their fast text classifier, is often on par with deep learning classifiers in terms of accuracy while being many orders of magnitude faster for training and evaluation
  • Using a standard multicore CPU, they were able to train FastText on over one billion words in less than ten minutes and classify half a million sentences among 312K classes in less than a minute
  • This approach offers significant advantages over traditional deep learning methods that require large amounts of data and computing power
  • The authors suggest that their method could be useful for applications where speed is critical such as real-time sentiment analysis or spam detection
  • FastText can be used to quickly process large volumes of text data with minimal resources required.

Key points written by AI: they may contain errors, check them against the paper.

AI-generated keywords: fastText Text Classification Deep Learning Natural Language Processing Efficiency

Read further

  • Comprehensive Summary
  • Layman's Summary
  • Blog article
  • Original abstract
The paper "Bag of Tricks for Efficient Text Classification" by Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov explores a simple and efficient baseline for text classification. The authors conducted experiments that demonstrated their fast text classifier, fastText, is often on par with deep learning classifiers in terms of accuracy while being many orders of magnitude faster for training and evaluation. Using a standard multicore CPU, they were able to train fastText on over one billion words in less than ten minutes and classify half a million sentences among 312K classes in less than a minute. This approach offers significant advantages over traditional deep learning methods that require large amounts of data and computing power. The authors suggest that their method could be useful for applications where speed is critical such as real-time sentiment analysis or spam detection. Overall, the paper provides a valuable contribution to the field of natural language processing by offering an efficient alternative to traditional deep learning methods for text classification. By leveraging its speed and accuracy benefits, fastText can be used to quickly process large volumes of text data with minimal resources required. This makes it an attractive option for applications where time or resource constraints are present.

Was this synthesis useful?

Score: 1

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Figures and plots

Ask about a figure by number or use its question button to request visual analysis. Up to two figures and four image panels per question. Values read from plots are approximate; check the original data.

AI assistant instructions?

Tools for this paper

Explain it differently 1 free rewrite left today
Save to a collection Group papers in collections to compare them, chat with them and export them. Continue with Google Create a free account Log in

Advanced tools Premium

For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium

Zotero Zotero guide
Citation alerts More than 300 papers already cite it: be emailed each new one. Citation alerts guide
Who works on it?

Among the 31 papers around this one, the name that comes back most: Xiang Zhang (2 papers)

4 more people sign several of them. The full map adds their institutions and approaches, from the papers of the last years.

Continue with Google Create a free account: your first map is free
Weekly letter The latest letter of its field, "Artificial intelligence this week" of 16 November 2026: Artificial intelligence this week: What impressive scores leave out · Receive one by email for free

Papers around this one

See them on a map

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

  1. Very close
    Analysis and Optimization of fastText Linear Text Classifier
    Vladimir Zolotov, David Kung · 2017 · Computation and Language · preprint, not peer reviewed · cited 15 times · AI summary available
    Abstract

    The paper [1] shows that simple linear classifier can compete with complex deep learning algorithms in text classification applications. Combining bag of words (BoW) and linear classification techniques, fastText [1] attains same or only slightly lower accuracy than deep learning algorithms [2-9] that are orders of magnitude slower. We proved formally that fastText can be transformed into a simpler equivalent classifier, which unlike fastText does not have any hidden layer. We also proved that the necessary and sufficient dimensionality of the word vector embedding space is exactly the number of document classes. These results help constructing more optimal linear text classifiers with guara…

  2. Close
    A Deep Investigation into fastText
    Jincheng Xu, Qing-Feng Du · 2019 · 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Confer · cited 18 times · Not summarized yet
    Abstract

    The researches on text classification range from traditional machine learning methods which depend on handcrafted features to recent deep learning models which learn complex and non-linear relationships automatically. While neural models have been proven to be effective for text classification, they operate like a black box and offer no interpretability for humans. In this paper, we study the state-of-the-art text classification model fastText and propose a mathematical approach to interpret the behavior of fastText into a score matrix. By decomposing the output of fastText into the sum of word-level importance scores, we can observe how each word contributes to the final prediction and unde…

  3. Close
    A Scalable Attention Mechanism Based Neural Network for Text Classification
    Jian-Yun Zheng, J. Pang, Xiaochuan Zhang et al. · 2020 · Journal of Physics, Conference Series · Not summarized yet
    Abstract

    In general, deep learning based text classification methods are considered to be effective but tend to be relatively slow especially for model training. In this work, we present a powerful, so-called “scalable attention mechanism”, which performs better than conventional attention mechanism in terms of both effectiveness and the speed of model training. Based on the scalable attention mechanism, we propose a neural network for text classification. The experimental results on eight representative datasets show that our method can obtain similar accuracy to state-of-the-art methods with training in less than 4 minutes on an NVIDIA GTX 1080Ti GPU. To the best of our knowledge, our method is at …

  4. Close Cites this paper
    FastText.zip: Compressing text classification models
    Armand Joulin, Édouard Grave, Piotr Bojanowski et al. · 2016 · arXiv (Cornell University) preprint · preprint, not peer reviewed · cited 1,333 times · Not summarized yet
    Abstract

    We consider the problem of producing compact architectures for text classification, such that the full model fits in a limited amount of memory. After considering different solutions inspired by the hashing literature, we propose a method built upon product quantization to store word embeddings. While the original technique leads to a loss in accuracy, we adapt this method to circumvent quantization artefacts. Our experiments carried out on several benchmarks show that our approach typically requires two orders of magnitude less memory than fastText while being only slightly inferior with respect to accuracy. As a result, it outperforms the state of the art by a good margin in terms of the c…

  5. Close
    Less is More: Parameter-Free Text Classification with Gzip
    Zhi-Ying Jiang, Matthew Y. R. Yang, Mikhail Tsirlin et al. · 2022 · arXiv.org · preprint, not peer reviewed · cited 12 times · Not summarized yet
    Abstract

    Deep neural networks (DNNs) are often used for text classification tasks as they usually achieve high levels of accuracy. However, DNNs can be computationally intensive with billions of parameters and large amounts of labeled data, which can make them expensive to use, to optimize and to transfer to out-of-distribution (OOD) cases in practice. In this paper, we propose a non-parametric alternative to DNNs that's easy, light-weight and universal in text classification: a combination of a simple compressor like gzip with a $k$-nearest-neighbor classifier. Without any training, pre-training or fine-tuning, our method achieves results that are competitive with non-pretrained deep learning method…

  6. Close
    Linear Classifier: An Often-Forgotten Baseline for Text Classification
    Yu-Chen Lin, Si-An Chen, Jie-Jyun Liu et al. · 2023 · Computation and Language · cited 26 times · AI summary available
    Abstract

    Large-scale pre-trained language models such as BERT are popular solutions for text classification. Due to the superior performance of these advanced methods, nowadays, people often directly train them for a few epochs and deploy the obtained model. In this opinion paper, we point out that this way may only sometimes get satisfactory results. We argue the importance of running a simple baseline like linear classifiers on bag-of-words features along with advanced methods. First, for many text data, linear methods show competitive performance, high efficiency, and robustness. Second, advanced models such as BERT may only achieve the best results if properly applied. Simple baselines help to co…

  7. Close*
    MDLText: An efficient and lightweight text classifier
    Renato Moraes Silva, Tiago A. Almeida, A. Yamakami · 2017 · Knowledge-Based Systems · cited 22 times · Not summarized yet
  8. Close Cites this paper
    An Optimized Preprocessing Algorithm That Enhances Efficiency and Effectiveness of Text Classification
    H. Ramprasanth, W. S. Vinisha · 2026 · 2026 International Conference on Connected Intelligence for Industrial Applications (CI2A) · Not summarized yet
    Abstract

    In Natural Language Processing (NLP) the efficiency of the preprocessing methods have an important part in defining the sky with text classification. This study introduces advanced preprocessing algorithm which combine Orthogonal Matching Pursuit algorithm based on 3 different techniques for text classification. The basic idea behind this new type of preprocessing is usage of stopword removal and/or regular filtering in combination with tokenization and lowercasing. This is a holistic integration which not only provides a significant reducing in the feature dimensions of the data, but improves the overall quality of the matrix. Extensive simulation experiments on the well recognized 30 newsg…

Prior work it builds on

  1. Related A method it uses
    Character-level Convolutional Networks for Text Classification
    Xiang Zhang, J. Zhao, Yann LeCun · 2015 · Neural Information Processing Systems · cited 7,558 times · Not summarized yet
    Abstract

    This article offers an empirical exploration on the use of character-level convolutional networks (ConvNets) for text classification. We constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results. Comparisons are offered against traditional models such as bag of words, n-grams and their TFIDF variants, and deep learning models such as word-based ConvNets and recurrent neural networks.

  2. Related A method it uses
    Classes for fast maximum entropy training
    Joshua Goodman · 2001 · 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221) · cited 266 times · Not summarized yet
    Abstract

    Maximum entropy models are considered by many to be one of the most promising avenues of language modeling research. Unfortunately, long training times make maximum entropy research difficult. We present a speedup technique: we change the form of the model to use classes. Our speedup works by creating two maximum entropy models, the first of which predicts the class of each word, and the second of which predicts the word itself. This factoring of the model leads to fewer nonzero indicator functions, and faster normalization, achieving speedups of up to a factor of 35 over one of the best previous techniques. It also results in typically slightly lower perplexities. The same trick can be used…

  3. Related
    Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
    Yijun Xiao, Kyunghyun Cho · 2016 · arXiv.org · preprint, not peer reviewed · cited 223 times · Not summarized yet
    Abstract

    Document classification tasks were primarily tackled at word level. Recent research that works with character-level inputs shows several benefits over word-level approaches such as natural incorporation of morphemes and better handling of rare words. We propose a neural network architecture that utilizes both convolution and recurrent layers to efficiently encode character inputs. We validate the proposed model on eight large scale document classification tasks and compare with character-level convolution-only models. It achieves comparable performances with much less parameters.

  4. Related
    Very Deep Convolutional Networks for Text Classification
    Alexis Conneau, Holger Schwenk, Loïc Barrault et al. · 2016 · Conference of the European Chapter of the Association for Computational Linguistics · cited 1,040 times · Not summarized yet
    Abstract

    The dominant approach for many NLP tasks are recurrent neural networks, in particular LSTMs, and convolutional neural networks. However, these architectures are rather shallow in comparison to the deep convolutional networks which have pushed the state-of-the-art in computer vision. We present a new architecture (VDCNN) for text processing which operates directly at the character level and uses only small convolutions and pooling operations. We are able to show that the performance of this model increases with the depth: using up to 29 convolutional layers, we report improvements over the state-of-the-art on several public text classification tasks. To the best of our knowledge, this is the …

  5. Related A method it uses
    Text Understanding from Scratch
    Xiang Zhang, Yann LeCun · 2015 · arXiv.org · preprint, not peer reviewed · cited 580 times · Not summarized yet
    Abstract

    This article demontrates that we can apply deep learning to text understanding from character-level inputs all the way up to abstract text concepts, using temporal convolutional networks (ConvNets). We apply ConvNets to various large-scale datasets, including ontology classification, sentiment analysis, and text categorization. We show that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language. Evidence shows that our models can work for both English and Chinese.

Papers that cite it

  1. Related Uses this paper in depth
    Luxical: High-Speed Lexical-Dense Text Embeddings
    Luke Merrick, Alex Fang, A. Carranza et al. · 2025 · arXiv.org · preprint, not peer reviewed · cited 2 times · Not summarized yet
    Abstract

    Frontier language model quality increasingly hinges on our ability to organize web-scale text corpora for training. Today's dominant tools trade off speed and flexibility: lexical classifiers (e.g., FastText) are fast but limited to producing classification output scores, while the vector-valued outputs of transformer text embedding models flexibly support numerous workflows (e.g., clustering, classification, and retrieval) but are computationally expensive to produce. We introduce Luxical, a library for high-speed"lexical-dense"text embeddings that aims to recover the best properties of both approaches for web-scale text organization. Luxical combines sparse TF--IDF features, a small ReLU n…

  2. Related Uses this paper in depth
    Lightweight AST-Fused TextCNN
    Zhiqiang Lyu · 2026 · 2026 8th International Conference on Artificial Intelligence Technologies and Applications (ICAITA) · Not summarized yet
    Abstract

    To meet the low-cost educational AI requirement of ICAITA 2026 Track 1, this paper proposes CodeAware-TextCNN, a zero-GPU lightweight model for closed-set programming error classification on hybrid text-code student queries, clearly distinguished from intent classification, knowledge retrieval and generative LLM tutoring. Existing LLMs and transformer-based code pre-training models demand GPU resources and incur heavy $\mathbf{O}\left(\mathbf{N}^{\mathbf{2}}\right)$ self-attention overhead, while vanilla CNNs ignore hierarchical AST syntax and misclassify token-identical queries with distinct syntax bugs. Our model extracts lexical GloVe embeddings, dual-branch AST structural features and TF…

  3. Related
    Graph Convolutional Networks for Text Classification
    Liang Yao, Chengsheng Mao, Yuan Luo · 2019 · Proceedings of the AAAI Conference on Artificial Intelligence · cited 2,171 times · Not summarized yet
    Abstract

    Text classification is an important and classical problem in natural language processing. There have been a number of studies that applied convolutional neural networks (convolution on regular grid, e.g., sequence) to classification. However, only a limited number of studies have explored the more flexible graph convolutional neural networks (convolution on non-grid, e.g., arbitrary graph) for the task. In this work, we propose to use graph convolutional networks for text classification. We build a single text graph for a corpus based on word co-occurrence and document word relations, then learn a Text Graph Convolutional Network (Text GCN) for the corpus. Our Text GCN is initialized with on…

  4. Related
    Deep Pyramid Convolutional Neural Networks for Text Categorization
    Rie Johnson, Tong Zhang · 2017 · Proceedings of the conference - Association for Computational Linguistics. Meeting · cited 862 times · Not summarized yet
    Abstract

    This paper proposes a low-complexity word-level deep convolutional neural network (CNN) architecture for text categorization that can efficiently represent longrange associations in text.In the literature, several deep and complex neural networks have been proposed for this task, assuming availability of relatively large amounts of training data.However, the associated computational complexity increases as the networks go deeper, which poses serious challenges in practical applications.Moreover, it was shown recently that shallow word-level CNNs are more accurate and much faster than the state-of-the-art very deep nets such as character-level CNNs even in the setting of large training data.M…

  5. Related
    High-Dimensional Text Classification Using TF-IDF Features and Deep Neural Networks
    Sudeep S S, C. H, B. Gurubasavaraja et al. · 2026 · 2026 World Conference on Computational Science and Technology (WcCST) · Not summarized yet
    Abstract

    The rapid rise of digital text data in news articles, forums, and online communication has created the need to develop more accurate text classifying systems. Term-weighting techniques, such as Term Frequency-Inverse Document Frequency (TF-IDF), are the most commonly used text classifying techniques due to their simplicity and effectiveness; however, they are often faced with the challenge of high-dimensional noise, class imbalance, and class overlap in text data. To assess the effectiveness of the term weighting technique in modern text classifying systems, we propose the use of a regularized Multi-Layer Perceptron (MLP) classifier, which is trained using the TF-IDF feature representation o…

How are these papers found?

We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.

Updated on 6 November 2026.

The illustrated guide to the papers around a paper

The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.