# Enriching Word Vectors with Subword Information

**Authors:** Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov  
**arXiv:** [1607.04606](https://arxiv.org/abs/1607.04606)  
**Date:** 2016-07-15  
**Summary page:** https://www.summarizepaper.com/en/paper/1607.04606v1/

## Key points

- Importance of continuous word representations in natural language processing tasks
- Limitations of existing models that do not consider word morphology
- Proposal of a novel method based on the skip-gram model
- Representation of each word as a collection of character n-grams
- Derivation of word representation as the sum of individual character n-gram vectors
- Benefits including capturing morphological structure and faster training on large corpora
- Evaluation through testing on word similarity and analogy tasks in five languages
- Results showing significant enhancement in quality of word representations with subword information integration

## Summary

In their paper titled "Enriching Word Vectors with Subword Information," authors Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov discuss the importance of continuous word representations in natural language processing tasks. They highlight that many existing models for learning word representations do not take into account the morphology of words, instead assigning a unique vector to each word. This approach poses limitations, especially for languages with complex morphologies and extensive vocabularies containing numerous rare words. To address this issue, the authors propose a novel method based on the skip-gram model. In this approach, each word is represented as a collection of character n-grams, with a vector representation assigned to each character n-gram. The word representation is then derived as the sum of these individual character n-gram vectors. This technique not only captures the morphological structure of words but also allows for faster training on large corpora. The authors evaluate the effectiveness of their proposed method by testing it on word similarity and analogy tasks in five different languages. Their results demonstrate that incorporating subword information significantly enhances the quality of word representations, particularly in languages with intricate morphologies and diverse vocabularies. Overall, this innovative approach offers a promising solution for improving natural language processing tasks by enriching word vectors with subword information.