# Contrastive language and vision learning of general fashion concepts

**Authors:** Patrick John Chia, Giuseppe Attanasio, Federico Bianchi, Silvia Terragni, Ana Rita Magalhães, Diogo Goncalves, Ciro Greco, Jacopo Tagliabue  
**arXiv:** [2204.03972](https://arxiv.org/abs/2204.03972)  
**Date:** 2022-04-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2204.03972v4/

## Key points

- The rise of online shopping has led to the development of complex ML and NLP models.
- Most models are designed for specialized supervised learning problems, limiting their transferability.
- FashionCLIP is a model that uses contrastive learning techniques to learn representations of fashion concepts.
- FashionCLIP can accurately retrieve similar fashion items based on descriptions or images.
- It achieves high accuracy in classifying fashion attributes or categories.
- FashionCLIP can localize specific regions within an image corresponding to certain attributes or categories.
- The authors have released their trained model and code to promote further research and collaboration in the field.
- Transferable representations improve the performance and applicability of ML and NLP models for online shopping.

## Summary

The steady rise of online shopping has led to the development of increasingly complex machine learning (ML) and natural language processing (NLP) models. However, most of these models are designed for specialized supervised learning problems, which limits their transferability across different domains. To address this limitation, the authors propose the use of more transferable representations of products in the fashion industry. In their work, they leverage recent advancements in contrastive learning to train a model called FashionCLIP. This model is inspired by CLIP (Contrastive Language-Image Pretraining), a state-of-the-art model that learns joint representations of images and text. By applying contrastive learning techniques specifically tailored for fashion data, FashionCLIP is able to learn rich and meaningful representations of fashion concepts. The authors demonstrate the capabilities of FashionCLIP in various tasks such as retrieval, classification, and grounding. In retrieval tasks, FashionCLIP can accurately retrieve similar fashion items based on textual descriptions or images. In classification tasks, it achieves high accuracy in classifying fashion attributes or categories. Additionally, FashionCLIP can ground its understanding of fashion concepts by localizing specific regions within an image that correspond to certain attributes or categories. To promote further research and collaboration in the field, the authors have released their trained model and code to the community. This enables other researchers and practitioners to utilize and build upon their work in developing more advanced ML and NLP models for the fashion industry. Overall, this study highlights the importance of transferable representations in improving the performance and applicability of ML and NLP models for online shopping. The development of FashionCLIP showcases how contrastive learning techniques can be effectively applied to learn meaningful representations in the context of fashion data. The availability of their model and code encourages further exploration and innovation in this domain.