# VTON-IT: Virtual Try-On using Image Translation

**Authors:** Santosh Adhikari, Bishnu Bhusal, Prashant Ghimire, Anil Shrestha  
**arXiv:** [2310.04558](https://arxiv.org/abs/2310.04558)  
**Date:** 2023-10-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.04558v2/

## Key points

- Virtual Try-On (VTON) is a promising application of Generative Adversarial Networks (GANs)
- Challenges include transferring clothing items onto different body sizes, poses, and occlusions
- VTON-IT (Virtual Try-On using Image Translation) utilizes semantic segmentation and generative adversarial architecture-based image translation network
- Evaluation metrics used include Structural Similarity Index (SSIM), Multi-Scale Structural Similarity (MS-SSIM), Fréchet Inception Distance (FID), and Kernel Inspection Distance (KID) scores
- VTON-IT outperformed existing approaches in producing high-resolution natural images with detailed textures on variant images
- User study showed 70% similarity to ground truth images and 60% photo-realism
- Challenges faced in training human body segmentation network due to improper annotations in existing datasets
- Future work may involve expanding the application of VTON-IT to include different types of clothing items

## Summary

, , , , 

Virtual Try-On, also known as trying clothes virtually, is a promising application of Generative Adversarial Networks (GANs). However, it poses challenges in transferring clothing items onto different body sizes, poses, and occlusions such as hair and overlapped clothes. To address this issue, a novel approach called VTON-IT (Virtual Try-On using Image Translation) is introduced in this paper. It utilizes semantic segmentation and a generative adversarial architecture-based image translation network to produce photo-realistic translated images. The proposed method takes an RGB image, segments the desired body part, and overlays the target cloth over the segmented body region. To evaluate the effectiveness of VTON-IT, various quantitative metrics such as Structural Similarity Index (SSIM), Multi-Scale Structural Similarity (MS-SSIM), Fréchet Inception Distance (FID), and Kernel Inspection Distance (KID) scores were used to measure the similarity between ground truth images manually wrapped with clothes and synthesized images generated by the model. The results showed that VTON-IT outperformed existing approaches in producing high-resolution natural images with detailed textures on variant images. Additionally, a user study involving 60 volunteers was conducted to assess the realism and visual quality of the synthesized images. Volunteers were asked to score based on how real the clothes looked on the person and how well the texture of the clothing was preserved. The results indicated that VTON-IT achieved a 70% similarity to ground truth images and 60% photo-realism. The paper also discusses the challenges faced in training a human body segmentation network due to improper annotations in existing datasets. To address this issue, 6000 high-quality images from the FGC6 dataset were manually curated for training purposes. In conclusion, VTON-IT presents an innovative solution for Virtual Try-On applications by effectively transferring clothing onto human images while considering variations in body size, pose, and lighting conditions. The proposed architecture demonstrates superior performance in generating natural-looking synthesized images compared to existing methods. Future work may involve expanding the application of VTON-IT to include different types of clothing items such as dresses, shorts, shoes, and beyond.