# SFNet: Learning Object-aware Semantic Correspondence

**Authors:** Junghyup Lee, Dohyung Kim, Jean Ponce, Bumsub Ham  
**arXiv:** [1904.01810](https://arxiv.org/abs/1904.01810)  
**Date:** 2019-04-03  
**Summary page:** https://www.summarizepaper.com/en/paper/1904.01810v1/

## Key points

- Paper title: "SFNet: Learning Object-aware Semantic Correspondence"
- Addresses the problem of establishing semantic correspondence between images depicting different instances of the same object or scene category
- Proposes a novel approach that uses images annotated with binary foreground masks and synthetic geometric deformations to train a CNN
- Incorporates masks into the supervisory signal to strike a balance between semantic flow methods and semantic alignment methods
- Introduces SFNet, a new CNN architecture that leverages a differentiable version of the argmax function for end-to-end training
- Loss function combines mask and flow consistency with smoothness terms
- Experimental results show significant improvement over state-of-the-art methods on standard benchmarks
- Presents an innovative solution by utilizing binary foreground masks and synthetic deformations in training a CNN
- SFNet architecture offers improvements in establishing dense flow fields between similar objects or scenes.

## Summary

The paper titled "SFNet: Learning Object-aware Semantic Correspondence" addresses the problem of establishing semantic correspondence between images depicting different instances of the same object or scene category. The authors propose a novel approach that utilizes images annotated with binary foreground masks and synthetic geometric deformations to train a convolutional neural network (CNN) for this task. By incorporating these masks into the supervisory signal, the proposed method strikes a balance between semantic flow methods, which are limited by manually selecting point correspondences, and semantic alignment methods, which may be sensitive to image-specific details. To implement their idea, the authors introduce SFNet, a new CNN architecture that leverages a differentiable version of the argmax function for end-to-end training. The loss function of SFNet combines mask and flow consistency with smoothness terms. Experimental results demonstrate the effectiveness of their approach as it significantly outperforms state-of-the-art methods on standard benchmarks. Overall, this paper presents an innovative solution to the problem of semantic correspondence by utilizing binary foreground masks and synthetic deformations in training a CNN. The proposed SFNet architecture shows promising results and offers improvements over existing techniques in establishing dense flow fields between images depicting similar objects or scenes.