# Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

**Authors:** Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, Christian Theobalt  
**arXiv:** [2305.10973](https://arxiv.org/abs/2305.10973)  
**Date:** 2023-05-18  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.10973v1/

## Key points

- Synthesizing visual content that meets users' needs requires precise controllability of pose, shape, expression, and layout of generated objects.
- Existing approaches for controlling generative adversarial networks (GANs) lack flexibility, precision, and generality.
- The authors propose a novel approach called DragGAN that enables interactive point-based manipulation on the generative image manifold of a GAN.
- DragGAN consists of two main components: feature-based motion supervision and a new point tracking approach.
- Through DragGAN's interactive manipulation capabilities, anyone can deform an image with precise control over where pixels go and manipulate diverse categories such as animals, cars, humans, landscapes with ease.
- Both qualitative and quantitative comparisons demonstrate the advantage of DragGAN over prior approaches in tasks such as image manipulation and point tracking.
- The proposed approach also showcases the manipulation of real images through GAN inversion.
- This work was supported by Saarbrücken Research Center for Visual Computing Interaction and AI along with Christian Theobalt's ERC Consolidator Grant 4DReply (770784), while Lingjie Liu received Lise Meitner Postdoctoral Fellowship.
- The authors presented their findings at SIGGRAPH '23 Conference P where they demonstrated how DragGAN provides flexible and accurate image manipulation capabilities compared to prior approaches.
- Overall, this study offers a powerful yet much less explored way of controlling GANs that has significant potential for various applications in computer vision research fields.

## Summary

The ability to synthesize visual content that meets users' needs often requires precise controllability of the pose, shape, expression, and layout of generated objects. However, existing approaches for controlling generative adversarial networks (GANs) lack flexibility, precision, and generality. In this work, the authors propose a novel approach called DragGAN that enables interactive point-based manipulation on the generative image manifold of a GAN. The method allows users to "drag" any points of an image to precisely reach target points in a user-interactive manner. DragGAN consists of two main components: feature-based motion supervision that drives handle points towards target positions and a new point tracking approach that leverages discriminative generator features to localize handle points' position accurately. Through DragGAN's interactive manipulation capabilities, anyone can deform an image with precise control over where pixels go and manipulate diverse categories such as animals, cars, humans, landscapes with ease. The learned generative image manifold of GANs produces realistic outputs even for challenging scenarios such as hallucinating occluded content or deforming shapes consistently following object rigidity. Both qualitative and quantitative comparisons demonstrate the advantage of DragGAN over prior approaches in tasks such as image manipulation and point tracking. The proposed approach also showcases the manipulation of real images through GAN inversion. This work was supported by Saarbrücken Research Center for Visual Computing Interaction and AI along with Christian Theobalt's ERC Consolidator Grant 4DReply (770784), while Lingjie Liu received Lise Meitner Postdoctoral Fellowship. The authors presented their findings at SIGGRAPH '23 Conference P where they demonstrated how DragGAN provides flexible and accurate image manipulation capabilities compared to prior approaches. Overall, this study offers a powerful yet much less explored way of controlling GANs that has significant potential for various applications in computer vision research fields.