Diffusion Models already have a Semantic Latent Space

Diffusion models have been successful in achieving outstanding generative performance across various domains. However, they lack a refined semantic latent space that is crucial for controlling the generative process. To address this issue, Mingi Kwon, Jaeseok Jeong, and Youngjung Uh propose an asymmetric reverse process (Asyrp) that discovers the semantic latent space in frozen pretrained diffusion models. Their proposed semantic latent space, named h-space, has several desirable properties for accommodating semantic image manipulation such as homogeneity, linearity, robustness and consistency across timesteps. The authors also introduce a principled design of the generative process that allows for versatile editing and quality boosting by quantifiable measures such as editing strength of an interval and quality deficiency at a timestep. The Asyrp method is applicable to various architectures such as DDPM++, iD-DPM and ADM as well as datasets including CelebA-HQ, AFHQ-dog, LSUN-church, LSUN-bedroom and METFACES. The authors' project page provides further details on their work.

Exploring the Semantic Latent Space with Asyrp

Diffusion models have been successful in achieving outstanding generative performance across various domains. However, they lack a refined semantic latent space that is crucial for controlling the generative process. To address this issue, Mingi Kwon, Jaeseok Jeong, and Youngjung Uh propose an asymmetric reverse process (Asyrp) that discovers the semantic latent space in frozen pretrained diffusion models.

Introducing h-space: A Refined Semantic Latent Space

The authors introduce a new semantic latent space called h-space which has several desirable properties for accommodating semantic image manipulation such as homogeneity, linearity, robustness and consistency across timesteps. The proposed h-space consists of two components: a low-dimensional vector representation of the input image and a high-dimensional feature map generated by convolutional layers of pretrained diffusion models. The low dimensional vector is used to represent the global structure of an image while the high dimensional feature map captures its local details. This allows for more precise control over both global and local features when manipulating images semantically.

Designing Generative Processes with Asyrp

The authors also introduce a principled design of the generative process that allows for versatile editing and quality boosting by quantifiable measures such as editing strength of an interval and quality deficiency at a timestep. The Asyrp method is applicable to various architectures such as DDPM++, iD-DPM and ADM as well as datasets including CelebA-HQ, AFHQ-dog, LSUN-church, LSUN-bedroom and METFACES. By introducing these quantitative measures into their generative processes design they are able to achieve better results than traditional methods like GANs or VAEs which rely on qualitative metrics only.

Conclusion

Mingi Kwon et al.'s work introduces an innovative approach to discovering semantic latent spaces in frozen pretrained diffusion models using their proposed asymmetric reverse process (Asyrp). Their proposed h-space has several desirable properties for accommodating semantic image manipulation while their design of generative processes allows for versatile editing and quality boosting by quantifiable measures such as editing strength of an interval and quality deficiency at a timestep. For further details on their work please visit their project page linked below!

Link to Project Page: https://mingikwon0309.github.io/asyrp/