# Controllable Multi-domain Semantic Artwork Synthesis

**Authors:** Yuantian Huang, Satoshi Iizuka, Edgar Simo-Serra, Kazuhiro Fukui  
**arXiv:** [2308.10111](https://arxiv.org/abs/2308.10111)  
**Date:** 2023-08-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2308.10111v1/

## Key points

- Authors present a novel framework for multi-domain synthesis of artwork from semantic layouts
- They propose a dataset called ArtSem consisting of 40,000 images of artwork from four different domains along with their corresponding semantic label maps
- Conditional GAN-based approach used to generate high-quality artwork from extracted semantic maps without paired training data
- Introduction of domain-dependent variational encoders for high-quality multi-domain synthesis
- Introduction of Spatially STyle-Adaptive Normalization (SSTAN) method to normalize both semantic and style jointly for better generation quality
- Model learns joint representation of style and semantic information, allowing fine-grained control over synthesized artwork
- Combination of proposed dataset and approach leads to higher quality user-controllable artwork compared to existing methods
- Contributes to computational visual media field by providing framework for controllable multi-domain semantic artwork synthesis

## Summary

In this research article, the authors present a novel framework for multi-domain synthesis of artwork from semantic layouts. They address the main limitation of this task - the lack of publicly available segmentation datasets for art synthesis - by proposing a dataset called ArtSem. This dataset consists of 40,000 images of artwork from four different domains along with their corresponding semantic label maps. To generate the dataset, the authors first extract semantic maps from landscape photography and then propose a conditional Generative Adversarial Network (GAN)-based approach to generate high-quality artwork from these semantic maps without requiring paired training data. Additionally, they introduce an artwork synthesis model that utilizes domain-dependent variational encoders for high-quality multi-domain synthesis. The authors further enhance their model by introducing a normalization method called Spatially STyle-Adaptive Normalization (SSTAN). This method normalizes both the semantic and style jointly, leading to better generation quality when synthesizing artistic images. Unlike previous methods that only consider semantic layout as input, their model learns a joint representation of both style and semantic information. The results indicate that the authors' model successfully learns to separate the domains in the latent space. By identifying the hyperplanes that separate different domains, fine-grained control over synthesized artwork can be achieved. By combining their proposed dataset and approach, they are able to generate user-controllable artwork of higher quality than existing methods. Overall, this research contributes to the field of computational visual media by providing a framework for controllable multi-domain semantic artwork synthesis. The proposed dataset and approach offer new possibilities for generating high-quality artistic images based on semantic layouts while considering both style and semantic information.