InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generation

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Not relevant

   InstantID: Zero-shot Identity-Preserving Generation in Seconds

   Qi-Xun Wang, Xu Bai, Hao-Fan Wang et al. · 2024 · arXiv.org · preprint, not peer reviewed · cited 557 times · Not summarized yet

   Abstract

   There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly han…

2. Very close Cites this paper Not relevant

   DynamicID: Zero-Shot Multi-ID Image Personalization With Flexible Facial Editability

   Xirui Hu, Jia-Hao Wang, Hao Chen et al. · 2025 · IEEE International Conference on Computer Vision · cited 21 times · Not summarized yet

   Abstract

   Recent advances in text-to-image generation have driven interest in generating personalized human images that depict specific identities from reference images. Although existing methods achieve high-fidelity identity preservation, they are generally limited to single-ID scenarios and offer insufficient facial editability. We present DynamicID, a tuningfree framework that inherently facilitates both single-ID and multi-ID personalized generation with high fidelity and flexible facial editability. Our key innovations include: 1) Semantic-Activated Attention (SAA), which employs query-level activation gating to minimize disruption to the base model when injecting ID features and achieve multi-I…

3. Very close Cites this paper Not relevant

   GroupPortrait: Multi-ID Portrait Generation with High Identity Preservation and Fine-Grained Control

   Mei-Jia Huang, Ruida Li, Bing Ma et al. · 2026 · IEEE Workshop/Winter Conference on Applications of Computer Vision · Not summarized yet

   Abstract

   Identity-preserving portrait generation has achieved tremendous advancements with the development of diffusion models. However, multi-ID generation remains challenging due to degraded identity fidelity and insufficient control over layout, pose, and expression. To address these challenges, we propose GroupPortrait, a novel approach for multi-ID portrait generation with three key innovations:(1) LatentID for high-fidelity identity preservation, (2) Facial Controller enabling layout guidance and fine-grained facial control, and (3) Mask-Attention Controller allocating identity embeddings to specific facial regions. First, the LatentID module improves identity preservation by adding LatentID lo…

4. Very close Cited by this paper A method it uses Not relevant

   Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm

   Yi Wu, Ziqiang Li, Heliang Zheng et al. · 2024 · European Conference on Computer Vision · cited 46 times · Not summarized yet

   Abstract

   Drawing on recent advancements in diffusion models for text-to-image generation, identity-preserved personalization has made significant progress in accurately capturing specific identities with just a single reference image. However, existing methods primarily integrate reference images within the text embedding space, leading to a complex entanglement of image and text information, which poses challenges for preserving both identity fidelity and semantic consistency. To tackle this challenge, we propose Infinite-ID, an ID-semantics decoupling paradigm for identity-preserved personalization. Specifically, we introduce identity-enhanced training, incorporating an additional image cross-atten…

5. Very close Not relevant

   FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network

   Jiayu Wang, Yue Yu, Jing-Jing Chen et al. · 2025 · AAAI Conference on Artificial Intelligence · cited 3 times · Not summarized yet

   Abstract

   Recent advances in diffusion-based generative models have demonstrated superior performance in subject-driven image generation. Identity (ID) preserving image generation, as a subtask of subject-driven image generation, aims to generate customized images for specific human identity and has broad application potential. However, this task remains challenging due to the requirement for high ID fidelity and precise detail preservation. Additionally, generating high-quality context presents another challenge, as existing methods struggle to achieve both high ID fidelity and satisfactory context simultaneously. To address the issues of insufficient ID fidelity, we introduce a simple yet effective …

6. Very close Not relevant

   Advancements in Text-to-Image Diffusion Models for Personalized Image Generation: A Review of ID-Preserving Techniques of InstantID

   Ardhian Kurniawan · 2024 · IC-ITECHS · Not summarized yet

   Abstract

   The evolution of text-to-image diffusion models, such as GLIDE, DALL-E 2, and Stable Diffusion, has significantly enhanced image generation capabilities. However, achieving image personalization with precise facial detail retention, minimal reference images, and reduced computational costs remains challenging. Traditional methods like DreamBooth and Textual Inversion rely on extensive fine-tuning, while techniques like IP-Adapter, which avoid fine-tuning, often compromise accuracy. Addressing these gaps, InstantID introduces a novel plug-and-play module that uses a single reference image to enable efficient identity preservation with high fidelity and flexibility. InstantID departs from conv…

7. Very close Not relevant

   Pair-ID: A Dual Modal Framework for Identity Preserving Image Generation

   Jingyu Lin, Yongrong Wu, Zeyu Wang et al. · 2024 · IEEE Signal Processing Letters · cited 13 times · Not summarized yet

   Abstract

   The acquisition of large-scale paired visible and thermal images is crucial for enhancing face recognition systems, especially in low-light environments where visible spectrum images fail. However, the task is hindered by the scarcity of thermal images and the need for identity consistency during image generation. In this paper, we propose Pair-ID, an innovative framework that addresses these challenges by creating a shared latent space for simultaneous generation of paired visible and thermal images. Pair-ID integrates identity information into text embeddings and employs fixed templates for diverse facial poses, streamlining the customization process and reducing computational demands. The…

8. Very close Cited by this paper A method it uses Not relevant

   PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding

   Zhen Li, Ming-Deng Cao, Xin-Tao Wang et al. · 2023 · Computer Vision and Pattern Recognition · cited 434 times · Not summarized yet

   Abstract

   Recent advances in text-to-image generation have made remarkable progress in synthesizing realistic human photos conditioned on given text prompts. However, existing per-sonalized generation methods cannot simultaneously sat-isfy the requirements of high efficiency, promising identity (ID) fidelity, and flexible text controllability. In this work, we introduce PhotoMaker, an efficient personalized text-to-image generation method, which mainly encodes an arbitrary number of input ID images into a stack ID embed-ding for preserving ID information. Such an embedding, serving as a unified ID representation, can not only encap-sulate the characteristics of the same input ID comprehen-sively, but …

Prior work it builds on

1. Close A method it uses Not relevant

   PortraitBooth: A Versatile Portrait Model for Fast Identity-Preserved Personalization

   Xu Peng, Jun-Wei Zhu, Bo-Yuan Jiang et al. · 2023 · Computer Vision and Pattern Recognition · cited 99 times · Not summarized yet

   Abstract

   Recent advancements in personalized image generation using diffusion models have been noteworthy. However, existing methods suffer from inefficiencies due to the requirement for subject-specific fine-tuning. This computationally intensive process hinders efficient deployment, limiting practical usability. Moreover, these methods often grapple with identity distortion and limited expression diversity. In light of these challenges, we propose Portrait-Booth, an innovative approach designed for high efficiency, robust identity preservation, and expression-editable text-to-image generation, without the need for fine-tuning. Por-traitBooth leverages subject embeddingsfrom aface recognition model …

2. Close A method it uses Not relevant

   DreamIdentity: Improved Editability for Efficient Face-identity Preserved Image Generation

   Zhuowei Chen, Shan-Cheng Fang, Wei Liu et al. · 2023 · arXiv.org · preprint, not peer reviewed · cited 32 times · Not summarized yet

   Abstract

   While large-scale pre-trained text-to-image models can synthesize diverse and high-quality human-centric images, an intractable problem is how to preserve the face identity for conditioned face images. Existing methods either require time-consuming optimization for each face-identity or learning an efficient encoder at the cost of harming the editability of models. In this work, we present an optimization-free method for each face identity, meanwhile keeping the editability for text-to-image models. Specifically, we propose a novel face-identity encoder to learn an accurate representation of human faces, which applies multi-scale face features followed by a multi-embedding projector to direc…

3. Close A method it uses Not relevant

   OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models

   Zhe Kong, Yong Zhang, Tianyu Yang et al. · 2024 · European Conference on Computer Vision · cited 84 times · Not summarized yet

   Abstract

   Personalization is an important topic in text-to-image generation, especially the challenging multi-concept personalization. Current multi-concept methods are struggling with identity preservation, occlusion, and the harmony between foreground and background. In this work, we propose OMG, an occlusion-friendly personalized generation framework designed to seamlessly integrate multiple concepts within a single image. We propose a novel two-stage sampling solution. The first stage takes charge of layout generation and visual comprehension information collection for handling occlusions. The second one utilizes the acquired visual comprehension information and the designed noise blending to inte…

4. Close Used in depth by this paper A method it uses Not relevant

   FastComposer: Tuning-Free Multi-subject Image Generation with Localized Attention

   Guang-Xuan Xiao, Tianwei Yin, W. Freeman et al. · 2023 · International Journal of Computer Vision · cited 439 times · Not summarized yet

   Abstract

   Diffusion models excel at text-to-image generation, especially in subject-driven generation for personalized images. However, existing methods are inefficient due to the subject-specific fine-tuning, which is computationally intensive and hampers efficient deployment. Moreover, existing methods struggle with multi-subject generation as they often blend identity among subjects. We present FastComposer which enables efficient, personalized, multi-subject text-to-image generation without fine-tuning. FastComposer uses subject embeddings extracted by an image encoder to augment the generic text conditioning in diffusion models, enabling personalized image generation based on subject images and t…

5. Close A method it uses Not relevant

   Face0: Instantaneously Conditioning a Text-to-Image Model on a Face

   Dani Valevski, Danny Lumen, Yossi Matias et al. · 2023 · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia · cited 107 times · Not summarized yet

   Abstract

   We present Face0, a novel way to instantaneously condition a text-to-image generation model on a face without any optimization procedures such as fine-tuning or inversions. We augment a dataset of annotated images with embeddings of the included faces and train an image generation model on the augmented dataset. Once trained, our system is practically identical at inference time to the underlying base model, and is therefore able to generate face-conditioned images in just a couple of seconds. Our method achieves pleasing results, is remarkably simple, extremely fast, and equips the underlying model with new capabilities, like controlling the generated images both via text or via direct mani…

Papers that cite it

1. Close Uses this paper in depth Not relevant

   ID-Patch: Robust ID Association for Group Photo Personalization

   Yimeng Zhang, Tian-Cheng Zhi, Jing Liu et al. · 2024 · Computer Vision and Pattern Recognition · cited 12 times · Not summarized yet

   Abstract

   The ability to synthesize personalized group photos and specify the positions of each identity offers immense creative potential. While such imagery can be visually appealing, it presents significant challenges for existing technologies. A persistent issue is identity (ID) leakage, where injected facial features interfere with one another, resulting in low face resemblance, incorrect positioning, and visual artifacts. Existing methods suffer from limitations such as the reliance on segmentation models, increased runtime, or a high probability of ID leakage. To address these challenges, we propose ID-Patch, a novel method that provides robust association between identities and 2D positions. O…

2. Close Not relevant

   RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global Complementation

   Zhaoyang Sun, Fei Du, Weihua Chen et al. · 2024 · AAAI Conference on Artificial Intelligence · cited 1 time · Not summarized yet

   Abstract

   Recently, the success of text-to-image synthesis has greatly advanced the development of identity customization techniques, whose main goal is to produce realistic identity-specific photographs based on text prompts and reference face images. However, it is difficult for existing identity customization methods to simultaneously meet the various requirements of different real-world applications, including the identity fidelity of small face, the control of face location, pose and expression, as well as the customization of multiple persons. To this end, we propose a scale-robust and fine-controllable method, namely RealisID, which learns different control capabilities through the cooperation …

3. Close Uses this paper in depth Not relevant

   MultiHuman-Testbench: Benchmarking Image Generation for Multiple Humans

   Shubhankar Borse, Seokeon Choi, Sunghyun Park et al. · 2025 · Neural Information Processing Systems · cited 10 times · Not summarized yet

   Abstract

   Generation of images containing multiple humans, performing complex actions, while preserving their facial identities, is a significant challenge. A major factor contributing to this is the lack of a dedicated benchmark. To address this, we introduce MultiHuman-Testbench, a novel benchmark for rigorously evaluating generative models for multi-human generation. The benchmark comprises 1,800 samples, including carefully curated text prompts, describing a range of simple to complex human actions. These prompts are matched with a total of 5,550 unique human face images, sampled uniformly to ensure diversity across age, ethnic background, and gender. Alongside captions, we provide human-selected …

4. Close Not relevant

   Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face Personalization

   Lianyu Pang, Ji Zhou, Qiping Wang et al. · 2025 · International Conference on Multimedia Retrieval · Not summarized yet

   Abstract

   Tuning-free face personalization methods have developed along two distinct paradigms: text embedding approaches that map facial features into the text embedding space, and adapter-based methods that inject features through auxiliary cross-attention layers. While both paradigms have shown promise, existing methods struggle to simultaneously achieve high identity fidelity and flexible text controllability. We introduce UniID, a unified tuning-free framework that synergistically integrates both paradigms. Our key insight is that when merging these approaches, they should mutually reinforce only identity-relevant information while preserving the original diffusion prior for non-identity attribut…

5. Close Not relevant

   FPGA: Flexible Portrait Generation Approach

   Zhaoli Deng, Fan-Yi Wang, Jun-Kang Zhang et al. · 2024 · cited 2 times · Not summarized yet

   Abstract

   Portrait Fidelity Generation is a prominent research area in generative models.Current methods face challenges in generating full-body images with low-resolution faces, especially in multi-ID photo phenomenon.To tackle these issues, we propose a comprehensive system called FPGA and construct a million-level multi-modal dataset IDZoom for training.FPGA consists of Multi-Mode Fusion training strategy (MMF) and DDIM Inversion based ID Restoration inference framework (DIIR). The MMF aims to activate the specified ID in the specified facial region. The DIIR aims to address the issue of face artifacts while keeping the background.Furthermore, DIIR is plug-and-play and can be applied to any diffusi…