InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generation
Auteurs : Chanran Kim, Jeongin Lee, Shichang Joung, Bongmo Kim, Yeul-Min Baek
Résumé scientifique
In the field of personalized image generation, the ability to create images preserving concepts has significantly improved. Creating an image that naturally integrates multiple concepts in a cohesive and visually appealing composition can indeed be challenging. This paper introduces "InstantFamily," an approach that employs a novel masked cross-attention mechanism and a multimodal embedding stack to achieve zero-shot multi-ID image generation. Our method effectively preserves ID as it utilizes global and local features from a pre-trained face recognition model integrated with text conditions. Additionally, our masked cross-attention mechanism enables the precise control of multi-ID and composition in the generated images. We demonstrate the effectiveness of InstantFamily through experiments showing its dominance in generating images with multi-ID, while resolving well-known multi-ID generation problems. Additionally, our model achieves state-of-the-art performance in both single-ID and multi-ID preservation. Furthermore, our model exhibits remarkable scalability with a greater number of ID preservation than it was originally trained with.
Les articles autour de lui, par date et par proximité
Plus haut, c’est plus proche de cet article ; à gauche, c’est plus ancien. Les traits relient les articles proches entre eux : les groupes de traits sont des sous-thèmes. Sélectionnez un point pour voir pourquoi il est là. Retour au résumé de cet article
Les articles autour de celui-ci
Classés selon la proximité de leur titre et de leur résumé avec ceux de cet article. Sélectionnez-en pour les lire ensemble.
Les articles les plus proches
-
InstantID: Zero-shot Identity-Preserving Generation in SecondsQi-Xun Wang, Xu Bai, Hao-Fan Wang et al. · 2024 · arXiv.org · preprint, non relu par les pairs · cité 557 fois · Pas encore résumé
Résumé scientifique
There has been significant progress in personalized image synthesis with methods such as Textual Inversion, DreamBooth, and LoRA. Yet, their real-world applicability is hindered by high storage demands, lengthy fine-tuning processes, and the need for multiple reference images. Conversely, existing ID embedding-based methods, while requiring only a single forward inference, face challenges: they either necessitate extensive fine-tuning across numerous model parameters, lack compatibility with community pre-trained models, or fail to maintain high face fidelity. Addressing these limitations, we introduce InstantID, a powerful diffusion model-based solution. Our plug-and-play module adeptly han…
-
DynamicID: Zero-Shot Multi-ID Image Personalization With Flexible Facial EditabilityXirui Hu, Jia-Hao Wang, Hao Chen et al. · 2025 · IEEE International Conference on Computer Vision · cité 21 fois · Pas encore résumé
Résumé scientifique
Recent advances in text-to-image generation have driven interest in generating personalized human images that depict specific identities from reference images. Although existing methods achieve high-fidelity identity preservation, they are generally limited to single-ID scenarios and offer insufficient facial editability. We present DynamicID, a tuningfree framework that inherently facilitates both single-ID and multi-ID personalized generation with high fidelity and flexible facial editability. Our key innovations include: 1) Semantic-Activated Attention (SAA), which employs query-level activation gating to minimize disruption to the base model when injecting ID features and achieve multi-I…
-
GroupPortrait: Multi-ID Portrait Generation with High Identity Preservation and Fine-Grained ControlMei-Jia Huang, Ruida Li, Bing Ma et al. · 2026 · IEEE Workshop/Winter Conference on Applications of Computer Vision · Pas encore résumé
Résumé scientifique
Identity-preserving portrait generation has achieved tremendous advancements with the development of diffusion models. However, multi-ID generation remains challenging due to degraded identity fidelity and insufficient control over layout, pose, and expression. To address these challenges, we propose GroupPortrait, a novel approach for multi-ID portrait generation with three key innovations:(1) LatentID for high-fidelity identity preservation, (2) Facial Controller enabling layout guidance and fine-grained facial control, and (3) Mask-Attention Controller allocating identity embeddings to specific facial regions. First, the LatentID module improves identity preservation by adding LatentID lo…
-
Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling ParadigmYi Wu, Ziqiang Li, Heliang Zheng et al. · 2024 · European Conference on Computer Vision · cité 46 fois · Pas encore résumé
Résumé scientifique
Drawing on recent advancements in diffusion models for text-to-image generation, identity-preserved personalization has made significant progress in accurately capturing specific identities with just a single reference image. However, existing methods primarily integrate reference images within the text embedding space, leading to a complex entanglement of image and text information, which poses challenges for preserving both identity fidelity and semantic consistency. To tackle this challenge, we propose Infinite-ID, an ID-semantics decoupling paradigm for identity-preserved personalization. Specifically, we introduce identity-enhanced training, incorporating an additional image cross-atten…
-
FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation NetworkJiayu Wang, Yue Yu, Jing-Jing Chen et al. · 2025 · AAAI Conference on Artificial Intelligence · cité 3 fois · Pas encore résumé
Résumé scientifique
Recent advances in diffusion-based generative models have demonstrated superior performance in subject-driven image generation. Identity (ID) preserving image generation, as a subtask of subject-driven image generation, aims to generate customized images for specific human identity and has broad application potential. However, this task remains challenging due to the requirement for high ID fidelity and precise detail preservation. Additionally, generating high-quality context presents another challenge, as existing methods struggle to achieve both high ID fidelity and satisfactory context simultaneously. To address the issues of insufficient ID fidelity, we introduce a simple yet effective …
-
Advancements in Text-to-Image Diffusion Models for Personalized Image Generation: A Review of ID-Preserving Techniques of InstantIDArdhian Kurniawan · 2024 · IC-ITECHS · Pas encore résumé
Résumé scientifique
The evolution of text-to-image diffusion models, such as GLIDE, DALL-E 2, and Stable Diffusion, has significantly enhanced image generation capabilities. However, achieving image personalization with precise facial detail retention, minimal reference images, and reduced computational costs remains challenging. Traditional methods like DreamBooth and Textual Inversion rely on extensive fine-tuning, while techniques like IP-Adapter, which avoid fine-tuning, often compromise accuracy. Addressing these gaps, InstantID introduces a novel plug-and-play module that uses a single reference image to enable efficient identity preservation with high fidelity and flexibility. InstantID departs from conv…
-
Pair-ID: A Dual Modal Framework for Identity Preserving Image GenerationJingyu Lin, Yongrong Wu, Zeyu Wang et al. · 2024 · IEEE Signal Processing Letters · cité 13 fois · Pas encore résumé
Résumé scientifique
The acquisition of large-scale paired visible and thermal images is crucial for enhancing face recognition systems, especially in low-light environments where visible spectrum images fail. However, the task is hindered by the scarcity of thermal images and the need for identity consistency during image generation. In this paper, we propose Pair-ID, an innovative framework that addresses these challenges by creating a shared latent space for simultaneous generation of paired visible and thermal images. Pair-ID integrates identity information into text embeddings and employs fixed templates for diverse facial poses, streamlining the customization process and reducing computational demands. The…
-
PhotoMaker: Customizing Realistic Human Photos via Stacked ID EmbeddingZhen Li, Ming-Deng Cao, Xin-Tao Wang et al. · 2023 · Computer Vision and Pattern Recognition · cité 434 fois · Pas encore résumé
Résumé scientifique
Recent advances in text-to-image generation have made remarkable progress in synthesizing realistic human photos conditioned on given text prompts. However, existing per-sonalized generation methods cannot simultaneously sat-isfy the requirements of high efficiency, promising identity (ID) fidelity, and flexible text controllability. In this work, we introduce PhotoMaker, an efficient personalized text-to-image generation method, which mainly encodes an arbitrary number of input ID images into a stack ID embed-ding for preserving ID information. Such an embedding, serving as a unified ID representation, can not only encap-sulate the characteristics of the same input ID comprehen-sively, but …
Les travaux antérieurs sur lesquels il s’appuie
-
PortraitBooth: A Versatile Portrait Model for Fast Identity-Preserved PersonalizationXu Peng, Jun-Wei Zhu, Bo-Yuan Jiang et al. · 2023 · Computer Vision and Pattern Recognition · cité 99 fois · Pas encore résumé
Résumé scientifique
Recent advancements in personalized image generation using diffusion models have been noteworthy. However, existing methods suffer from inefficiencies due to the requirement for subject-specific fine-tuning. This computationally intensive process hinders efficient deployment, limiting practical usability. Moreover, these methods often grapple with identity distortion and limited expression diversity. In light of these challenges, we propose Portrait-Booth, an innovative approach designed for high efficiency, robust identity preservation, and expression-editable text-to-image generation, without the need for fine-tuning. Por-traitBooth leverages subject embeddingsfrom aface recognition model …
-
DreamIdentity: Improved Editability for Efficient Face-identity Preserved Image GenerationZhuowei Chen, Shan-Cheng Fang, Wei Liu et al. · 2023 · arXiv.org · preprint, non relu par les pairs · cité 32 fois · Pas encore résumé
Résumé scientifique
While large-scale pre-trained text-to-image models can synthesize diverse and high-quality human-centric images, an intractable problem is how to preserve the face identity for conditioned face images. Existing methods either require time-consuming optimization for each face-identity or learning an efficient encoder at the cost of harming the editability of models. In this work, we present an optimization-free method for each face identity, meanwhile keeping the editability for text-to-image models. Specifically, we propose a novel face-identity encoder to learn an accurate representation of human faces, which applies multi-scale face features followed by a multi-embedding projector to direc…
-
OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion ModelsZhe Kong, Yong Zhang, Tianyu Yang et al. · 2024 · European Conference on Computer Vision · cité 84 fois · Pas encore résumé
Résumé scientifique
Personalization is an important topic in text-to-image generation, especially the challenging multi-concept personalization. Current multi-concept methods are struggling with identity preservation, occlusion, and the harmony between foreground and background. In this work, we propose OMG, an occlusion-friendly personalized generation framework designed to seamlessly integrate multiple concepts within a single image. We propose a novel two-stage sampling solution. The first stage takes charge of layout generation and visual comprehension information collection for handling occlusions. The second one utilizes the acquired visual comprehension information and the designed noise blending to inte…
-
FastComposer: Tuning-Free Multi-subject Image Generation with Localized AttentionGuang-Xuan Xiao, Tianwei Yin, W. Freeman et al. · 2023 · International Journal of Computer Vision · cité 439 fois · Pas encore résumé
Résumé scientifique
Diffusion models excel at text-to-image generation, especially in subject-driven generation for personalized images. However, existing methods are inefficient due to the subject-specific fine-tuning, which is computationally intensive and hampers efficient deployment. Moreover, existing methods struggle with multi-subject generation as they often blend identity among subjects. We present FastComposer which enables efficient, personalized, multi-subject text-to-image generation without fine-tuning. FastComposer uses subject embeddings extracted by an image encoder to augment the generic text conditioning in diffusion models, enabling personalized image generation based on subject images and t…
-
Face0: Instantaneously Conditioning a Text-to-Image Model on a FaceDani Valevski, Danny Lumen, Yossi Matias et al. · 2023 · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia · cité 107 fois · Pas encore résumé
Résumé scientifique
We present Face0, a novel way to instantaneously condition a text-to-image generation model on a face without any optimization procedures such as fine-tuning or inversions. We augment a dataset of annotated images with embeddings of the included faces and train an image generation model on the augmented dataset. Once trained, our system is practically identical at inference time to the underlying base model, and is therefore able to generate face-conditioned images in just a couple of seconds. Our method achieves pleasing results, is remarkably simple, extremely fast, and equips the underlying model with new capabilities, like controlling the generated images both via text or via direct mani…
Les articles qui le citent
-
ID-Patch: Robust ID Association for Group Photo PersonalizationYimeng Zhang, Tian-Cheng Zhi, Jing Liu et al. · 2024 · Computer Vision and Pattern Recognition · cité 12 fois · Pas encore résumé
Résumé scientifique
The ability to synthesize personalized group photos and specify the positions of each identity offers immense creative potential. While such imagery can be visually appealing, it presents significant challenges for existing technologies. A persistent issue is identity (ID) leakage, where injected facial features interfere with one another, resulting in low face resemblance, incorrect positioning, and visual artifacts. Existing methods suffer from limitations such as the reliance on segmentation models, increased runtime, or a high probability of ID leakage. To address these challenges, we propose ID-Patch, a novel method that provides robust association between identities and 2D positions. O…
-
RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global ComplementationZhaoyang Sun, Fei Du, Weihua Chen et al. · 2024 · AAAI Conference on Artificial Intelligence · cité 1 fois · Pas encore résumé
Résumé scientifique
Recently, the success of text-to-image synthesis has greatly advanced the development of identity customization techniques, whose main goal is to produce realistic identity-specific photographs based on text prompts and reference face images. However, it is difficult for existing identity customization methods to simultaneously meet the various requirements of different real-world applications, including the identity fidelity of small face, the control of face location, pose and expression, as well as the customization of multiple persons. To this end, we propose a scale-robust and fine-controllable method, namely RealisID, which learns different control capabilities through the cooperation …
-
MultiHuman-Testbench: Benchmarking Image Generation for Multiple HumansShubhankar Borse, Seokeon Choi, Sunghyun Park et al. · 2025 · Neural Information Processing Systems · cité 10 fois · Pas encore résumé
Résumé scientifique
Generation of images containing multiple humans, performing complex actions, while preserving their facial identities, is a significant challenge. A major factor contributing to this is the lack of a dedicated benchmark. To address this, we introduce MultiHuman-Testbench, a novel benchmark for rigorously evaluating generative models for multi-human generation. The benchmark comprises 1,800 samples, including carefully curated text prompts, describing a range of simple to complex human actions. These prompts are matched with a total of 5,550 unique human face images, sampled uniformly to ensure diversity across age, ethnic background, and gender. Alongside captions, we provide human-selected …
-
Training for Identity, Inference for Controllability: A Unified Approach to Tuning-Free Face PersonalizationLianyu Pang, Ji Zhou, Qiping Wang et al. · 2025 · International Conference on Multimedia Retrieval · Pas encore résumé
Résumé scientifique
Tuning-free face personalization methods have developed along two distinct paradigms: text embedding approaches that map facial features into the text embedding space, and adapter-based methods that inject features through auxiliary cross-attention layers. While both paradigms have shown promise, existing methods struggle to simultaneously achieve high identity fidelity and flexible text controllability. We introduce UniID, a unified tuning-free framework that synergistically integrates both paradigms. Our key insight is that when merging these approaches, they should mutually reinforce only identity-relevant information while preserving the original diffusion prior for non-identity attribut…
-
FPGA: Flexible Portrait Generation ApproachZhaoli Deng, Fan-Yi Wang, Jun-Kang Zhang et al. · 2024 · cité 2 fois · Pas encore résumé
Résumé scientifique
Portrait Fidelity Generation is a prominent research area in generative models.Current methods face challenges in generating full-body images with low-resolution faces, especially in multi-ID photo phenomenon.To tackle these issues, we propose a comprehensive system called FPGA and construct a million-level multi-modal dataset IDZoom for training.FPGA consists of Multi-Mode Fusion training strategy (MMF) and DDIM Inversion based ID Restoration inference framework (DIIR). The MMF aims to activate the specified ID in the specified facial region. The DIIR aims to address the issue of face artifacts while keeping the background.Furthermore, DIIR is plug-and-play and can be applied to any diffusi…
Aller plus loin
Comment ces articles sont-ils trouvés ?
Nous comparons le titre et le résumé de cet article avec ceux de tous les articles résumés sur ce site, et avec les articles que Semantic Scholar recommande, ceux qu’il cite et ceux qui le citent (pour un article très cité, ses citations les plus récentes et, depuis OpenAlex, les plus citées). Parmi les travaux antérieurs et les suites, les articles qui s’utilisent en profondeur, une méthode qu’il utilise et les suites très citées remontent un peu. Les niveaux (très proche, proche, apparenté) viennent de la similarité des textes, pas d’un jugement de qualité ; * signale un article sans résumé, comparé par son titre. La phrase sous un article est écrite par une IA à partir des deux résumés : vérifiez-la dans les articles. Un article jugé hors sujet par trois lecteurs quitte la liste.
Mis à jour le 3 novembre 2026.