XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulation

Xu Wang (Shanghai Jiao Tong University) · Xinglong Wu (ByteDance Inc.) · Li Chen (Zhongguancun Laboratory) · Bowen Chen (Tsinghua University) · Brynn zhao (Beijing University of Aeronautics and Astronautics) · Haomiao Sun (ByteDance) · Daniel Du (ByteDance Inc.)

attribute entanglementcoherencecomplex scene generationdiffusion transformerseditabilityeditable image synthesisfine-grained control high-fidelity synthesis image latentsmulti-subject controlled generationpersonalized scene generationprecise controlsemantic attributes text-to-image generationtoken-specific text-stream modulation

Achieving fine-grained control over subject identity and semantic attributes (pose, style, lighting) in text-to-image generation, particularly for multiple subjects, often undermines the editability and coherence of Diffusion Transformers (DiTs). Many approaches introduce artifacts or suffer from attribute entanglement. To overcome these challenges, we propose a novel multi-subject controlled generation model XVerse. By transforming reference images into offsets for token-specific text-stream modulation, XVerse allows for precise and independent control for specific subject without disrupting image latents or features. Consequently, XVerse offers high-fidelity, editable multi-subject image synthesis with robust control over individual subject characteristics and semantic attributes. This advancement significantly improves personalized and complex scene generation capabilities.