Dual Optimal Transport for Multi-Concept Composition: Structural Alignment and Texture Injection in Diffusion Models

Abstract

Diffusion models have shown impressive capabilities in text-to-image synthesis. However, multi-concept personalized generation remains challenging, particularly in aligning multiple reference concepts while preserving fidelity. To address this, we propose a novel Sketch-to-Rendering framework that leverages $\textit{Dual Optimal Transport (OT)}$ for structure alignment and texture injection. Our approach consists of two key components: $\textit{Structure Sketching via Barycentric Soft-Transport}$, which ensures shape alignment by using mass-preserving OT for spatial consistency, and $\textit{Texture Rendering via Geometry-Guided Transport}$, which leverages low-frequency structure alignment to inject high-frequency texture details via OT-based residual transfer, thereby preserving texture fidelity without distorting structure. Extensive experiments demonstrate that our method significantly enhances both conceptual fidelity and visual quality. Ablation studies further validate the effectiveness of our optimal transport guidance and the decoupling of structure and texture during the generation process. Our code is available at https://github.com/fuhao7i/OTComp.