Personalized Visual Content Generation in Conversational Systems

Kai Zhang (The Ohio State University) · Zhaocheng Du (Huawei Technologies Ltd.) · Qi Liu (Bytedance Inc.) · Jieming Zhu (Huawei Noah's Ark Lab) · Xianquan Wang (University of Science and Technology of China) · Huibo Xu (University of Science and Technology of China) · Shukang Yin (University of Science and Technology of China) · Yupeng Han
benchmark conversational datasetsconversational systemsdiffusion modelsdual-stage lora mechanismfidelity of generated imageshistorical visual contextinclinations analyzerpersonalized content generationpersonalized promptspersonalized visual content generationpreferred visual elementstask-specific visual styleuser engagementvisual content condition method

With the rapid progress of large language models (LLMs) and diffusion models, there has been growing interest in personalized content generation. However, current conversational systems often present the same recommended content to all users, falling into the dilemma of "one-size-fits-all." To break this limitation and boost user engagement, in this paper, we introduce PCG (**P**ersonalized Visual **C**ontent **G**eneration), a unified framework for personalizing item images within conversational systems. We tackle two key bottlenecks: the depth of personalization and the fidelity of generated images. Specifically, an LLM-powered Inclinations Analyzer is adopted to capture user likes and dislikes from context to construct personalized prompts. Moreover, we design a dual-stage LoRA mechanism—Global LoRA for understanding task-specific visual style, and Local LoRA for capturing preferred visual elements from conversation history. During training, we introduce the visual content condition method to ensure LoRA learns both historical visual context and maintains fidelity to the original item images. Extensive experiments on benchmark conversational datasets—including objective metrics and GPT-based evaluations—demonstrate that our framework outperforms strong baselines, which highlight its potential to redefine personalization in visual content generation for conversational scenarios like e-commerce and real-world recommendation.