OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation

Rui Wang (Shanghai Jiao Tong University) · Xianfang Zeng (Stepfun) · Wei Cheng (NEC Labs America) · Gang Yu (Megvii Inc) · Jingjing Chang (Shanghai Jiao Tong University) · Yixiao Fang (StepFun) · Peng Xing (Nanjing University of Science and Technology) · Shuhan Wu (StepFun) · Hai-Bao Chen (Shanghai Jiao Tong University)
benchmark frameworkcomprehensive evaluationscross-model comparisonsdiversity assessmentevaluation systemsfine-grained evaluationknowledge modelingmodel performance analysisprecision metricsreasoning-driven generationreproducible evaluationstyle transfersubject-element alignmenttext renderingtext-to-image models

Text-to-image (T2I) models have garnered significant attention for generating high-quality images aligned with text prompts. However, rapid T2I model advancements reveal limitations in early benchmarks, lacking comprehensive evaluations, especially for text rendering and style. Notably, recent state-of-the-art models, with their rich knowledge modeling capabilities, show potential in reasoning-driven image generation, yet existing evaluation systems have not adequately addressed this frontier. To systematically address these gaps, we introduce $\textbf{OneIG-Bench}$, a meticulously designed comprehensive benchmark framework for fine-grained evaluation of T2I models across multiple dimensions, including subject-element alignment, text rendering precision, reasoning-generated content, stylization, and diversity. By structuring the evaluation, this benchmark enables in-depth analysis of model performance, helping researchers and practitioners pinpoint strengths and bottlenecks in the full pipeline of image generation. Our codebase and dataset are now publicly available to facilitate reproducible evaluation studies and cross-model comparisons within the T2I research community.