Hi3DEval: Advancing 3D Generation Evaluation with Hierarchical Validity

Tong Wu (Beijing Institute for General Artificial Intelligence) · Ziwei Liu (Nanyang Technological University) · Dahua Lin (TTI Chicago) · Liang Pan (Nanyang Technological University) · Yuhan Zhang (Fudan University) · Long Zhuo (Shanghai AI Laboratory) · Ziyang Chu (Tsinghua University) · Zhibing Li (The Chinese University of Hong Kong)

3d content generation3d-aware automated scoringfine-grained quality analysishierarchical evaluation frameworkhuman preference alignmenthybrid 3d representationslarge-scale datasetmaterial authenticitymaterial realismmulti-agent annotation pipelinepart-level evaluationpretrained 3d featuresquality assessment spatial coherence spatio-temporal consistencytexture evaluation

Despite rapid advances in 3D content generation, quality assessment for the generated 3D assets remains challenging.Existing methods mainly rely on image-based metrics and operate solely at the object level, limiting their ability to capture spatial Despite rapid advances in 3D content generation, quality assessment for the generated 3D assets remains challenging.Existing methods mainly rely on image-based metrics and operate solely at the object level, limiting their ability to capture spatial coherence, material authenticity, and high-fidelity local details.1) To address these challenges, we introduce Hi3DEval, a hierarchical evaluation framework tailored for 3D generative content. It combines both object-level and part-level evaluation, enabling holistic assessments across multiple dimensions as well as fine-grained quality analysis. Additionally, we extend texture evaluation beyond aesthetic appearance by explicitly assessing material realism, focusing on attributes such as albedo, saturation, and metallicness. 2) To support this framework, we construct Hi3DBench, a large-scale dataset comprising diverse 3D assets and high-quality annotations, accompanied by a reliable multi-agent annotation pipeline.We further propose a 3D-aware automated scoring system based on hybrid 3D representations. Specifically, we leverage video-based representations for object-level and material-subject evaluations to enhance modeling of spatio-temporal consistency and employ pretrained 3D features for part-level perception.Extensive experiments demonstrate that our approach outperforms existing image-based metrics in modeling 3D characteristics and achieves superior alignment with human preference, providing a scalable alternative to manual evaluations.