EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code

Zhijiang Guo (The Hong Kong University of Science and Technology (Guangzhou)) · See-Kiong Ng (National University of Singapore) · Jie Zhang (SPY Lab -- ETH Zurich) · Yuhao Qing (The University of Hong Kong) · Boyu Zhu (University College London, University of London) · Mingzhe Du (Nanyang Technological University) · Terry Yue Zhuo (Data61, CSIRO) · Qianru Zhang (the University of Hong Kong, The University of Hong Kong) · Heming Cui (the University of Hong Kong, University of Hong Kong) · Siu Ming Yiu (the University of Hong Kong, The University of Hong Kong) · Dong HUANG (National University of Singapore) · Anh Tuan Luu (Nanyang Technological University, Singapore)

code efficiencycode generationcompetitive programming tasksefficiency baselinesefficiency evaluationevaluation infrastructurefunctional correctnesshuman efficiencyhuman-expert solutionslanguage-specific variationllm-generated codemulti-language benchmarkoptimization-oriented methodsstate-of-the-art llms

Existing code generation benchmarks primarily evaluate functional correctness, with limited attention to code efficiency, and they are often restricted to a single language such as Python. To address this gap, we introduce EffiBench‑X, the first large‑scale multi‑language benchmark specifically designed for robust efficiency evaluation of LLM‑generated code. EffiBench‑X supports Python, C++, Java, JavaScript, Ruby, and Go, and comprises competitive programming tasks paired with human‑expert solutions as efficiency baselines. Evaluating state‑of‑the‑art LLMs on EffiBench‑X reveals that while models frequently generate functionally correct code, they consistently underperform human experts in efficiency. Even the most efficient LLM‑generated solutions (e.g., Qwen3‑32B) achieve only around 62% of human efficiency on average, with significant language‑specific variation: models tend to perform better in Python, Ruby, and JavaScript than in Java, C++, and Go (e.g., DeepSeek‑R1’s Python code is markedly more efficient than its Java code). These findings highlight the need for research into optimization‑oriented methods to improve the efficiency of LLM‑generated code across diverse languages. The dataset and evaluation infrastructure are publicly available at https://github.com/EffiBench/EffiBench-X.git and https://huggingface.co/datasets/EffiBench/effibench-x.