MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization

Chen Wang (University of Pennsylvania) · Wenjie Qiu (Rutgers University) · Ran Cheng (Hong Kong Polytechnic University) · Hongshu Guo (South China University of Technology) · Zeyuan Ma (South China University of Technology) · Yining Ma (MIT) · Yue-Jiao Gong · Sijie Ma (South China University of Technology) · Hongqiao Lian (South China University of Technology) · Jiajun Zhan (South China University of Technology) · Kaixu Chen (South China University of Technology) · Zhiyang Huang (South China University of Technology) · Zechuan Huang (South China University of Technology) · Guojun Peng (South China University of Technology)
benchmark suitebi-level structurecustom analysisevolutionary approachesgradient-based approachesmeta-black-box optimizationmeta-learningmeta-trainingmulti-model optimizationmulti-objective optimizationmulti-task optimizationoptimization performanceparallelization schemesreinforcement learningsingle-objective optimizationvisualization interfaces

Meta-Black-Box Optimization (MetaBBO) streamlines the automation of optimization algorithm design through meta-learning. It typically employs a bi-level structure: the meta-level policy undergoes meta-training to reduce the manual effort required in developing algorithms for low-level optimization tasks. The original MetaBox (2023) provided the first open-source framework for reinforcement learning-based single-objective MetaBBO. However, its relatively narrow scope no longer keep pace with the swift advancement in this field. In this paper, we introduce MetaBox-v2 (\url{https://github.com/MetaEvo/MetaBox}) as a milestone upgrade with four novel features: 1) a unified architecture supporting RL, evolutionary, and gradient-based approaches, by which we reproduce $23$ up-to-date baselines; 2) efficient parallelization schemes, which reduce the training/testing time by $10-40$x; 3) a comprehensive benchmark suite of $18$ synthetic/realistic tasks ($1900$+ instances) spanning single-objective, multi-objective, multi-model, and multi-task optimization scenarios; 4) plentiful and extensible interfaces for custom analysis/visualization and integrating to external optimization tools/benchmarks. To show the utility of MetaBox-v2, we carry out a systematic case study that evaluates the built-in baselines in terms of the optimization performance, generalization ability and learning efficiency. Valuable insights are concluded from thorough and detailed analysis for practitioners and those new to the field.