Google AI Introduces GraphWorld: A Methodology For Analyzing The Performance Of GNN Architectures On Millions Of Synthetic Benchmark Datasets

Abstraction: GraphWorld uses synthetic graphs to benchmark GNNs beyond standard academic datasets

Key points:

• Most GNN research reuses only 5–10 benchmark datasets (citation networks, molecular graphs), all clustered in a small region of graph-property space
• GraphWorld (Google graph mining team) generates millions of synthetic benchmark datasets using parameterized stochastic block models (SBM), sampling across homophily, degree distribution, and cluster count
• Running GraphWorld is cost-efficient: hundreds of thousands of GNN experiments on synthetic data cost the same as one experiment on a large OGB dataset
• Classic datasets like Cora and CiteSeer have high homophily; GNN rankings change rapidly as graphs shift toward low-homophily regions — a blind spot hidden by standard benchmarks
• GraphWorld maps GNN performance on a 3D surface (homophily × average degree × model accuracy), revealing performance cliffs invisible in academic benchmarks
• The framework supports GNN pre-training and exploration of novel random/generative graph models

Connections: Google · Graph Neural Networks · Benchmarking · Synthetic Data

Source: https://www.marktechpost.com/2022/05/11/google-ai-introduces-graphworld-a-methodology-for-analyzing-the-performance-of-gnn-architectures-on-millions-of-synthetic-benchmark-datasets/