Graph Classification with Transformers

Abstraction: Tutorial fine-tuning Microsoft Graphormer for graph classification via HuggingFace Transformers

Key points:

• Only graph transformer model available in HuggingFace Transformers at time of writing is Microsoft's Graphormer (arXiv:2106.05234)
• Uses the OGB ogbg-molhiv dataset from Stanford's Open Graph Benchmark as a binary classification example
• Graphormer preprocessing generates in/out degree info and shortest-path node matrices as structural features
• Fine-tuning loads pretrained checkpoint clefourrier/pcqm4mv2_graphormer_base with a custom classification head via ignore_mismatched_sizes=True
• Training on CPU (Intel i7) takes ~1 day for 20 epochs; GPU parallelization recommended for large graphs
• Data collator GraphormerDataCollator is required to batch graphs properly during training

Connections: Hugging Face · Microsoft · Graph Neural Networks · Transformers · Graph Classification

Source: https://huggingface.co/blog/graphml-classification