Google Research

entities · 4 notes linked

Related: Transformers · Hyperparameter Tuning · Deep Learning · Model Training · Exphormer · Graph Transformers · Graph Neural Networks · Sparse Attention

Notes

• A New AI Research Introduces EXPHORMER: A Framework For Scaling Graph Transformers While Slashing Costs — Expander-graph sparse attention scales graph transformers to linear complexity
• Beyond automatic differentiation — Google's AutoBound computes polynomial function bounds enabling hyperparameter-free optimizers
• GitHub - google-research/tuning_playbook: A playbook for systematically maximizing the performance of deep learning models. — Systematic hyperparameter tuning playbook from Google Brain researchers
• PEGASUS: A State-of-the-Art Model for Abstractive Text Summarization — Google model using gap-sentence pre-training for abstractive summarization