Sparse Attention

concepts · 1 notes linked

Related: Google Research · Exphormer · Graph Transformers · Graph Neural Networks · Transformers

Notes

• A New AI Research Introduces EXPHORMER: A Framework For Scaling Graph Transformers While Slashing Costs — Expander-graph sparse attention scales graph transformers to linear complexity