positional encoding

7 papers

• Causality-Induced Positional Encoding for Transformer-Based Representation Learning of Non-Sequential Features
• Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation
• Dynamical Properties of Tokens in Self-Attention and Effects of Positional Encoding
• Orthogonal Contrastive Learning for Multi-Representation fMRI Analysis
• Revisiting LRP: Positional Attribution as the Missing Ingredient for Transformer Explainability
• Spectral Graph Neural Networks are Incomplete on Graphs with a Simple Spectrum
• Vocabulary In-Context Learning in Transformers: Benefits of Positional Encoding