attention weights
Values computed in attention mechanisms that determine the influence of different parts of the input on the output. They allow models to focus on relevant portions of data during processing.
- CLAWS:Creativity detection for LLM-generated solutions using Attention Window of Sections
- Order-Level Attention Similarity Across Language Models: A Latent Commonality
- Spiking Meets Attention: Efficient Remote Sensing Image Super-Resolution with Attention Spiking Neural Networks
- Structured Initialization for Vision Transformers
- The Atlas of In-Context Learning: How Attention Heads Shape In-Context Retrieval Augmentation