pre-trained transformers

4 papers

• Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency
• Model Editing for Vision Transformers
• X-Mahalanobis: Transformer Feature Mixing for Reliable OOD Detection
• Zebra-Llama: Towards Extremely Efficient Hybrid Models