rope

3 papers

• Knee-Deep in C-RASP: A Transformer Depth Hierarchy
• LEDiT: Your Length-Extrapolatable Diffusion Transformer without Positional Encoding
• Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers