embedding dimension
The dimensionality of the space into which data points are projected when transformed by an embedding technique, critically influencing the model’s performance and the representational capacity for various tasks like classification and clustering.
- Compositional Reasoning with Transformers, RNNs, and Chain of Thought
- Depth-Width Tradeoffs for Transformers on Graph Tasks
- Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
- Hierarchical Retrieval: The Geometry and a Pretrain-Finetune Recipe
- On the Emergence of Linear Analogies in Word Embeddings