tokens

3 papers

• A multiscale analysis of mean-field transformers in the moderate interaction regime
• A multiscale analysis of mean-field transformers in the moderate interaction regime
• Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation is Wasteful