$450 and 19 hours is all it takes to rival OpenAI's o1-preview
open-source-aireasoning-modelsfine-tuninglow-cost-trainingsky-t1
Abstraction: UC Berkeley Sky-T1-32B open-source reasoning model built for $450 in 19 hours
Key points:
- NovaSky team at UC Berkeley released Sky-T1-32B-Preview: fine-tuned from Alibaba's Qwen2.5-32B-Instruct using synthetic data from QwQ-32B-Preview
- Total training cost $450 (Lambda Cloud pricing) on 8 Nvidia H100 GPUs in 19 hours
- Matches or exceeds OpenAI's o1-preview on math and coding benchmarks; does not surpass o1 on GPQA-Diamond graduate-level science questions
- Full open-source release: weights, data, infrastructure, and technical details
- Demonstrates that high-level reasoning capabilities can be replicated affordably using synthetic training data and open models
- Contrast: GPT-4 suspected to cost ~$78M in compute; Sky-T1 shows academic labs can compete at a fraction of the cost
Connections: Openai · Uc Berkeley · Large Language Models · Open Source AI
Source: https://www.zdnet.com/article/450-and-19-hours-is-all-it-takes-to-rival-openais-o1-preview/