Researchers at Boston University Release the Platypus Family of Fine-Tuned LLMs

Abstraction: Cheap fast LLM fine-tuning via curated Open-Platypus dataset and LoRA merging

Key points:

• Platypus models topped HuggingFace's Open LLM Leaderboard at time of release (August 2023)
• Open-Platypus dataset: 11 public text datasets focused on STEM and logic knowledge, with rigorous deduplication
• A 13B Platypus model can be trained in 5 hours on a single A100 GPU using only 25k questions
• Method: parameter-efficient fine-tuning (PEFT) via LoRA module selection and merging on base LLMs
• Data contamination checking was a core contribution — methodology described as a guide for further work
• Strong performance achieved while using a fraction of fine-tuning data and compute vs. comparable models

Connections: Boston University · Huggingface · Large Language Models · Fine Tuning · Parameter Efficient Tuning

Source: https://www.marktechpost.com/2023/08/14/researchers-at-boston-university-release-the-platypus-family-of-fine-tuned-llms-to-achieve-cheap-fast-and-powerful-refinement-of-base-llms/