Paper page - Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models
llm-evaluationbenchmarkstem-educationgpt-4few-shot-learning
Abstraction: GPT-4 achieves near-perfect scores on MIT Math and EECS curriculum
Key points:
- Dataset of 4,550 questions and solutions from all MIT Mathematics and EECS degree-required courses (problem sets, midterms, finals)
- GPT-3.5 solved ~1/3 of the full curriculum; GPT-4 with prompt engineering achieved a near-perfect solve rate on text-only questions
- Open-source LLM fine-tuned on the dataset as a comparison baseline
- GPT-4 used as an automatic grader, providing performance breakdown by course, question type, and answer format
- Embedding questions in low-dimensional space reveals prerequisite structure and which classes/topics are needed to solve others (via few-shot learning analysis)
- Authors argue findings highlight LLMs' potential for improving Mathematics and EECS education
Connections: GPT-4 · Mit · Large Language Models · Prompt Engineering · Benchmark Evaluation · Few Shot Learning