systematic evaluation
Systematic evaluation is a comprehensive approach to assessing a model's performance that involves predefined protocols, metrics, and rigorous testing conditions. It ensures that results are reliable and that the model's strengths and weaknesses are well understood.
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
- Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Search
- BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks
- Clean First, Align Later: Benchmarking Preference Data Cleaning for Reliable LLM Alignment
- DAVE: Diagnostic benchmark for Audio Visual Evaluation
- DroneAudioset: An Audio Dataset for Drone-based Search and Rescue
- Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations
- Exploring the Limits of Vision-Language-Action Manipulation in Cross-task Generalization
- GC4NC: A Benchmark Framework for Graph Condensation on Node Classification with New Insights
- KScope: A Framework for Characterizing the Knowledge Status of Language Models
- Mars-Bench: A Benchmark for Evaluating Foundation Models for Mars Science Tasks
- Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models
- Meta-Learning Objectives for Preference Optimization
- SAGE-Eval: Evaluating LLMs for Systematic Generalizations of Safety Facts
- Solving Inequality Proofs with Large Language Models
- URLs Help, Topics Guide: Understanding Metadata Utility in LLM Training