Precise Information Control in Long-Form Text Generation

Luke Zettlemoyer (University of Washington; Meta) · Weijia Shi (University of Washington, Seattle) · Yulia Tsvetkov (Department of Computer Science, University of Washington) · Danqi Chen (Princeton University) · Zhiyuan Zeng (University of Washington) · Margaret Li (University of Washington) · Jacqueline He (University of Washington) · Howard Yen (Princeton University) · Stella Li (University of Washington) · Pang Wei Koh (University of Washington)
biography generationexact match recallfactual precisionfaithfulness hallucinationgrounded generationlong-form outputspic-benchpic-lmpost-training frameworkprecise information controlself-contained statementssummarizationverifiable input claimsweakly supervised preference data

A central challenge in language models (LMs) is faithfulness hallucination: the generation of information unsubstantiated by input context. To study this problem, we propose Precise Information Control (PIC), a new task formulation that requires models to generate long-form outputs grounded in a provided set of short self-contained statements, without adding any unsupported ones. PIC includes a full setting that tests a model’s ability to include exactly all input claims, and a partial setting that requires the model to selectively incorporate only relevant claims. We present PIC-Bench, a benchmark of eight long-form generation tasks (e.g., summarization, biography generation) adapted to the PIC setting, where LMs are supplied with well-formed, verifiable input claims. Our evaluation of a range of open and proprietary LMs on PIC-Bench reveals that, surprisingly, state-of-the-art LMs still hallucinate against user-provided input in over 70% of generations. To alleviate this lack of faithfulness, we introduce a post-training framework that uses a weakly supervised preference data construction method to train an 8B PIC-LM with stronger PIC ability—improving from 69.1% to 91.0% F1 in the full PIC setting. When integrated into end-to-end factual generation pipelines, PIC-LM improves exact match recall by 17.1% on ambiguous QA with retrieval, and factual precision by 30.5% on a birthplace fact-checking task, underscoring the potential of precisely grounded generation.