ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation

Hao Chen (Harbin Institute of Technology) · Pengcheng Huang (Northeastern University) · Zhenghao Liu (Northeastern University) · Yukun Yan (Tsinghua University, Tsinghua University) · Haiyan Zhao (Tsinghua University) · Xiaoyuan Yi (Microsoft) · Zhiyuan Liu (Tsinghua University) · Maosong Sun (Tsinghua University, Tsinghua University) · Tong Xiao (Northeastern University (CN)) · Ge Yu (Northeastern University, China) · Chenyan Xiong (School of Computer Science, Carnegie Mellon University)

benchmark evaluationcofaithfulqaconfiqa benchmarkcontextual faithfulnessevidence groundingfeed-forward networksffn suppressioninternal knowledge conflictsinternal parametric knowledgemodel calibrationparametric knowledge mutingparametric memoryretrieval-augmented generation trustworthinessunfaithful generation

Large language models (LLMs) integrated with retrieval-augmented generation (RAG) have improved factuality by grounding outputs in external evidence. However, they remain susceptible to unfaithful generation, where outputs contradict retrieved context despite its relevance and accuracy. Existing approaches aiming to improve faithfulness primarily focus on enhancing the utilization of external context, but often overlook the persistent influence of internal parametric knowledge during generation. In this work, we investigate the internal mechanisms behind unfaithful generation and identify a subset of mid-to-deep feed-forward networks (FFNs) that are disproportionately activated in such cases. Building on this insight, we propose Parametric Knowledge Muting through FFN Suppression (ParamMute), a framework that improves contextual faithfulness by suppressing the activation of unfaithfulness-associated FFNs and calibrating the model toward retrieved knowledge. To evaluate our approach, we introduce CoFaithfulQA, a benchmark specifically designed to evaluate faithfulness in scenarios where internal knowledge conflicts with accurate external evidence. Experimental results show that ParamMute significantly enhances faithfulness across both CoFaithfulQA and the established ConFiQA benchmark, achieving substantial reductions in reliance on parametric memory. These findings underscore the importance of mitigating internal knowledge dominance and provide a new direction for improving LLM trustworthiness in RAG. All codes are available at https://github.com/OpenBMB/ParamMute.