Reliable Lifelong Multimodal Editing: Conflict-Aware Retrieval Meets Multi-Level Guidance

Fanrui Zhang (University of Science and Technology of China) · Qiang Zhang (Dalian Martime University) · Jiawei Liu (University of Illinois Urbana-Champaign) · Ming Hu (Monash University) · Junjun He (Shanghai AI Laboratory) · Zheng-Jun Zha (University of Science and Technology of China)

conflict-aware dynamic retrievalcontinuous prompt prefixescross-modal associationsedit scope classifierhard correction mechanisminter-modal conflict quantificationintra-modal uncertaintyknowledge editingknowledge retrievallifelong multimodal editingmulti-level guidancemultimodal large language models output logitsstatic knowledge prompttoken-level context-aware dynamic prompt

The dynamic nature of real-world information demands efficient knowledge editing in multimodal large language models (MLLMs) to ensure continuous knowledge updates. However, existing methods often struggle with precise matching in large-scale knowledge retrieval and lack multi-level guidance for coordinated editing, leading to less reliable outcomes. To tackle these challenges, we propose CARML, a novel retrieval-augmented editing framework that integrates conflict-aware dynamic retrieval with multi-level implicit and explicit guidance for reliable lifelong multimodal editing. Specifically, CARML introduces intra-modal uncertainty and inter-modal conflict quantification to dynamically integrate multi-channel retrieval results, so as to pinpoint the most relevant knowledge to the incoming edit samples. Afterwards, an edit scope classifier discerns whether the edit sample semantically aligns with the edit scope of the retrieved knowledge. If deemed in-scope, CARML refines the retrieved knowledge into information-rich continuous prompt prefixes, serving as the implicit knowledge guide. These prefixes not only include static knowledge prompt that capture key textual semantics but also incorporate token-level, context-aware dynamic prompt to explore fine-grained cross-modal associations between the edit sample and retrieved knowledge. To further enhance reliability, CARML incorporates a "hard correction" mechanism, leveraging explicit label knowledge to adjust the model’s output logits. Extensive experiments across multiple MLLMs and datasets indicate the superior performance of CARML in lifelong multimodal editing scenarios.