Don’t Forget the Enjoin: FocalLoRA for Instruction Hierarchical Alignment in Large Language Models

Wei Wang (Beijing University of Post and Telecommunication) · Ruoyan Li (University of California, Los Angeles) · Guancheng Wan (University of California, Los Angeles) · Zijie Huang (Google Deepmind) · Haixin Wang (UCLA Computer Science Department, University of California, Los Angeles) · Xiao Luo (UCLA) · Yizhou Sun (UCLA) · Carl Yang (Emory University) · Wanjia Zhao (Stanford University) · Zitong Shi (Wuhan University) · Yijia Xiao (University of California, Los Angeles)

attention headsembedding-level modificationsfine-tuningfocal headshierarchical instruction adherencehierarchical promptsinstruction conflictsinstruction hierarchy awarenessparameter-efficient prompt engineeringstructural modelingstructure-aware frameworksystem instruction compliance

Recent studies reveal that large language models (LLMs) often struggle to resolve conflicting instructions embedded within hierarchical prompts, resulting in decreased compliance with system-level directives and compromising the reliability of safety-critical applications. While earlier approaches attempt to improve instruction hierarchy awareness through prompt engineering or embedding-level modifications, they typically lack structural modeling and either offer limited gains or require extensive fine-tuning. In this work, we introduce $\textbf{FocalLoRA}$, a parameter-efficient and structure-aware framework that strengthens hierarchical instruction adherence by selectively optimizing structurally critical attention heads, referred to as $\textit{focal heads}$, which exhibit heightened sensitivity to instruction conflicts. Experiments across multiple models and a dedicated benchmark demonstrate that FocalLoRA markedly enhances system instruction compliance with minimal tuning cost. For instance, on Llama-8B, fine-tuning only 0.0188\% of parameters yields a 35.52\% $\uparrow$ in system instruction compliance.