Deferring Concept Bottleneck Models: Learning to Defer Interventions to Inaccurate Experts

Pietro Barbiero (IBM Research) · Andrea Pugnana (University of Trento) · Riccardo Massidda (University of Pisa) · Francesco Giannini (Scuola Normale Superiore) · Mateo Espinosa Zarlenga (University of Cambridge) · Roberto Pellungrini (Scuola Normale Superiore) · Gabriele Dominici (Università della Svizzera Italiana (USI)) · Fosca Giannotti (School of Education Pisa) · Davide Bacciu (University of Pisa)

concept bottleneck models decision-making processdeferring cbmsdeferring systemsexplainabilityhuman error-pronenessinterpretable machine learningintervention necessityl2d losslearning to defermodel compositionperformance improvement predictive performance supervised learning targeted interventions

Concept Bottleneck Models (CBMs) are interpretable machine learning models that ground their predictions on human-understandable concepts, allowing for targeted interventions in their decision-making process. However, when intervened on, CBMs assume the availability of humans that can identify the need to intervene and always provide correct interventions. Both assumptions are unrealistic and impractical, considering labor costs and human error-proneness. In contrast, Learning to Defer (L2D) extends supervised learning by allowing machine learning models to identify cases where a human is more likely to be correct than the model, thus leading to deferring systems with improved performance. In this work, we gain inspiration from L2D and propose Deferring CBMs (DCBMs), a novel framework that allows CBMs to learn when an intervention is needed. To this end, we model DCBMs as a composition of deferring systems and derive a consistent L2D loss to train them. Moreover, by relying on a CBM architecture, DCBMs can explain the reasons for deferring on the final task. Our results show that DCBMs can achieve high predictive performance and interpretability by deferring only when needed.