Bayesian Concept Bottleneck Models with LLM Priors

Chandan Singh (Microsoft Research) · Jean Feng (UCSF) · Avni Kothari (University of California, San Francisco) · Lucas Zier (University of California, San Francisco/Zuckerberg San Francisco General Hospital) · Yan Shuo Tan (National University of Singapore)

bayesian frameworkblack-box models concept bottleneck modelsconcept extraction mechanismhallucination interpretabilityinterpretability-accuracy tradeoffinterpretable baselinesmiscalibration out-of-distribution samples rapid convergencesparse subsetstatistical inferencetransparent prediction modeluncertainty quantification

Concept Bottleneck Models (CBMs) have been proposed as a compromise between white-box and black-box models, aiming to achieve interpretability without sacrificing accuracy. The standard training procedure for CBMs is to predefine a candidate set of human-interpretable concepts, extract their values from the training data, and identify a sparse subset as inputs to a transparent prediction model. However, such approaches are often hampered by the tradeoff between exploring a sufficiently large set of concepts versus controlling the cost of obtaining concept extractions, resulting in a large interpretability-accuracy tradeoff. This work investigates a novel approach that sidesteps these challenges: BC-LLM iteratively searches over a potentially infinite set of concepts within a Bayesian framework, in which Large Language Models (LLMs) serve as both a concept extraction mechanism and prior. Even though LLMs can be miscalibrated and hallucinate, we prove that BC-LLM can provide rigorous statistical inference and uncertainty quantification. Across image, text, and tabular datasets, BC-LLM outperforms interpretable baselines and even black-box models in certain settings, converges more rapidly towards relevant concepts, and is more robust to out-of-distribution samples.