Diffusion Models Meet Contextual Bandits
action dependenciesalgorithmic performancecomplex dependenciescomputational efficiencycontextual banditsdecision policiesempirical resultsexpressive priorsfast updatesonline decision-makingposterior approximationpractical algorithmpre-trained diffusion modelssamplingstatistical efficiencyversatility
Efficient online decision-making in contextual bandits is challenging, as methods without informative priors often suffer from computational or statistical inefficiencies. In this work, we leverage pre-trained diffusion models as expressive priors to capture complex action dependencies and develop a practical algorithm that efficiently approximates posteriors under such priors, enabling both fast updates and sampling. Empirical results demonstrate the effectiveness and versatility of our approach across diverse contextual bandit settings.