Amortized Variational Transdimensional Inference

Rafael Oliveira (CSIRO's Data61) · Laurence Davies (University of Wollongong) · Daniel MacKinlay (CSIRO Data61) · Scott SIsson (University of New South Wales, Sydney)

amortized variational densitybayesian optimizationconditional target distributioncosmic normalizing flowsflow-based modelshigh-cardinality model spacesmonte carlo gradient estimationmulti-model bayesian inferenceneural autoregressive conditional normalizing flownumerical experimentsoptimization-based bayesian inferenceparameter spacestochastic variational inferencestochastic variational transdimensional inferencetransdimensional joint model

The expressiveness of flow-based models combined with stochastic variational inference (SVI) has expanded the application of optimization-based Bayesian inference to highly complex problems. However, despite the importance of multi-model Bayesian inference, defined over a transdimensional joint model and parameter space, flow-based SVI has been limited to problems defined over a fixed-dimensional parameter space. We introduce CoSMIC normalizing flows (COntextually-Specified Masking for Identity-mapped Components), an extension to neural autoregressive conditional normalizing flow architectures that enables use of a single amortized variational density for inference over a transdimensional (multi-model) conditional target distribution. We propose a combined stochastic variational transdimensional inference (VTI) approach to training CoSMIC flows using ideas from Bayesian optimization and Monte Carlo gradient estimation. Numerical experiments show the performance of VTI on challenging problems that scale to high-cardinality model spaces.