Architectural and Inferential Inductive Biases for Exchangeable Sequence Modeling

Thomson Yen (Columbia University) · C. Guetta (Columbia University) · Hongseok Namkoong (Columbia University) · Leon Li (New York University) · Daksh Mittal (Columbia University)

active learning aleatoric uncertainty architectural innovations autoregressive modelsbanditscausal maskingcomputational overhead decision-making epistemic uncertaintyexchangeable sequenceslatent factormulti-step generationposterior inference transformer architectures uncertainty quantification

Autoregressive models have emerged as a powerful framework for modeling exchangeable sequences