Do-PFN: In-Context Learning for Causal Effect Estimation

Bernhard Schölkopf (MPI for Intelligent Systems, Tübingen) · Frank Hutter (ELLIS Institute Tübingen & University of Freiburg) · Siyuan Guo (University of Cambridge) · Jake Robertson (ELLIS Institute Tübingen) · Arik Reuter (University of Cambridge, Max-Planck Institute for Intelligent Systems) · Noah Hollmann (Uni of Freiburg / Charité Hospital Berlin)

accurate estimationcausal effect estimation causal graph causal inference causal structures in-context learning interventional datainterventional outcomesobservational data predictive performanceprior-data fitted networkssemi-synthetic settingssynthetic causal datatabular machine learningunconfoundedness

Causal effect estimation is critical to a range of scientific disciplines. Existing methods for this task either require interventional data, knowledge about the ground-truth causal graph, or rely on assumptions such as unconfoundedness, restricting their applicability in real-world settings. In the domain of tabular machine learning, Prior-data fitted networks (PFNs) have achieved state-of-the-art predictive performance, having been pre-trained on synthetic causal data to solve tabular prediction problems via in-context learning. To assess whether this can be transferred to the problem of causal effect estimation, we pre-train PFNs on synthetic data drawn from a wide variety of causal structures, including interventions, to predict interventional outcomes given observational data. Through extensive experiments in synthetic and semi-synthetic settings, we show that our approach allows for the accurate estimation of causal effects without knowledge of the underlying causal graph.