Improving the Generation and Evaluation of Synthetic Data for Downstream Medical Causal Inference

Mihaela van der Schaar (University of Cambridge) · Dennis Frauen (LMU Munich) · Stefan Feuerriegel (LMU Munich) · Harry Amad (University of Cambridge) · Zhaozhi Qian (University of Cambridge) · Julianna Piskorz (University of Cambridge)

causal inferencecovariate distributiondata-generating processdesideratadownstream utilityempirical demonstration evaluation metrics generative modelsmedical interventionsoutcome generation mechanismregulatory barriersstate-of-the-art performance synthetic datatreatment assignment mechanismtreatment effect analysis

Causal inference is essential for developing and evaluating medical interventions, yet real-world medical datasets are often difficult to access due to regulatory barriers. This makes synthetic data a potentially valuable asset that enables these medical analyses, along with the development of new inference methods themselves. Generative models can produce synthetic data that closely approximate real data distributions, yet existing methods do not consider the unique challenges that downstream causal inference tasks, and specifically those focused on treatments, pose. We establish a set of desiderata that synthetic data containing treatments should satisfy to maximise downstream utility: preservation of (i) the covariate distribution, (ii) the treatment assignment mechanism, and (iii) the outcome generation mechanism. Based on these desiderata, we propose a set of evaluation metrics to assess such synthetic data. Finally, we present STEAM: a novel method for generating Synthetic data for Treatment Effect Analysis in Medicine that mimics the data-generating process of data containing treatments and optimises for our desiderata. We empirically demonstrate that STEAM achieves state-of-the-art performance across our metrics as compared to existing generative models, particularly as the complexity of the true data-generating process increases.