Tabula: A Tabular Self-Supervised Foundation Model for Single-Cell Transcriptomics

Jiliang Tang (Michigan State University) · Min Li (Central South University) · Xiaojie Qiu (Stanford University) · Jiayuan Ding (Stanford University) · Jianhui Lin (Central South University) · Shiyu Jiang (University of Southern California) · Yixin Wang (Stanford University) · Ziyang Miao (Central South University) · Zhaoyu Fang (Central South University)

cell contrastive learningcell type annotationcell-by-gene expression matrixcell-level heterogeneitydecentralized datasetsfederated learning foundation modelsgene imputationgene reconstructiongene-level relationshipsmulti-omics integrationperturbation predictionprivacy-preserving pretrainingsingle-cell genomicstabular modeling

Foundation models (FMs) have shown great promise in single-cell genomics, yet current approaches, such as scGPT, Geneformer, and scFoundation, rely on centralized training and language modeling objectives that overlook the tabular nature of single-cell data and raise significant privacy concerns. We present TABULA, a foundation model designed for single-cell transcriptomics, which integrates a novel tabular modeling objective and federated learning framework to enable privacy-preserving pretraining across decentralized datasets. TABULA directly models the cell-by-gene expression matrix through column-wise gene reconstruction and row-wise cell contrastive learning, capturing both gene-level relationships and cell-level heterogeneity without imposing artificial gene sequence order. Extensive experiments demonstrate the effectiveness of TABULA: despite using only half the pretraining data, TABULA achieves state-of-the-art performance across key tasks, including gene imputation, perturbation prediction, cell type annotation, and multi-omics integration. It is important to note that as public single-cell datasets continue to grow, TABULA provides a scalable and privacy-aware foundation that not only validates the feasibility of federated tabular modeling but also establishes a generalizable framework for training future models under similar privacy-preserving settings.