Continual Release Moment Estimation with Differential Privacy

Nikita Kalinin (Institute of Science and Technology) · Jalaj Upadhyay (Rutgers University) · Christoph Lampert (Institute of Science and Technology Austria (ISTA))
accuracy improvementcontinual estimationcovariance matrixdata streamdifferential privacydp-adamgaussian density estimationjoint moment estimationjoint sensitivity analysismatrix mechanismmodel trainingnoise reductionprivacy regimeprivate estimationrunning meansecond-moment estimation

We propose *Joint Moment Estimation* (JME), a method for continually and privately estimating both the first and second moments of a data stream with reduced noise compared to naive approaches. JME supports the *matrix mechanism* and exploits a joint sensitivity analysis to identify a privacy regime in which the second-moment estimation incurs no additional privacy cost, thereby improving accuracy while maintaining privacy. We demonstrate JME’s effectiveness in two applications: estimating the running mean and covariance matrix for Gaussian density estimation and model training with DP-Adam.