Linearly Constrained Diffusion Implicit Models

Vivek Jayaram (University of Washington) · Ira Kemelmacher-Shlizerman (University of Washington) · Steve Seitz (University of Washington) · John Thickstun (Cornell University)

3d point cloud reprojectionadaptive alignmentconstrained inferencedeblurring diffusion modelsforward diffusion processinpaintinglinearly constrained diffusion implicit modelsmeasurement consistencynoise-free linear inverse problemsnoisy linear inverse problemsprojection stepsresidual measurement energysuper-resolutiontheoretical distributionunconditional denoising

We introduce Linearly Constrained Diffusion Implicit Models (CDIM), a fast and accurate approach to solving noisy linear inverse problems using diffusion models. Traditional diffusion-based inverse methods rely on numerous projection steps to enforce measurement consistency in addition to unconditional denoising steps. CDIM achieves a 10–50× reduction in projection steps by dynamically adjusting the number and size of projection steps to align a residual measurement energy with its theoretical distribution under the forward diffusion process. This adaptive alignment preserves measurement consistency while substantially accelerating constrained inference. For noise-free linear inverse problems, CDIM exactly satisfies the measurement constraints with few projection steps, even when existing methods fail. We demonstrate CDIM’s effectiveness across a range of applications, including super-resolution, denoising, inpainting, deblurring, and 3D point cloud reprojection.