A brief history of diffusion, the tech at the heart of modern image-generating AI | TechCrunch

Abstraction: Origins and capabilities of diffusion models powering modern image generation AI

Key points:

• Diffusion models work by progressively adding noise to data until only noise remains, then learning to reverse the process — inspired by non-equilibrium thermodynamics
• GANs (generative adversarial networks) preceded diffusion but suffered from training instability and mode collapse; diffusion superseded them as the architecture of choice
• OpenAI's CLIP (Contrastive Language-Image Pre-Training) made diffusion practical for text-to-image by scoring each denoising step against a text prompt, acting as a guide
• CLIP was introduced alongside DALL-E and later incorporated into DALL-E 2 and open-source Stable Diffusion (Stability AI)
• Diffusion is being applied beyond images: music generation (Harmonai's Dance Diffusion, Riffusion via spectrograms), protein design (Generate Biomedicines, UW), and regulatory DNA sequence generation (DNA-Diffusion by OpenBioML)
• A UW diffusion model designed a protein that binds the parathyroid hormone better than existing drugs, an early drug-discovery success

Connections: Openai · Stability AI · Dall E · Stable Diffusion · Diffusion Models · Generative AI · Text To Image

Source: https://techcrunch.com/2022/12/22/a-brief-history-of-diffusion-the-tech-at-the-heart-of-modern-image-generating-ai/