A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective

Qing Qu (University of Michigan) · Huijie Zhang (University of Michigan - Ann Arbor) · Molei Tao (Georgia Tech) · Lianghe Shi (University of Michigan) · Meng Wu (University of Michigan - Ann Arbor) · Zekai Zhang (University of Michigan)

data selection strategy diffusion models distribution shift diversity entropy generalization memorization model collapsemodel degradationperformance degradationrecursive generationrecursive iterationssynthetic datavariance shrinkagevisual quality

The widespread use of diffusion models has led to an abundance of AI-generated data, raising concerns about model collapse