MACS: Multi-Agent Reinforcement Learning for Optimization of Crystal Structures

Elena Zamaraeva (University of Liverpool, University of Manchester) · Christopher Collins (University of Liverpool) · George Darling (University of Liverpool) · Matthew S Dyer (University of Liverpool) · Bei Peng (University of Sheffield) · Rahul Savani (University of Liverpool) · Dmytro Antypov (University of Liverpool) · Vladimir Gusev (University of Liverpool) · Judith Clymo (University of California, Santa Cruz) · Paul Spirakis (University of Liverpool) · Matthew Rosseinsky (University of Liverpool)

atomic structurescomputational chemistrycrystalline materialsenergy calculationsfailure rategeometry optimizationmacsmulti-agent reinforcement learning optimization methodspartially observable markov gameperiodic crystal structurepolicy optimization scalabilitystable configurationzero-shot transferability

Geometry optimization of atomic structures is a common and crucial task in computational chemistry and materials design. Following the learning to optimize paradigm, we propose a new multi-agent reinforcement learning method called Multi-Agent Crystal Structure optimization (MACS) to address the problem of periodic crystal structure optimization. MACS treats geometry optimization as a partially observable Markov game in which atoms are agents that adjust their positions to collectively discover a stable configuration. We train MACS across various compositions of reported crystalline materials to obtain a policy that successfully optimizes structures from the training compositions as well as structures of larger sizes and unseen compositions, confirming its excellent scalability and zero-shot transferability. We benchmark our approach against a broad range of state-of-the-art optimization methods and demonstrate that MACS optimizes periodic crystal structures significantly faster, with fewer energy calculations, and the lowest failure rate.