Policy Gradient Methods Converge Globally in Imperfect-Information Extensive-Form Games
$\epsilon$-approximate nebehavioral strategydirect policy parametrizationequilibrium convergenceextensive-form gamesgradient dominanceimperfect-informationmarkov gamesmulti-agent reinforcement learningnash equilibriumnonconvex optimizationperfect-recallproximal polyak-łojasiewicz conditionregularized alternating policy gradientsoftmax policy parametrizationtrajectory samples
Multi-agent reinforcement learning (MARL) has long been seen as inseparable from Markov games (Littman 1994). Yet, the most remarkable achievements of practical MARL have arguably been in extensive-form games (EFGs)