Guarantees for Alternating Least Squares in Overparameterized Tensor Decompositions
alternating least squarescomputational challengesfactor analysisglobal convergencegradient descenthigh probabilitylatent variable modelslow-rank approximationmatrix anticoncentrationnon-convex optimizationoverparameterizationrandom initializationrank approximationscalable iterative methodstensor decompositiontheoretical techniques
Tensor decomposition is a canonical non-convex optimization problem that is computationally challenging, and yet important due to applications in factor analysis and parameter estimation of latent variable models. In practice, scalable iterative methods, particularly Alternating Least Squares (ALS), remain the workhorse for tensor decomposition despite the lack of global convergence guarantees. A popular approach to tackle challenging non-convex optimization problems is overparameterization