AAAI Publications, Thirty-First AAAI Conference on Artificial Intelligence

Font Size: 
Polynomial Optimization Methods for Matrix Factorization
Po-Wei Wang, Chun-Liang Li, J. Zico Kolter

Last modified: 2017-02-13

Abstract


Matrix factorization is a core technique in many machine learning problems, yet also presents a nonconvex and often difficult-to-optimize problem. In this paper we present an approach based upon polynomial optimization techniques that both improves the convergence time of matrix factorization algorithms and helps them escape from local optima. Our method is based on the realization that given a joint search direction in a matrix factorization task, we can solve the ``subspace search'' problem (the task of jointly finding the steps to take in each direction) by solving a bivariate quartic polynomial optimization problem. We derive two methods for solving this problem based upon sum of squares moment relaxations and the Durand-Kerner method, then apply these techniques on matrix factorization to derive a direct coordinate descent approach and a method for speeding up existing approaches. On three benchmark datasets we show the method substantially improves convergence speed over state-of-the-art approaches, while also attaining lower objective value.

Keywords


polynomial optimization; matrix factorization; Durand-Kerner method

Full Text: PDF