Efficient, Certifiably Optimal Clustering with Applications to Latent Variable Graphical Models

Abstract

We consider SDP relaxation methods for data and variable clustering problems, which have been shown in the literature to have good statistical properties in a variety of settings, but remain intractable to solve in practice. In particular, we propose FORCE, a new algorithm to solve the Peng-Wei $K$-means SDP. Compared to the naive interior point method, our method reduces the computational complexity of solving the SDP from $\tilde{O}(d^7 \log \epsilon^{-1})$ to $\tilde{O}(d^{6}K^{-2}\epsilon^{-1})$. Our method combines a primal first-order method with a dual optimality certificate search, which when successful, allows for early termination of the primal method. We show under certain data generating distributions that, with high probability, FORCE is guaranteed to find the optimal solution to the SDP relaxation and provide a certificate of exact optimality. As verified by our numerical experiments, this allows FORCE to solve the Peng-Wei SDP with dimensions in the hundreds in only tens of seconds. We also consider a variation of the Peng-Wei SDP for the case when $K$ is not known a priori and show that a slight modification of FORCE reduces the computational complexity of solving this problem as well: from $\tilde{O}(d^7 \log \epsilon^{-1})$ using a standard SDP solver to $\tilde{O}(d^{4}\epsilon^{-1})$.