Timezone: »
Poster
Iteratively Reweighted Least Squares for Basis Pursuit with Global Linear Convergence Rate
Christian Kümmerle · Claudio Mayrink Verdun · Dominik Stöger
The recovery of sparse data is at the core of many applications in machine learning and signal processing. While such problems can be tackled using $\ell_1$-regularization as in the LASSO estimator and in the Basis Pursuit approach, specialized algorithms are typically required to solve the corresponding high-dimensional non-smooth optimization for large instances.Iteratively Reweighted Least Squares (IRLS) is a widely used algorithm for this purpose due to its excellent numerical performance. However, while existing theory is able to guarantee convergence of this algorithm to the minimizer, it does not provide a global convergence rate. In this paper, we prove that a variant of IRLS converges \emph{with a global linear rate} to a sparse solution, i.e., with a linear error decrease occurring immediately from any initialization if the measurements fulfill the usual null space property assumption. We support our theory by numerical experiments showing that our linear rate captures the correct dimension dependence. We anticipate that our theoretical findings will lead to new insights for many other use cases of the IRLS algorithm, such as in low-rank matrix recovery.
Author Information
Christian Kümmerle (Johns Hopkins University)
Claudio Mayrink Verdun (Technical University Munich)
Dominik Stöger (Katholische Universität Eichstätt-Ingolstadt)
Related Events (a corresponding poster, oral, or spotlight)
-
2021 Spotlight: Iteratively Reweighted Least Squares for Basis Pursuit with Global Linear Convergence Rate »
Dates n/a. Room
More from the Same Authors
-
2022 Poster: Global Linear and Local Superlinear Convergence of IRLS for Non-Smooth Robust Regression »
Liangzu Peng · Christian Kümmerle · Rene Vidal -
2021 Poster: Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction »
Dominik Stöger · Mahdi Soltanolkotabi