Timezone: »

 
Poster
Automatic and Harmless Regularization with Constrained and Lexicographic Optimization: A Dynamic Barrier Approach
Chengyue Gong · Xingchao Liu · Qiang Liu

Tue Dec 07 04:30 PM -- 06:00 PM (PST) @
in Poster Session 2 »

Many machine learning tasks have to make a trade-off between two loss functions, typically the main data-fitness loss and an auxiliary loss. The most widely used approach is to optimize the linear combination of the objectives, which, however, requires manual tuning of the combination coefficient and is theoretically unsuitable for non-convex functions. In this work, we consider constrained optimization as a more principled approach for trading off two losses, with a special emphasis on lexicographic optimization, a degenerated limit of constrained optimization which optimizes a secondary loss inside the optimal set of the main loss. We propose a dynamic barrier gradient descent algorithm which provides a unified solution of both constrained and lexicographic optimization. We establish the convergence of the method for general non-convex functions.

Keywords: Optimization · Machine Learning · Graph Learning

Author Information

Chengyue Gong (Peking University)
Xingchao Liu (University of Texas at Austin)
Qiang Liu (Dartmouth College)

More from the Same Authors