Timezone: »

 
Poster
Distributionally Robust Optimization with Data Geometry
Jiashuo Liu · Jiayun Wu · Bo Li · Peng Cui

Tue Nov 29 09:00 AM -- 11:00 AM (PST) @ Hall J #530
in Poster Session 1 »
Distributionally Robust Optimization (DRO) serves as a robust alternative to empirical risk minimization (ERM), which optimizes the worst-case distribution in an uncertainty set typically specified by distance metrics including $f$-divergence and the Wasserstein distance. The metrics defined in the ostensible high dimensional space lead to exceedingly large uncertainty sets, resulting in the underperformance of most existing DRO methods. It has been well documented that high dimensional data approximately resides on low dimensional manifolds. In this work, to further constrain the uncertainty set, we incorporate data geometric properties into the design of distance metrics, obtaining our novel Geometric Wasserstein DRO (GDRO). Empowered by Gradient Flow, we derive a generically applicable approximate algorithm for the optimization of GDRO, and provide the bounded error rate of the approximation as well as the convergence rate of our algorithm. We also theoretically characterize the edge cases where certain existing DRO methods are the degeneracy of GDRO. Extensive experiments justify the superiority of our GDRO to existing DRO methods in multiple settings with strong distributional shifts, and confirm that the uncertainty set of GDRO adapts to data geometry.
Keywords: distributional robustness · Over-flexibility Problem · Data Geometry
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Jiashuo Liu (Tsinghua University)
Jiayun Wu (Computer Science, Tsinghua University)
Bo Li (Tsinghua University)
Peng Cui (Tsinghua University)

More from the Same Authors