Timezone: »

 
Poster
Dynamic Local Regret for Non-convex Online Forecasting
Sergul Aydore · Tianhao Zhu · Dean Foster

Thu Dec 12 05:00 PM -- 07:00 PM (PST) @ East Exhibition Hall B + C #20
in Algorithms -- Online Learning »

We consider online forecasting problems for non-convex machine learning models. Forecasting introduces several challenges such as (i) frequent updates are necessary to deal with concept drift issues since the dynamics of the environment change over time, and (ii) the state of the art models are non-convex models. We address these challenges with a novel regret framework. Standard regret measures commonly do not consider both dynamic environment and non-convex models. We introduce a local regret for non-convex models in a dynamic environment. We present an update rule incurring a cost, according to our proposed local regret, which is sublinear in time T. Our update uses time-smoothed gradients. Using a real-world dataset we show that our time-smoothed approach yields several benefits when compared with state-of-the-art competitors: results are more stable against new data; training is more robust to hyperparameter selection; and our approach is more computationally efficient than the alternatives.

Keywords: Time Series Analysis · Supervised Deep Networks · Applications · Algorithms -> Online Learning; Deep Learning -> Optimization for Deep Networks; Deep Learning
[ Paper]  [ Slides] 

Author Information

Sergul Aydore (Stevens Institute of Technology)
Tianhao Zhu (Stevens Institute of Techonlogy)
Dean Foster (Amazon)

More from the Same Authors