Timezone: »

Convex Two-Layer Modeling
Özlem Aslan · Hao Cheng · Xinhua Zhang · Dale Schuurmans

Fri Dec 06 07:00 PM -- 11:59 PM (PST) @ Harrah's Special Events Center, 2nd Floor #None

Latent variable prediction models, such as multi-layer networks, impose auxiliary latent variables between inputs and outputs to allow automatic inference of implicit features useful for prediction. Unfortunately, such models are difficult to train because inference over latent variables must be performed concurrently with parameter optimization---creating a highly non-convex problem. Instead of proposing another local training method, we develop a convex relaxation of hidden-layer conditional models that admits global training. Our approach extends current convex modeling approaches to handle two nested nonlinearities separated by a non-trivial adaptive latent layer. The resulting methods are able to acquire two-layer models that cannot be represented by any single-layer model over the same features, while improving training quality over local heuristics.

Author Information

Özlem Aslan (University of Alberta)
Hao Cheng (University of Washington)
Xinhua Zhang (University of Illinois at Chicago (UIC))
Dale Schuurmans (Google Brain & University of Alberta)

More from the Same Authors