Timezone: »

 
Spotlight
Convex Two-Layer Modeling
Özlem Aslan · Hao Cheng · Xinhua Zhang · Dale Schuurmans

Fri Dec 06 11:48 AM -- 11:52 AM (PST) @ Harvey's Convention Center Floor, CC
in Spotlight Session 2 »

Latent variable prediction models, such as multi-layer networks, impose auxiliary latent variables between inputs and outputs to allow automatic inference of implicit features useful for prediction. Unfortunately, such models are difficult to train because inference over latent variables must be performed concurrently with parameter optimization---creating a highly non-convex problem. Instead of proposing another local training method, we develop a convex relaxation of hidden-layer conditional models that admits global training. Our approach extends current convex modeling approaches to handle two nested nonlinearities separated by a non-trivial adaptive latent layer. The resulting methods are able to acquire two-layer models that cannot be represented by any single-layer model over the same features, while improving training quality over local heuristics.

Author Information

Özlem Aslan (University of Alberta)
Hao Cheng (University of Washington)
Xinhua Zhang (University of Illinois at Chicago (UIC))
Dale Schuurmans (Google Brain & University of Alberta)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors