Timezone: »
Poster
A Rate Distortion Approach for Semi-Supervised Conditional Random Fields
Yang Wang · Gholamreza Haffari · Shaojun Wang · Greg Mori
We propose a novel information theoretic approach for semi-supervised learning of conditional random fields. Our approach defines a training objective that combines the conditional likelihood on labeled data and the mutual information on unlabeled data. Different from previous minimum conditional entropy semi-supervised discriminative learning methods, our approach can be naturally cast into the rate distortion theory framework in information theory. We analyze the tractability of the framework for structured prediction and present a convergent variational training algorithm to defy the combinatorial explosion of terms in the sum over label configurations. Our experimental results show that the rate distortion approach outperforms standard $l_2$ regularization and minimum conditional entropy regularization on both multi-class classification and sequence labeling problems.
Author Information
Yang Wang (University of Manitoba)
Gholamreza Haffari (Monash University)
Shaojun Wang (Wright State University)
Greg Mori (Borealis AI)
More from the Same Authors
-
2021 : [O2] Not too close and not too far: enforcing monotonicity requires penalizing the right points »
Joao Monteiro · · Hossein Hajimirsadeghi · Greg Mori -
2021 Poster: Continuous Latent Process Flows »
Ruizhi Deng · Marcus Brubaker · Greg Mori · Andreas Lehrmann -
2020 Poster: Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows »
Ruizhi Deng · Bo Chang · Marcus Brubaker · Greg Mori · Andreas Lehrmann -
2018 Poster: Probabilistic Neural Programmed Networks for Scene Generation »
Zhiwei Deng · Jiacheng Chen · YIFANG FU · Greg Mori -
2018 Spotlight: Probabilistic Neural Programmed Networks for Scene Generation »
Zhiwei Deng · Jiacheng Chen · YIFANG FU · Greg Mori -
2016 Poster: Infinite Hidden Semi-Markov Modulated Interaction Point Process »
matt zhang · Peng Lin · Peng Lin · Ting Guo · Yang Wang · Yang Wang · Fang Chen -
2013 Poster: Direct 0-1 Loss Minimization and Margin Maximization with Boosting »
Shaodan Zhai · Tian Xia · Ming Tan · Shaojun Wang -
2013 Poster: Action is in the Eye of the Beholder: Eye-gaze Driven Model for Spatio-Temporal Action Localization »
Nataliya Shapovalova · Michalis Raptis · Leonid Sigal · Greg Mori -
2013 Poster: Latent Maximum Margin Clustering »
Guang-Tong Zhou · Tian Lan · Arash Vahdat · Greg Mori -
2012 Poster: Kernel Latent SVM for Visual Recognition »
Weilong Yang · Yang Wang · Arash Vahdat · Greg Mori -
2010 Poster: A Discriminative Latent Model of Image Region and Object Tag Correspondence »
Yang Wang · Greg Mori -
2010 Poster: Beyond Actions: Discriminative Models for Contextual Group Activities »
Tian Lan · Yang Wang · Weilong Yang · Greg Mori -
2008 Poster: Learning a discriminative hidden part model for human action recognition »
Yang Wang · Greg Mori -
2008 Spotlight: Learning a discriminative hidden part model for human action recognition »
Yang Wang · Greg Mori -
2006 Poster: Learning to Model Spatial Dependency: Semi-Supervised Discriminative Random Fields »
Chi-Hoon Lee · Shaojun Wang · Feng Jiao · Dale Schuurmans · Russell Greiner -
2006 Poster: implicit Online Learning with Kernels »
Li Cheng · Vishwanathan S V N · Dale Schuurmans · Shaojun Wang · Terry Caelli