Timezone: »
Deep topic models have been proven as a promising way to extract hierarchical latent representations from documents represented as high-dimensional bag-of-words vectors.However, the representation capability of existing deep topic models is still limited by the phenomenon of "posterior collapse", which has been widely criticized in deep generative models, resulting in the higher-level latent representations exhibiting similar or meaningless patterns.To this end, in this paper, we first develop a novel deep-coupling generative process for existing deep topic models, which incorporates skip connections into the generation of documents, enforcing strong links between the document and its multi-layer latent representations.After that, utilizing data augmentation techniques, we reformulate the deep-coupling generative process as a Markov decision process and develop a corresponding Policy Gradient (PG) based training algorithm, which can further alleviate the information reduction at higher layers.Extensive experiments demonstrate that our developed methods can effectively alleviate "posterior collapse" in deep topic models, contributing to providing higher-quality latent document representations.
Author Information
Yewen Li (nanyang technological university)
Chaojie Wang (Nanyang Technological University)
Zhibin Duan (Xidian University)
Dongsheng Wang (Xidian University)
Bo Chen (Xidian University)
Bo An (Nanyang Technological University)
Mingyuan Zhou (University of Texas at Austin)
More from the Same Authors
-
2022 Poster: Generalizing Consistent Multi-Class Classification with Rejection to be Compatible with Arbitrary Losses »
Yuzhou Cao · Tianchi Cai · Lei Feng · Lihong Gu · Jinjie GU · Bo An · Gang Niu · Masashi Sugiyama -
2022 Poster: Knowledge-Aware Bayesian Deep Topic Model »
Dongsheng Wang · Yishi Xu · Miaoge Li · Zhibin Duan · Chaojie Wang · Bo Chen · Mingyuan Zhou -
2022 Poster: HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding »
Yishi Xu · Dongsheng Wang · Bo Chen · Ruiying Lu · Zhibin Duan · Mingyuan Zhou -
2022 : Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-Oriented Dialogue Systems »
Yihao Feng · Shentao Yang · Shujian Zhang · Jianguo Zhang · Caiming Xiong · Mingyuan Zhou · Huan Wang -
2022 : Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-Oriented Dialogue Systems »
Yihao Feng · Shentao Yang · Shujian Zhang · Jianguo Zhang · Caiming Xiong · Mingyuan Zhou · Huan Wang -
2022 : Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning »
Zhendong Wang · jonathan j hunt · Mingyuan Zhou -
2023 Poster: Beta Diffusion »
Mingyuan Zhou · Tianqi Chen · Huangjie Zheng · Zhendong Wang -
2023 Poster: Offline RL with Discrete Proxy Representations for Generalizability in POMDPs »
Pengjie Gu · Xinyu Cai · Dong Xing · Xinrun Wang · Mengchen Zhao · Bo An -
2023 Poster: Regression with Cost-based Rejection »
Xin Cheng · Yuzhou Cao · Haobo Wang · Hongxin Wei · Bo An · Lei Feng -
2023 Poster: Computing Optimal Nash Equilibria in Multiplayer Games »
Youzhi Zhang · Bo An · Venkatramanan Subrahmanian -
2023 Poster: Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models »
Zhendong Wang · Yifan Jiang · Huangjie Zheng · Peihao Wang · Pengcheng He · Zhangyang Wang · Weizhu Chen · Mingyuan Zhou -
2023 Poster: In Defense of Softmax Parametrization for Calibrated and Consistent Learning to Defer »
Yuzhou Cao · Hussein Mozannar · Lei Feng · Hongxin Wei · Bo An -
2023 Poster: Few-shot Generation via Recalling the Episodic-Semantic Memory like Human Being »
Zhibin Duan · Zhiyi Lv · Chaojie Wang · Bo Chen · Bo An · Mingyuan Zhou -
2023 Poster: Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection »
Ruiying Lu · YuJie Wu · Long Tian · Dongsheng Wang · Bo Chen · Xiyang Liu · Ruimin Hu -
2023 Poster: Context-guided Embedding Adaptation for Effective Topic Modeling in Low-Resource Regimes »
Yishi Xu · Jianqiao Sun · Yudi Su · Xinyang Liu · Zhibin Duan · Bo Chen · Mingyuan Zhou -
2023 Poster: State Regularized Policy Optimization on Data with Dynamics Shift »
Zhenghai Xue · Qingpeng Cai · Shuchang Liu · Dong Zheng · Peng Jiang · Kun Gai · Bo An -
2023 Poster: On the Importance of Feature Separability in Predicting Out-Of-Distribution Error »
RENCHUNZI XIE · Hongxin Wei · Lei Feng · Yuzhou Cao · Bo An -
2023 Poster: Tuning Multi-mode Token-level Prompt Alignment across Modalities »
Dongsheng Wang · Miaoge Li · Xinyang Liu · MingSheng Xu · Bo Chen · Hanwang Zhang -
2023 Poster: Preference-grounded Token-level Guidance for Language Model Fine-tuning »
Shentao Yang · Shujian Zhang · Congying Xia · Yihao Feng · Caiming Xiong · Mingyuan Zhou -
2023 Poster: In-Context Learning Unlocked for Diffusion Models »
Zhendong Wang · Yifan Jiang · Yadong Lu · yelong shen · Pengcheng He · Weizhu Chen · Zhangyang Wang · Mingyuan Zhou -
2023 Poster: TradeMaster: A Holistic Quantitative Trading Platform Empowered by Reinforcement Learning »
Shuo Sun · Molei Qin · Wentao Zhang · Haochong Xia · Chuqiao Zong · Jie Ying · Yonggang Xie · Lingxuan Zhao · Xinrun Wang · Bo An -
2022 Spotlight: Deep Attentive Belief Propagation: Integrating Reasoning and Learning for Solving Constraint Optimization Problems »
Yanchen Deng · Shufeng Kong · Caihua Liu · Bo An -
2022 Spotlight: Lightning Talks 6A-1 »
Ziyi Wang · Nian Liu · Yaming Yang · Qilong Wang · Yuanxin Liu · Zongxin Yang · Yizhao Gao · Yanchen Deng · Dongze Lian · Nanyi Fei · Ziyu Guan · Xiao Wang · Shufeng Kong · Xumin Yu · Daquan Zhou · Yi Yang · Fandong Meng · Mingze Gao · Caihua Liu · Yongming Rao · Zheng Lin · Haoyu Lu · Zhe Wang · Jiashi Feng · Zhaolin Zhang · Deyu Bo · Xinchao Wang · Chuan Shi · Jiangnan Li · Jiangtao Xie · Jie Zhou · Zhiwu Lu · Wei Zhao · Bo An · Jiwen Lu · Peihua Li · Jian Pei · Hao Jiang · Cai Xu · Peng Fu · Qinghua Hu · Yijie Li · Weigang Lu · Yanan Cao · Jianbin Huang · Weiping Wang · Zhao Cao · Jie Zhou -
2022 Spotlight: Lightning Talks 5B-4 »
Yuezhi Yang · Zeyu Yang · Yong Lin · Yishi Xu · Linan Yue · Tao Yang · Weixin Chen · Qi Liu · Jiaqi Chen · Dongsheng Wang · Baoyuan Wu · Yuwang Wang · Hao Pan · Shengyu Zhu · Zhenwei Miao · Yan Lu · Lu Tan · Bo Chen · Yichao Du · Haoqian Wang · Wei Li · Yanqing An · Ruiying Lu · Peng Cui · Nanning Zheng · Li Wang · Zhibin Duan · Xiatian Zhu · Mingyuan Zhou · Enhong Chen · Li Zhang -
2022 Spotlight: HyperMiner: Topic Taxonomy Mining with Hyperbolic Embedding »
Yishi Xu · Dongsheng Wang · Bo Chen · Ruiying Lu · Zhibin Duan · Mingyuan Zhou -
2022 Spotlight: Lightning Talks 2A-4 »
Sarthak Mittal · Richard Grumitt · Zuoyu Yan · Lihao Wang · Dongsheng Wang · Alexander Korotin · Jiangxin Sun · Ankit Gupta · Vage Egiazarian · Tengfei Ma · Yi Zhou · Yishi Xu · Albert Gu · Biwei Dai · Chunyu Wang · Yoshua Bengio · Uros Seljak · Miaoge Li · Guillaume Lajoie · Yiqun Wang · Liangcai Gao · Lingxiao Li · Jonathan Berant · Huang Hu · Xiaoqing Zheng · Zhibin Duan · Hanjiang Lai · Evgeny Burnaev · Zhi Tang · Zhi Jin · Xuanjing Huang · Chaojie Wang · Yusu Wang · Jian-Fang Hu · Bo Chen · Chao Chen · Hao Zhou · Mingyuan Zhou -
2022 Spotlight: Knowledge-Aware Bayesian Deep Topic Model »
Dongsheng Wang · Yishi Xu · Miaoge Li · Zhibin Duan · Chaojie Wang · Bo Chen · Mingyuan Zhou -
2022 Poster: Learning to Re-weight Examples with Optimal Transport for Imbalanced Classification »
Dandan Guo · Zhuo Li · meixi zheng · He Zhao · Mingyuan Zhou · Hongyuan Zha -
2022 Poster: Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transport »
Dandan Guo · Long Tian · He Zhao · Mingyuan Zhou · Hongyuan Zha -
2022 Poster: Deep Attentive Belief Propagation: Integrating Reasoning and Learning for Solving Constraint Optimization Problems »
Yanchen Deng · Shufeng Kong · Caihua Liu · Bo An -
2022 Poster: A Variational Edge Partition Model for Supervised Graph Representation Learning »
Yilin He · Chaojie Wang · Hao Zhang · Bo Chen · Mingyuan Zhou -
2022 Poster: Out-of-Distribution Detection with An Adaptive Likelihood Ratio on Informative Hierarchical VAE »
Yewen Li · Chaojie Wang · Xiaobo Xia · Tongliang Liu · xin miao · Bo An -
2022 Poster: A Unified Framework for Alternating Offline Model Training and Policy Learning »
Shentao Yang · Shujian Zhang · Yihao Feng · Mingyuan Zhou -
2022 Poster: CARD: Classification and Regression Diffusion Models »
Xizewen Han · Huangjie Zheng · Mingyuan Zhou -
2022 Poster: Pluralistic Image Completion with Gaussian Mixture Models »
Xiaobo Xia · Wenhao Yang · Jie Ren · Yewen Li · Yibing Zhan · Bo Han · Tongliang Liu -
2021 Poster: Exploiting Chain Rule and Bayes' Theorem to Compare Probability Distributions »
Huangjie Zheng · Mingyuan Zhou -
2021 Poster: RMIX: Learning Risk-Sensitive Policies for Cooperative Reinforcement Learning Agents »
Wei Qiu · Xinrun Wang · Runsheng Yu · Rundong Wang · Xu He · Bo An · Svetlana Obraztsova · Zinovi Rabinovich -
2021 Poster: Alignment Attention by Matching Key and Query Distributions »
Shujian Zhang · Xinjie Fan · Huangjie Zheng · Korawat Tanwisuth · Mingyuan Zhou -
2021 Poster: Probabilistic Margins for Instance Reweighting in Adversarial Training »
qizhou wang · Feng Liu · Bo Han · Tongliang Liu · Chen Gong · Gang Niu · Mingyuan Zhou · Masashi Sugiyama -
2021 Poster: Convex Polytope Trees »
Mohammadreza Armandpour · Ali Sadeghian · Mingyuan Zhou -
2021 Poster: TopicNet: Semantic Graph-Guided Topic Discovery »
Zhibin Duan · Yishi Xu · Bo Chen · Dongsheng Wang · Chaojie Wang · Mingyuan Zhou -
2021 Poster: A Prototype-Oriented Framework for Unsupervised Domain Adaptation »
Korawat Tanwisuth · Xinjie Fan · Huangjie Zheng · Shujian Zhang · Hao Zhang · Bo Chen · Mingyuan Zhou -
2021 Poster: Open-set Label Noise Can Improve Robustness Against Inherent Label Noise »
Hongxin Wei · Lue Tao · RENCHUNZI XIE · Bo An -
2021 Poster: CARMS: Categorical-Antithetic-REINFORCE Multi-Sample Gradient Estimator »
Alek Dimitriev · Mingyuan Zhou -
2020 Poster: Bidirectional Convolutional Poisson Gamma Dynamical Systems »
wenchao chen · Chaojie Wang · Bo Chen · Yicheng Liu · Hao Zhang · Mingyuan Zhou -
2020 Poster: Implicit Distributional Reinforcement Learning »
Yuguang Yue · Zhendong Wang · Mingyuan Zhou -
2020 Poster: Deep Relational Topic Modeling via Graph Poisson Gamma Belief Network »
Chaojie Wang · Hao Zhang · Bo Chen · Dongsheng Wang · Zhengjue Wang · Mingyuan Zhou -
2020 Poster: Bayesian Attention Modules »
Xinjie Fan · Shujian Zhang · Bo Chen · Mingyuan Zhou -
2019 Poster: Manipulating a Learning Defender and Ways to Counteract »
Jiarui Gan · Qingyu Guo · Long Tran-Thanh · Bo An · Michael Wooldridge -
2019 Poster: Variational Graph Recurrent Neural Networks »
Ehsan Hajiramezanali · Arman Hasanzadeh · Krishna Narayanan · Nick Duffield · Mingyuan Zhou · Xiaoning Qian -
2019 Poster: Semi-Implicit Graph Variational Auto-Encoders »
Arman Hasanzadeh · Ehsan Hajiramezanali · Krishna Narayanan · Nick Duffield · Mingyuan Zhou · Xiaoning Qian -
2019 Poster: Poisson-Randomized Gamma Dynamical Systems »
Aaron Schein · Scott Linderman · Mingyuan Zhou · David Blei · Hanna Wallach -
2018 Poster: Nonparametric Bayesian Lomax delegate racing for survival analysis with competing risks »
Quan Zhang · Mingyuan Zhou -
2018 Poster: Deep Poisson gamma dynamical systems »
Dandan Guo · Bo Chen · Hao Zhang · Mingyuan Zhou -
2018 Poster: Dirichlet belief networks for topic structure learning »
He Zhao · Lan Du · Wray Buntine · Mingyuan Zhou -
2018 Poster: Parsimonious Bayesian deep networks »
Mingyuan Zhou -
2018 Poster: Masking: A New Perspective of Noisy Supervision »
Bo Han · Jiangchao Yao · Gang Niu · Mingyuan Zhou · Ivor Tsang · Ya Zhang · Masashi Sugiyama -
2018 Poster: Bayesian multi-domain learning for cancer subtype discovery from next-generation sequencing count data »
Ehsan Hajiramezanali · Siamak Zamani Dadaneh · Alireza Karbalayghareh · Mingyuan Zhou · Xiaoning Qian -
2016 Poster: Poisson-Gamma dynamical systems »
Aaron Schein · Hanna Wallach · Mingyuan Zhou -
2016 Oral: Poisson-Gamma dynamical systems »
Aaron Schein · Hanna Wallach · Mingyuan Zhou -
2015 Poster: The Poisson Gamma Belief Network »
Mingyuan Zhou · Yulai Cong · Bo Chen -
2014 Poster: Beta-Negative Binomial Process and Exchangeable Random Partitions for Mixed-Membership Modeling »
Mingyuan Zhou -
2012 Poster: Augment-and-Conquer Negative Binomial Processes »
Mingyuan Zhou · Lawrence Carin -
2012 Spotlight: Augment-and-Conquer Negative Binomial Processes »
Mingyuan Zhou · Lawrence Carin -
2011 Poster: On the Analysis of Multi-Channel Neural Spike Data »
Bo Chen · David Carlson · Lawrence Carin -
2009 Poster: Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations »
Mingyuan Zhou · Haojun Chen · John Paisley · Lu Ren · Guillermo Sapiro · Lawrence Carin -
2009 Oral: Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations »
Mingyuan Zhou · Haojun Chen · John Paisley · Lu Ren · Guillermo Sapiro · Larry Carin