Timezone: »
Dense event captioning aims to detect and describe all events of interest contained in a video. Despite the advanced development in this area, existing methods tackle this task by making use of dense temporal annotations, which is dramatically source-consuming. This paper formulates a new problem: weakly supervised dense event captioning, which does not require temporal segment annotations for model training. Our solution is based on the one-to-one correspondence assumption, each caption describes one temporal segment, and each temporal segment has one caption, which holds in current benchmark datasets and most real world cases. We decompose the problem into a pair of dual problems: event captioning and sentence localization and present a cycle system to train our model. Extensive experimental results are provided to demonstrate the ability of our model on both dense event captioning and sentence localization in videos.
Author Information
Xin Wang (Tsinghua University)
Wenbing Huang (Tencent AI Lab)
Chuang Gan (MIT-IBM Watson AI Lab)
Jingdong Wang (Microsoft Research,)
Wenwu Zhu (Tsinghua University)
Junzhou Huang (University of Texas at Arlington / Tencent AI Lab)
More from the Same Authors
-
2020 Poster: Revisiting Parameter Sharing for Automatic Neural Channel Number Search »
Jiaxing Wang · Haoli Bai · Jiaxiang Wu · Xupeng Shi · Junzhou Huang · Irwin King · Michael Lyu · Jian Cheng -
2020 Poster: MCUNet: Tiny Deep Learning on IoT Devices »
Ji Lin · Wei-Ming Chen · Yujun Lin · john cohn · Chuang Gan · Song Han -
2020 Poster: Implicit Graph Neural Networks »
Fangda Gu · Heng Chang · Wenwu Zhu · Somayeh Sojoudi · Laurent El Ghaoui -
2020 Spotlight: MCUNet: Tiny Deep Learning on IoT Devices »
Ji Lin · Wei-Ming Chen · Yujun Lin · john cohn · Chuang Gan · Song Han -
2020 Poster: Dirichlet Graph Variational Autoencoder »
Jia Li · Jianwei Yu · Jiajin Li · Honglei Zhang · Kangfei Zhao · Yu Rong · Hong Cheng · Junzhou Huang -
2020 Poster: TinyTL: Reduce Memory, Not Parameters for Efficient On-Device Learning »
Han Cai · Chuang Gan · Ligeng Zhu · Song Han -
2020 Poster: RetroXpert: Decompose Retrosynthesis Prediction Like A Chemist »
Chaochao Yan · Qianggang Ding · Peilin Zhao · Shuangjia Zheng · JINYU YANG · Yang Yu · Junzhou Huang -
2020 Spotlight: RetroXpert: Decompose Retrosynthesis Prediction Like A Chemist »
Chaochao Yan · Qianggang Ding · Peilin Zhao · Shuangjia Zheng · JINYU YANG · Yang Yu · Junzhou Huang -
2020 Poster: Self-Supervised Graph Transformer on Large-Scale Molecular Data »
Yu Rong · Yatao Bian · Tingyang Xu · Weiyang Xie · Ying WEI · Wenbing Huang · Junzhou Huang -
2020 Poster: Deep Multimodal Fusion by Channel Exchanging »
Yikai Wang · Wenbing Huang · Fuchun Sun · Tingyang Xu · Yu Rong · Junzhou Huang -
2020 Poster: Adversarial Sparse Transformer for Time Series Forecasting »
Sifan Wu · Xi Xiao · Qianggang Ding · Peilin Zhao · Ying Wei · Junzhou Huang -
2019 Poster: Semantic Conditioned Dynamic Modulation for Temporal Sentence Grounding in Videos »
Yitian Yuan · Lin Ma · Jingwen Wang · Wei Liu · Wenwu Zhu -
2019 Poster: Cross-channel Communication Networks »
Jianwei Yang · Zhile Ren · Chuang Gan · Hongyuan Zhu · Devi Parikh -
2019 Poster: Hyperparameter Learning via Distributional Transfer »
Ho Chung Law · Peilin Zhao · Leung Sing Chan · Junzhou Huang · Dino Sejdinovic -
2019 Poster: DTWNet: a Dynamic Time Warping Network »
Xingyu Cai · Tingyang Xu · Jinfeng Yi · Junzhou Huang · Sanguthevar Rajasekaran -
2019 Poster: Visual Concept-Metaconcept Learning »
Chi Han · Jiayuan Mao · Chuang Gan · Josh Tenenbaum · Jiajun Wu -
2019 Poster: Learning Disentangled Representations for Recommendation »
Jianxin Ma · Chang Zhou · Peng Cui · Hongxia Yang · Wenwu Zhu -
2019 Poster: NAT: Neural Architecture Transformer for Accurate and Compact Architectures »
Yong Guo · Yin Zheng · Mingkui Tan · Qi Chen · Jian Chen · Peilin Zhao · Junzhou Huang -
2019 Poster: Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement »
Chao Yang · Xiaojian Ma · Wenbing Huang · Fuchun Sun · Huaping Liu · Junzhou Huang · Chuang Gan -
2019 Spotlight: Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement »
Chao Yang · Xiaojian Ma · Wenbing Huang · Fuchun Sun · Huaping Liu · Junzhou Huang · Chuang Gan -
2018 Poster: Discrimination-aware Channel Pruning for Deep Neural Networks »
Zhuangwei Zhuang · Mingkui Tan · Bohan Zhuang · Jing Liu · Yong Guo · Qingyao Wu · Junzhou Huang · Jinhui Zhu -
2018 Poster: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2018 Spotlight: Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding »
Kexin Yi · Jiajun Wu · Chuang Gan · Antonio Torralba · Pushmeet Kohli · Josh Tenenbaum -
2018 Poster: Adaptive Sampling Towards Fast Graph Representation Learning »
Wenbing Huang · Tong Zhang · Yu Rong · Junzhou Huang -
2017 Poster: Efficient Optimization for Linear Dynamical Systems with Applications to Clustering and Sparse Coding »
Wenbing Huang · Mehrtash Harandi · Tong Zhang · Lijie Fan · Fuchun Sun · Junzhou Huang -
2012 Poster: Compressive Sensing MRI with Wavelet Tree Sparsity »
Chen Chen · Junzhou Huang