Timezone: »
Document intelligence automates the extraction of information from documents and supports many business applications. Recent self-supervised learning methods on large-scale unlabeled document datasets have opened up promising directions towards reducing annotation efforts by training models with self-supervised objectives. However, most of the existing document pretraining methods are still language-dominated. We present UDoc, a new unified pretraining framework for document understanding. UDoc is designed to support most document understanding tasks, extending the Transformer to take multimodal embeddings as input. Each input element is composed of words and visual features from a semantic region of the input document image. An important feature of UDoc is that it learns a generic representation by making use of three self-supervised losses, encouraging the representation to model sentences, learn similarities, and align modalities. Extensive empirical analysis demonstrates that the pretraining procedure learns better joint representations and leads to improvements in downstream tasks.
Author Information
Jiuxiang Gu (Adobe Research)
Jason Kuen (Nanyang Technological University)
Vlad I Morariu (University of Maryland, College Park)
Handong Zhao (Adobe Research)
Rajiv Jain (Adobe Systems)
Nikolaos Barmpalios (Adobe Systems)
Ani Nenkova (Adobe Research)
Tong Sun (Adobe Research)
Accomplished research thought-leader and technology innovator with a 15+ years proven track of leadership in incubating new concepts through state-of-art machine learning methods/tools, developing advanced rapid prototypes, and delivering competitive technologies to market opportunities in a cross-disciplinary and cross-functional team environment. Held 22 issued US Patents, 40+ peer-reviewed publications in prestigious conferences and journals. Specialties: R&D leadership, leading-edge innovation strategy, machine learning, natural language processing and understanding, data-driven cybersecurity, social media analytics, big data center of excellence, service oriented architecture, distributed & cloud computing.
More from the Same Authors
-
2021 : User-in-the-Loop Named Entity Recognition via Counterfactual Learning »
Tong Yu · Junda Wu · Ruiyi Zhang · Handong Zhao · Shuai Li -
2022 Poster: Delving into Out-of-Distribution Detection with Vision-Language Representations »
Yifei Ming · Ziyang Cai · Jiuxiang Gu · Yiyou Sun · Wei Li · Yixuan Li -
2021 Poster: Slow Learning and Fast Inference: Efficient Graph Similarity Computation via Knowledge Distillation »
Can Qin · Handong Zhao · Lichen Wang · Huan Wang · Yulun Zhang · Yun Fu -
2020 Poster: Self-Supervised Relationship Probing »
Jiuxiang Gu · Jason Kuen · Shafiq Joty · Jianfei Cai · Vlad I. Morariu · Handong Zhao · Tong Sun -
2008 Poster: Automatic online tuning for fast Gaussian summation »
Vlad I Morariu · Balaji Vasan Srinivasan · Vikas C Raykar · Ramani Duraiswami · Larry Davis