We propose RoTaR, a row-based table representation learning method, to address the efficiency and scalability issues faced by existing table representation learning methods. The key idea of RoTaR is to generate query-agnostic row representations that could be re-used via query-specific aggregation. In addition to the row-based architecture, we introduce several techniques: cell-aware position embedding, AutoEncoder objective in transformer models, teacher-student training paradigm, and selective backward to improve the performance of RoTaR model.
Zui Chen (Tsinghua University, Tsinghua University)
Zui Chen is an undergraudate student from Tsinghua University. His research interest includes natural language processing, knowledge graph, machine learning, database, and AI music.
Lei Cao (University of Arizona/MIT)
Assistant Professor of University of Arizona and Research Scientist at MIT
Samuel Madden (Massachusetts Institute of Technology)
More from the Same Authors
2022 : Generic Entity Resolution Models »
Jiawei Tang · Yifei Zuo · Lei Cao · Samuel Madden
2021 Poster: Self-Supervised Multi-Object Tracking with Cross-input Consistency »
Favyen Bastani · Songtao He · Samuel Madden