Timezone: »

 
Poster
Noise-Contrastive Estimation for Multivariate Point Processes
Hongyuan Mei · Tom Wan · Jason Eisner

Thu Dec 10 09:00 PM -- 11:00 PM (PST) @ Poster Session 6 #1762
in Poster Session 7 »

The log-likelihood of a generative model often involves both positive and negative terms. For a temporal multivariate point process, the negative term sums over all the possible event types at each time and also integrates over all the possible times. As a result, maximum likelihood estimation is expensive. We show how to instead apply a version of noise-contrastive estimation---a general parameter estimation method with a less expensive stochastic objective. Our specific instantiation of this general idea works out in an interestingly non-trivial way and has provable guarantees for its optimality, consistency and efficiency. On several synthetic and real-world datasets, our method shows benefits: for the model to achieve the same level of log-likelihood on held-out data, our method needs considerably fewer function evaluations and less wall-clock time.

Author Information

Hongyuan Mei (JOHNS HOPKINS UNIVERSITY)

I am a final-year Ph.D. student (2016-) in Department of Computer Science at Johns Hopkins University, affiliated with the Center for Language and Speech Processing, where I am advised by Jason Eisner. My research interests are rooted in designing models and algorithms to solve challenging real-life problems, with particular interest in continuous-time modeling and natural language processing.

Tom Wan (JOHNS HOPKINS UNIVERSITY)
Jason Eisner (Johns Hopkins + Microsoft)

Jason Eisner is Professor of Computer Science at Johns Hopkins University, as well as Director of Research at Microsoft Semantic Machines. He is a Fellow of the Association for Computational Linguistics. At Johns Hopkins, he is also affiliated with the Center for Language and Speech Processing, the Machine Learning Group, the Cognitive Science Department, and the national Center of Excellence in Human Language Technology. His goal is to develop the probabilistic modeling, inference, and learning techniques needed for a unified model of all kinds of linguistic structure. His 135+ papers have presented various algorithms for parsing, machine translation, and weighted finite-state machines; formalizations, algorithms, theorems, and empirical results in computational phonology; and unsupervised or semi-supervised learning methods for syntax, morphology, and word-sense disambiguation. He is also the lead designer of Dyna, a new declarative programming language that provides an infrastructure for AI research. He has received two school-wide awards for excellence in teaching, as well as recent Best Paper Awards at ACL 2017 and EMNLP 2019.

More from the Same Authors