Timezone: »
For several problems of interest, there are natural constraints which exist over the output label space. For example, for the joint task of NER and POS labeling, these constraints might specify that the NER label ‘organization’ is consistent only with the POS labels ‘noun’ and ‘preposition’. These constraints can be a great way of injecting prior knowledge into a deep learning model, thereby improving overall performance. In this paper, we present a constrained optimization formulation for training a deep network with a given set of hard constraints on output labels. Our novel approach first converts the label constraints into soft logic constraints over probability distributions outputted by the network. It then converts the constrained optimization problem into an alternating min-max optimization with Lagrangian variables defined for each constraint. Since the constraints are independent of the target labels, our framework easily generalizes to semi-supervised setting. We experiment on the tasks of Semantic Role Labeling (SRL), Named Entity Recognition (NER) tagging, and fine-grained entity typing and show that our constraints not only significantly reduce the number of constraint violations, but can also result in state-of-the-art performance
Author Information
Yatin Nandwani (Indian Institute Of Technology Delhi)
Abhishek Pathak (Indian Institute Of Technology, Delhi)
Mausam (IIT Dehli)
Parag Singla (Indian Institute of Technology Delhi)
More from the Same Authors
-
2018 : Spotlights 2 »
Mausam · Ankit Anand · Parag Singla · Tarik Koc · Tim Klinger · Habibeh Naderi · Sungwon Lyu · Saeed Amizadeh · Kshitij Dwivedi · Songpeng Zu · Wei Feng · Balaraman Ravindran · Edouard Pineau · Abdulkadir Celikkanat · Deepak Venugopal -
2018 : Spotlights »
Guangneng Hu · Ke Li · Aviral Kumar · Phi Vu Tran · Samuel Fadel · Rita Kuznetsova · Bong-Nam Kang · Behrouz Haji Soleimani · Jinwon An · Nathan de Lara · Anjishnu Kumar · Tillman Weyde · Melanie Weber · Kristen Altenburger · Saeed Amizadeh · Xiaoran Xu · Yatin Nandwani · Yang Guo · Maria Pacheco · William Fedus · Guillaume Jaume · Yuka Yoneda · Yunpu Ma · Yunsheng Bai · Berk Kapicioglu · Maximilian Nickel · Fragkiskos Malliaros · Beier Zhu · Aleksandar Bojchevski · Joshua Joseph · Gemma Roig · Esma Balkir · Xander Steenbrugge -
2018 Poster: Transfer of Deep Reactive Policies for MDP Planning »
Aniket (Nick) Bajpai · Sankalp Garg · Mausam -
2015 Poster: Fast Lifted MAP Inference via Partitioning »
Somdeb Sarkhel · Parag Singla · Vibhav Gogate -
2015 Poster: Lifted Inference Rules With Constraints »
Happy Mittal · Anuj Mahajan · Vibhav Gogate · Parag Singla -
2015 Poster: Lifted Symmetry Detection and Breaking for MAP Inference »
Timothy Kopp · Parag Singla · Henry Kautz -
2014 Poster: An Integer Polynomial Programming Based Framework for Lifted MAP Inference »
Somdeb Sarkhel · Deepak Venugopal · Parag Singla · Vibhav Gogate -
2014 Poster: New Rules for Domain Independent Lifted MAP Inference »
Happy Mittal · Prasoon Goyal · Vibhav Gogate · Parag Singla