Timezone: »
Sum-product networks are a new deep architecture that can perform fast, exact inference on high-treewidth models. Only generative methods for training SPNs have been proposed to date. In this paper, we present the first discriminative training algorithms for SPNs, combining the high accuracy of the former with the representational power and tractability of the latter. We show that the class of tractable discriminative SPNs is broader than the class of tractable generative ones, and propose an efficient backpropagation-style algorithm for computing the gradient of the conditional log likelihood. Standard gradient descent suffers from the diffusion problem, but networks with many layers can be learned reliably using ''hard'' gradient descent, where marginal inference is replaced by MPE inference (i.e., inferring the most probable state of the non-evidence variables). The resulting updates have a simple and intuitive form. We test discriminative SPNs on standard image classification tasks. We obtain the best results to date on the CIFAR-10 dataset, using fewer features than prior methods with an SPN architecture that learns local image structure discriminatively. We also report the highest published test accuracy on STL-10 even though we only use the labeled portion of the dataset.
Author Information
Robert Gens (University of Washington)
Pedro Domingos (University of Washington)
Related Events (a corresponding poster, oral, or spotlight)
-
2012 Poster: Discriminative Learning of Sum-Product Networks »
Thu. Dec 6th through Wed the 5th Room Harrah’s Special Events Center 2nd Floor
More from the Same Authors
-
2018 : Invited Talk 6 »
Pedro Domingos -
2018 Poster: Submodular Field Grammars: Representation, Inference, and Application to Image Parsing »
Abram Friesen · Pedro Domingos -
2015 : Discussion Panel with Morning Speakers (Day 1) »
Pedro Domingos · Stephen H Muggleton · Rina Dechter · Josh Tenenbaum -
2015 : Sum-Product Networks and Tractable Markov Logic: And End-to-End Neural-Symbolic System »
Pedro Domingos -
2014 Poster: Deep Symmetry Networks »
Robert Gens · Pedro Domingos -
2010 Poster: Learning Efficient Markov Networks »
Vibhav Gogate · William A Webb · Pedro Domingos -
2010 Poster: Approximate Inference by Compilation to Arithmetic Circuits »
Daniel Lowd · Pedro Domingos