Timezone: »
There are many ways to turn a high-level program into a sequence of instructions consistent with that computation. Selecting the most performant such instruction sequence for a given piece of hardware - optimized compilation - is a central challenge of computer science. Optimizing compilers perform this task through a series of reductions and local transformations (e.g. register allocation, instruction scheduling, peephole optimization) driven by heuristics. A natural and well-explored avenue of research is to replace current hand-written heuristics by data-driven, automatically-designed heuristics which may be obtained from machine learning. We propose a radically different approach, in which we view compilation as a combinatorial optimization problem which consists of finding the optimal (e.g. fastest executing or shortest) sequence of instructions subject to the constraint that it has the semantics of the specified program. We show how this problem can be practically framed as a finite Markov decision process, unlocking a rich space of potential algorithms from reinforcement learning. We implement one such algorithm in particular, an AlphaGo-like distributed neural Monte-Carlo tree search procedure, and demonstrate that it is able to directly generate optimized assembly. Unlike a traditional optimizing compiler, this approach does not rely on an existing library of optimizations to transform the code, but rather directly attempts to generate the most optimal program instruction-by-instruction, taking into account effects including register allocation, instruction scheduling and operation fusion.
Author Information
Wenda Zhou (NYU / Flatiron Institute)
Olga Solodova (Princeton University)
Ryan Adams (Princeton University)
Related Events (a corresponding poster, oral, or spotlight)
-
2022 : A code superoptimizer through neural Monte-Carlo tree search »
Dates n/a. Room
More from the Same Authors
-
2021 Spotlight: Slice Sampling Reparameterization Gradients »
David Zoltowski · Diana Cai · Ryan Adams -
2021 Spotlight: Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate »
Xingyuan Sun · Tianju Xue · Szymon Rusinkiewicz · Ryan Adams -
2021 : ProBF: Probabilistic Safety Certificates with Barrier Functions »
Sulin Liu · Athindran Ramesh Kumar · Jaime Fisac · Ryan Adams · Peter J. Ramadge -
2021 : Reading the Road: Leveraging Meta-Learning to Learn Other Driver Behavior »
Anat Kleiman · Ryan Adams -
2022 Poster: Multi-fidelity Monte Carlo: a pseudo-marginal approach »
Diana Cai · Ryan Adams -
2021 : Randomized Automatic Differentiation - Ryan Adams - Princeton University »
Ryan Adams -
2021 Poster: Slice Sampling Reparameterization Gradients »
David Zoltowski · Diana Cai · Ryan Adams -
2021 Poster: Autobahn: Automorphism-based Graph Neural Nets »
Erik Thiede · Wenda Zhou · Risi Kondor -
2021 Poster: Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate »
Xingyuan Sun · Tianju Xue · Szymon Rusinkiewicz · Ryan Adams -
2021 Poster: Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability »
Dibya Ghosh · Jad Rahme · Aviral Kumar · Amy Zhang · Ryan Adams · Sergey Levine -
2020 : Orals 1.1: Randomized Automatic Differentiation »
Deniz Oktay · Nick McGreivy · Alex Beatson · Ryan Adams -
2020 Workshop: Machine Learning for Engineering Modeling, Simulation and Design »
Alex Beatson · Priya Donti · Amira Abdel-Rahman · Stephan Hoyer · Rose Yu · J. Zico Kolter · Ryan Adams -
2020 Poster: On Warm-Starting Neural Network Training »
Jordan Ash · Ryan Adams -
2020 Poster: Task-Agnostic Amortized Inference of Gaussian Process Hyperparameters »
Sulin Liu · Xingyuan Sun · Peter J. Ramadge · Ryan Adams -
2020 Poster: Learning Composable Energy Surrogates for PDE Order Reduction »
Alex Beatson · Jordan Ash · Geoffrey Roeder · Tianju Xue · Ryan Adams -
2020 Oral: Learning Composable Energy Surrogates for PDE Order Reduction »
Alex Beatson · Jordan Ash · Geoffrey Roeder · Tianju Xue · Ryan Adams -
2019 Poster: SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers »
Igor Fedorov · Ryan Adams · Matthew Mattina · Paul Whatmough -
2019 Poster: Discrete Object Generation with Reversible Inductive Construction »
Ari Seff · Wenda Zhou · Farhan Damani · Abigail Doyle · Ryan Adams -
2018 : Discussion Panel: Ryan Adams, Nicolas Heess, Leslie Kaelbling, Shie Mannor, Emo Todorov (moderator: Roy Fox) »
Ryan Adams · Nicolas Heess · Leslie Kaelbling · Shie Mannor · Emo Todorov · Roy Fox -
2018 : Inference and Control of Learning Behavior in Rodents (Ryan Adams) »
Ryan Adams -
2018 Poster: A Bayesian Nonparametric View on Count-Min Sketch »
Diana Cai · Michael Mitzenmacher · Ryan Adams -
2017 Poster: PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference »
Jonathan Huggins · Ryan Adams · Tamara Broderick -
2017 Spotlight: PASS-GLM: polynomial approximate sufficient statistics for scalable Bayesian GLM inference »
Jonathan Huggins · Ryan Adams · Tamara Broderick -
2017 Poster: Reducing Reparameterization Gradient Variance »
Andrew Miller · Nick Foti · Alexander D'Amour · Ryan Adams -
2016 : Panel Discussion »
Shakir Mohamed · David Blei · Ryan Adams · José Miguel Hernández-Lobato · Ian Goodfellow · Yarin Gal -
2016 : A Tribute to David MacKay »
Ryan Adams -
2016 Workshop: Bayesian Optimization: Black-box Optimization and Beyond »
Roberto Calandra · Bobak Shahriari · Javier Gonzalez · Frank Hutter · Ryan Adams -
2016 : Leveraging Structure in Bayesian Optimization »
Ryan Adams -
2016 Poster: Bayesian latent structure discovery from multi-neuron recordings »
Scott Linderman · Ryan Adams · Jonathan Pillow -
2016 Poster: Composing graphical models with neural networks for structured representations and fast inference »
Matthew Johnson · David Duvenaud · Alex Wiltschko · Ryan Adams · Sandeep R Datta -
2015 Workshop: Bayesian Optimization: Scalability and Flexibility »
Bobak Shahriari · Ryan Adams · Nando de Freitas · Amar Shah · Roberto Calandra -
2015 Workshop: Statistical Methods for Understanding Neural Systems »
Alyson Fletcher · Jakob H Macke · Ryan Adams · Jascha Sohl-Dickstein -
2015 Poster: Convolutional Networks on Graphs for Learning Molecular Fingerprints »
David Duvenaud · Dougal Maclaurin · Jorge Iparraguirre · Rafael Bombarell · Timothy Hirzel · Alan Aspuru-Guzik · Ryan Adams -
2015 Poster: A Gaussian Process Model of Quasar Spectral Energy Distributions »
Andrew Miller · Albert Wu · Jeffrey Regier · Jon McAuliffe · Dustin Lang · Mr. Prabhat · David Schlegel · Ryan Adams -
2015 Poster: Spectral Representations for Convolutional Neural Networks »
Oren Rippel · Jasper Snoek · Ryan Adams -
2015 Poster: Dependent Multinomial Models Made Easy: Stick-Breaking with the Polya-gamma Augmentation »
Scott Linderman · Matthew Johnson · Ryan Adams -
2014 Workshop: Bayesian Optimization in Academia and Industry »
Zoubin Ghahramani · Ryan Adams · Matthew Hoffman · Kevin Swersky · Jasper Snoek -
2014 Poster: A framework for studying synaptic plasticity with neural spike train data »
Scott Linderman · Christopher H Stock · Ryan Adams -
2013 Workshop: Bayesian Optimization in Theory and Practice »
Matthew Hoffman · Jasper Snoek · Nando de Freitas · Michael A Osborne · Ryan Adams · Sebastien Bubeck · Philipp Hennig · Remi Munos · Andreas Krause -
2013 Poster: Multi-Task Bayesian Optimization »
Kevin Swersky · Jasper Snoek · Ryan Adams -
2013 Poster: Message Passing Inference with Chemical Reaction Networks »
Nils E Napp · Ryan Adams -
2013 Oral: Message Passing Inference with Chemical Reaction Networks »
Nils E Napp · Ryan Adams -
2013 Poster: A Determinantal Point Process Latent Variable Model for Inhibition in Neural Spiking Data »
Jasper Snoek · Richard Zemel · Ryan Adams -
2013 Poster: Contrastive Learning Using Spectral Methods »
James Y Zou · Daniel Hsu · David Parkes · Ryan Adams -
2012 Poster: Bayesian n-Choose-k Models for Classification and Ranking »
Kevin Swersky · Danny Tarlow · Richard Zemel · Ryan Adams · Brendan J Frey -
2012 Poster: Priors for Diversity in Generative Latent Variable Models »
James Y Zou · Ryan Adams -
2012 Poster: Cardinality Restricted Boltzmann Machines »
Kevin Swersky · Danny Tarlow · Ilya Sutskever · Richard Zemel · Russ Salakhutdinov · Ryan Adams -
2012 Poster: Practical Bayesian Optimization of Machine Learning Algorithms »
Jasper Snoek · Hugo Larochelle · Ryan Adams -
2011 Workshop: Bayesian Nonparametric Methods: Hope or Hype? »
Emily Fox · Ryan Adams -
2010 Workshop: Transfer Learning Via Rich Generative Models. »
Russ Salakhutdinov · Ryan Adams · Josh Tenenbaum · Zoubin Ghahramani · Tom Griffiths -
2010 Workshop: Monte Carlo Methods for Bayesian Inference in Modern Day Applications »
Ryan Adams · Mark A Girolami · Iain Murray -
2010 Oral: Tree-Structured Stick Breaking for Hierarchical Data »
Ryan Adams · Zoubin Ghahramani · Michael Jordan -
2010 Oral: Slice sampling covariance hyperparameters of latent Gaussian models »
Iain Murray · Ryan Adams -
2010 Poster: Tree-Structured Stick Breaking for Hierarchical Data »
Ryan Adams · Zoubin Ghahramani · Michael Jordan -
2010 Poster: Slice sampling covariance hyperparameters of latent Gaussian models »
Iain Murray · Ryan Adams -
2008 Poster: The Gaussian Process Density Sampler »
Ryan Adams · Iain Murray · David MacKay -
2008 Spotlight: The Gaussian Process Density Sampler »
Ryan Adams · Iain Murray · David MacKay