Timezone: »

 
Poster
Solving Quantitative Reasoning Problems with Language Models
Aitor Lewkowycz · Anders Andreassen · David Dohan · Ethan Dyer · Henryk Michalewski · Vinay Ramasesh · Ambrose Slone · Cem Anil · Imanol Schlag · Theo Gutman-Solo · Yuhuai Wu · Behnam Neyshabur · Guy Gur-Ari · Vedant Misra

Tue Nov 29 02:00 PM -- 04:00 PM (PST) @ Hall J #920
in Poster Session 2 »

Language models have achieved remarkable performance on a wide range of tasks that require natural language understanding. Nevertheless, state-of-the-art models have generally struggled with tasks that require quantitative reasoning, such as solving mathematics, science, and engineering questions at the college level. To help close this gap, we introduce Minerva, a large language model pretrained on general natural language data and further trained on technical content. The model achieves strong performance in a variety of evaluations, including state-of-the-art performance on the MATH dataset. We also evaluate our model on over two hundred undergraduate-level problems in physics, biology, chemistry, economics, and other sciences that require quantitative reasoning, and find that the model can correctly answer nearly a quarter of them.

Keywords: language models · transformers · quantitative reasoning · math and science word problems

Author Information

Aitor Lewkowycz (Inflection AI)
Anders Andreassen (Google)
David Dohan (Google Brain)
Ethan Dyer (Blueshift, Google Research)
Henryk Michalewski (Google)
Vinay Ramasesh (Google)
Ambrose Slone (Google)

Currently at Google X as a part of team doing deep learning research. Formerly at Apple working on computer vision and deep learning.

Cem Anil (University of Toronto)

I'm a first year PhD student at the University of Toronto and Vector Institute, supervised by Roger Grosse and Geoffrey Hinton.

Imanol Schlag (IDSIA)
Theo Gutman-Solo
Yuhuai Wu (Google)
Behnam Neyshabur (Google)
Guy Gur-Ari (Google)
Vedant Misra (Google)

More from the Same Authors