Skip to yearly menu bar Skip to main content


Oral
in
Workshop: MATH-AI: The 3rd Workshop on Mathematical Reasoning and AI

Learning the greatest divisor - Explainable predictions in transformers

Francois Charton

Keywords: [ transformers ] [ arithmetic ] [ explainability ]


Abstract: We train small transformers to calculate the greatest common divisor (GCD) of two positive integers, and show that their predictions are fully explainable. During training, models learn a list D of divisors, and predict the largest element of D that divides both inputs. We also show that training distributions have a large impact on performance. Models trained from uniform operands only learn a handful of GCD (up to 38 out of 100). Training from log-uniform operands boosts performance to 73 correct GCD, and training from a log-uniform distribution of GCD to 91.

Chat is not available.