Deep Gaussian Processes for Preference Learning

AI tools intended to assist human decision-making must be able to model the latent and potentially noisy preferences of their users. Pairwise comparisons between alternative items are an efficient means by which preference data can be solicited from users. In the preference learning literature, Gaussian processes have been used to construct flexible, generalizable models of pairwise preferences, but a key drawback of these approaches is runtime: performing Gaussian process inference requires computing a matrix inversion in cubic time. In this work, we introduce a new method for training Gaussian process preference models based on neural networks, for which a forward pass requires only linear time. Our models use Siamese neural network architectures, which enable the prediction of both utility function valuations for individual items as well as pairwise preference probabilities. Using two popular benchmark datasets, we show that our models can achieve predictive accuracy competitive with existing preference learning methods while requiring only a fraction of the time for evaluation.