NeurIPS Poster Pessimism for Offline Linear Contextual Bandits using $\ell

Poster

Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence Sets

Gene Li · Cong Ma · Nati Srebro

Hall J (level 1) #823

Keywords: [ offline reinforcement learning ] [ pessimism ] [ linear contextual bandits ]

[ Abstract ]

[ Paper] [ Poster] [ OpenReview]

Abstract: We present a family

{{\hat{π}}_{p}}_{p \geq 1}

$\{\widehat{\pi}_p\}_{p\ge 1}$ of pessimistic learning rules for offline learning of linear contextual bandits, relying on confidence sets with respect to different

ℓ_{p}

$\ell_p$ norms, where

{\hat{π}}_{2}

$\widehat{\pi}_2$ corresponds to Bellman-consistent pessimism (BCP), while

{\hat{π}}_{\infty}

$\widehat{\pi}_\infty$ is a novel generalization of lower confidence bound (LCB) to the linear setting. We show that the novel

{\hat{π}}_{\infty}

$\widehat{\pi}_\infty$ learning rule is, in a sense, adaptively optimal, as it achieves the minimax performance (up to log factors) against all

ℓ_{q}

$\ell_q$ -constrained problems, and as such it strictly dominates all other predictors in the family, including

{\hat{π}}_{2}

$\widehat{\pi}_2$ .

Chat is not available.

Poster

Pessimism for Offline Linear Contextual Bandits using ℓpℓp\ell_p Confidence Sets

Gene Li · Cong Ma · Nati Srebro

Hall J (level 1) #823

Pessimism for Offline Linear Contextual Bandits using $\ell_p$ Confidence Sets