Timezone: »

 
Poster
An Efficient Pessimistic-Optimistic Algorithm for Stochastic Linear Bandits with General Constraints
Xin Liu · Bin Li · Pengyi Shi · Lei Ying

Tue Dec 07 08:30 AM -- 10:00 AM (PST) @ Virtual
This paper considers stochastic linear bandits with general nonlinear constraints. The objective is to maximize the expected cumulative reward over horizon $T$ subject to a set of constraints in each round $\tau\leq T$. We propose a pessimistic-optimistic algorithm for this problem, which is efficient in two aspects. First, the algorithm yields $\tilde{\cal O}\left(\left(\frac{K^{0.75}}{\delta}+d\right)\sqrt{\tau}\right)$ (pseudo) regret in round $\tau\leq T,$ where $K$ is the number of constraints, $d$ is the dimension of the reward feature space, and $\delta$ is a Slater's constant; and {\em zero} constraint violation in any round $\tau>\tau',$ where $\tau'$ is {\em independent} of horizon $T.$ Second, the algorithm is computationally efficient. Our algorithm is based on the primal-dual approach in optimization and includes two components. The primal component is similar to unconstrained stochastic linear bandits (our algorithm uses the linear upper confidence bound algorithm (LinUCB)). The computational complexity of the dual component depends on the number of constraints, but is independent of the sizes of the contextual space, the action space, and the feature space. Thus, the computational complexity of our algorithm is similar to LinUCB for unconstrained stochastic linear bandits.

Author Information

Xin Liu (ShanghaiTech University)
Bin Li (Pennsylvania State University)
Pengyi Shi (Purdue University)
Lei Ying (University of Michigan, Ann Arbor)

More from the Same Authors