Bilinear Bandits with Partially Observable Features

Abstract

We study bilinear bandits with partially observable features on both the user and item sides. In each of $T$ rounds, the learner selects a user--item pair and observes only the reward for the chosen pair. The reward model is bilinear in the user and item features with an unknown parameter matrix. Existing work commonly reduces this problem to a linear bandit via Kronecker-product features, increasing dimensionality and losing bilinear structure. We propose BiRoLF, an algorithm robust to latent features, which directly leverages the bilinear structure without such linearization. It augments observed feature spaces on both sides with orthogonal complement bases and employs doubly robust (DR) estimation to impute rewards for unselected pairs, constructing matrix-valued pseudo-rewards. We estimate the effective parameter using a bilinear DR-Lasso estimator, which promotes sparsity in components orthogonal to observed features. BiRoLF achieves a $\tilde{O}(\sqrt{(d_x + d_{h_x})(d_y + d_{h_y}) T})$ regret bound, where $d_x$ and $d_y$ are observable feature dimensions, and $d_{h_x}$ and $d_{h_y}$ denote effective orthogonal-complement dimensions. By exploiting the induced block-diagonal Gram structure, BiRoLF performs an exact blockwise update that preserves the full DR-Lasso objective. Numerical experiments show strong regret performance and computational gains.