A Geometry-Based View of Mahalanobis OOD Detection

Abstract

Out-of-distribution (OOD) detection is critical for reliable deployment of vision models, and Mahalanobis-based detectors remain strong baselines. However, their performance varies widely across modern pretrained representations, making it unclear which feature-space properties determine success or failure. Across diverse foundation-model backbones and Mahalanobis variants, we show that performance is highly representation-dependent and can shift substantially with pretraining data and fine-tuning. We link this variability to in-distribution geometry and identify a two-term ID summary that consistently tracks Mahalanobis OOD behavior across detectors: within-class spectral structure and local intrinsic dimensionality. Finally, we introduce radially scaled $\ell_2$ normalization, $\phi_\beta(z)=z/\|z\|^\beta$, a direction-preserving transformation that changes feature radii and exposes a different ID geometry to the same quadratic detector. Selecting $\beta$ from ID-only geometry signals generally outperforms fixed normalization baselines.