Demystifying Mergeability: Interpretable Properties to Predict Model Merging Success

Abstract

Model merging combines knowledge from separately fine-tuned models, yet the factors driving its success remain poorly understood. While recent work treats mergeability as an intrinsic property of the models, we show with an architecture-agnostic framework that it fundamentally depends on both the merging method and the partner tasks. Using L1-regularized linear optimization over a set of interpretable pairwise metrics (e.g., gradient L2 distance), we uncover properties correlating with post-merge normalized accuracy across five merging methods. We find that the drivers of merge success vary across architectures and merging methods, overall with only a moderate agreement (64.0% average top-5 metric overlap; 79.3% sign agreement). Crucially, however, gradient alignment metrics consistently emerge as the most fundamental signals of mergeability. These findings provide a diagnostic foundation for understanding mergeability and motivate future merge-aware fine-tuning strategies.