Exécution de notre outil de synthèse sur un nouvel article

Résumé : Fine-tuning Vision Foundation Models (VFMs) has become a standard practice for adapting large pre-trained models to downstream tasks. Previously, with lightweight architectures such as ResNet and VGG, ensemble methods were widely adopted due to their strong empirical performance gains. However, as VFMs have grown substantially in scale, ensembling multiple fine-tuned models has become increasingly impractical, primarily due to prohibitive inference-time and computational costs. To retain the benefits of ensembling while avoiding these costs, weight averaging has emerged as a practical single-model alternative by merging parameters from multiple trained networks. However, naive weight averaging often suffers from performance degradation due to parameter misalignment across independently trained networks. To address this trade-off, we propose Branch Soup, a weight averaging method for parameter-efficient fine-tuning (PEFT) of VFMs, in which branches share a common initialization and optimization trajectory up to a predefined branching point before diverging independently. Beyond proposing this method, we ask when and why branch-and-average is reliable in the adapter subspace. Using linear-mode-connectivity and Hessian measurements, we decompose adapter merging into three effects. Sharing a shallow trained seed is what makes the branches linearly connected and prevents the collapse of naive averaging. Deepening the shared trajectory then trades branch diversity for basin flatness. Because these two effects offset, the branching point is non-critical over a broad range and can be set online without additional calibration. Consistent with this analysis, Branch Soup matches or exceeds previous weight averaging baselines, including SWA, Greedy Soup, and PAPA, across diverse benchmarks and backbone architectures, with a statistically significant gain over the strongest baseline in most configurations, five of these from unconditional two-sample tests against stochastic baselines and the remaining four conditional on a single Greedy Soup pool. Notably, on challenging medical datasets with high class imbalance, Branch Soup attains strong and reliable generalization without any additional inference cost. By grounding branch-and-average in the geometry of the loss landscape rather than in algorithmic novelty alone, our work offers a mechanistic account of reliable weight merging for adapter-based model adaptation.