# Personalized Language Models via Privacy-Preserving Evolutionary Model Merging

**Authors:** Kyuyoung Kim, Jinwoo Shin, Jaehyung Kim  
**arXiv:** [2503.18008](https://arxiv.org/abs/2503.18008)  
**Date:** 2025-03-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2503.18008v1/

## Key points

- The study focuses on tailoring large language models (LLMs) to individual user or user group preferences.
- Traditional methods like prompt-based and training-based approaches lack direct optimization of task-specific metrics and explicit privacy-preservation mechanisms.
- PriME is a novel approach that uses gradient-free methods to optimize task-specific metrics while preserving user privacy.
- PriME outperforms prompt-based and training-based methods, achieving up to a 45% performance improvement over existing techniques.
- PriME demonstrates a better privacy-utility trade-off compared to other methods, showcasing the potential of evolutionary approaches for privacy-preserving LLM personalization.
- In classification tasks, PriME shows relative improvements in accuracy and F1 score compared to Per-Pcs, highlighting its effectiveness in optimizing task-specific metrics.
- Challenges are observed in reproducing results from previous studies, suggesting sensitivity to hyperparameter choices or limitations in the autoregressive approach used by Per-Pcs.

## Summary

The study on Personalized Language Models via Privacy-Preserving Evolutionary Model Merging (PriME) focuses on tailoring large language models (LLMs) to individual user or user group preferences. Traditional methods such as prompt-based and training-based approaches have shown some success in personalizing LLMs, but they often lack direct optimization of task-specific metrics and explicit privacy-preservation mechanisms. To address these limitations, the researchers propose PriME, a novel approach that utilizes gradient-free methods to optimize task-specific metrics while preserving user privacy. PriME incorporates privacy preservation into the optimization process to create a personalized module that captures the target user's preferences effectively while minimizing privacy risks for users sharing their private information. Experimental results on the LaMP benchmark demonstrate that PriME outperforms both prompt-based and training-based methods, achieving up to a 45% performance improvement over existing techniques. Additionally, PriME shows a significantly better privacy-utility trade-off, highlighting the potential of evolutionary approaches for privacy-preserving LLM personalization. Further analysis reveals that PriME consistently outperforms Per-Pcs and other baseline methods across various public personalization tasks in the LaMP benchmark. In classification tasks, PriME demonstrates relative improvements in accuracy and F1 score compared to Per-Pcs, showcasing its effectiveness in optimizing task-specific metrics. However, there are challenges observed in reproducing results from previous studies, indicating potential sensitivity to hyperparameter choices or limitations in the autoregressive approach used by Per-Pcs. Overall, the study underscores the importance of incorporating privacy preservation mechanisms into personalized LLMs and highlights the promising results achieved by PriME through evolutionary algorithms for effective personalization while safeguarding user privacy.