Exécution de notre outil de synthèse sur un nouvel article
C'est la première fois que cet article est demandé et nos outils de synthèse d'IA n'ont jamais été exécutés dessus. Nous pouvons exécuter nos outils maintenant si vous cliquez sur le bouton "Exécuter" sur la page mais assurez-vous d'abord que c'est le bon article.
Watermarking Makes Language Models Radioactive
Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, Teddy Furon
Résumé : This paper investigates the radioactivity of LLM-generated texts, i.e. whether it is possible to detect that such input was used as training data. Conventional methods like membership inference can carry out this detection with some level of accuracy. We show that watermarked training data leaves traces easier to detect and much more reliable than membership inference. We link the contamination level to the watermark robustness, its proportion in the training set, and the fine-tuning process. We notably demonstrate that training on watermarked synthetic instructions can be detected with high confidence (p-value < 1e-5) even when as little as 5% of training text is watermarked. Thus, LLM watermarking, originally designed for detecting machine-generated text, gives the ability to easily identify if the outputs of a watermarked LLM were used to fine-tune another LLM.