# HETAL: Efficient Privacy-preserving Transfer Learning with Homomorphic Encryption

**Authors:** Seewoo Lee, Garam Lee, Jung Woo Kim, Junbum Shin, Mun-Kyu Lee  
**arXiv:** [2403.14111](https://arxiv.org/abs/2403.14111)  
**Date:** 2024-03-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2403.14111v1/

## Key points

- Transfer learning is a widely accepted method in machine learning for training models efficiently, especially when data is limited.
- The approach involves fine-tuning pre-trained models on large datasets by adding new classification layers to adapt them to specific tasks.
- Ensuring data privacy is a main challenge in transfer learning, leading researchers to explore homomorphic encryption as a solution.
- HETAL is an innovative Homomorphic Encryption based Transfer Learning algorithm introduced by Seewoo Lee, Garam Lee, Jung Woo Kim, Junbum Shin, and Mun-Kyu Lee.
- HETAL leverages the CKKS homomorphic encryption scheme to securely encrypt client data and preserves privacy during training tasks while achieving comparable accuracy to non-encrypted methods.
- The research team developed efficient encrypted matrix multiplication and softmax approximation algorithms that significantly enhance efficiency in encrypted computations.
- Experimental results show that HETAL's total training times across five benchmark datasets ranged from 567 to 3442 seconds, demonstrating practicality and effectiveness in preserving privacy without compromising performance.

## Summary

In the field of machine learning, transfer learning has become a widely accepted method for training models efficiently. This is especially useful in scenarios where data is limited. The approach involves taking a pre-trained model on large datasets and fine-tuning it by adding new classification layers to adapt it to specific tasks. One of the main challenges in transfer learning is ensuring data privacy, particularly in the context of machine learning as a service. To address this issue, researchers have explored the use of homomorphic encryption - a technique that allows computations to be performed on encrypted data without decrypting it. A team of researchers including Seewoo Lee, Garam Lee, Jung Woo Kim, Junbum Shin, and Mun-Kyu Lee introduced HETAL - an innovative Homomorphic Encryption based Transfer Learning algorithm. It stands out as the first practical scheme that enables encrypted training while safeguarding client privacy during training tasks. HETAL leverages the CKKS homomorphic encryption scheme to securely encrypt client data. By incorporating validation-based early stopping mechanisms and achieving comparable accuracy to non-encrypted training methods, HETAL demonstrates its effectiveness in preserving privacy without compromising performance. Moreover, the research team proposed novel techniques to enhance efficiency in encrypted computations. They developed an efficient encrypted matrix multiplication algorithm that outperforms existing methods by up to 323 times in speed. Additionally, they introduced a highly precise softmax approximation algorithm with improved coverage. The experimental results presented by the authors showcase the practicality and effectiveness of HETAL. Across five well-known benchmark datasets, total training times ranged from 567 to 3442 seconds - less than an hour overall. This study not only contributes significantly to advancing privacy-preserving transfer learning but also sets a new standard for secure and efficient machine learning practices in sensitive data environments.