Results of the summarizing process for the arXiv paper: 2502.20653v1

, , , , In the field of deep learning, dataset distillation has become a popular method for reducing data requirements. Among the various techniques used, distribution matching-based approaches have gained attention for their efficient computational performance and strong results. However, existing distance metrics employed in distribution matching often fall short in accurately capturing distributional differences, leading to unreliable measures of discrepancy. To address this issue, a new approach is proposed in this paper that reframes dataset distillation as a minmax optimization problem. The authors introduce Neural Characteristic Function Discrepancy (NCFD), a comprehensive metric grounded in theory for measuring distributional variances. NCFD leverages the Characteristic Function (CF) to encapsulate complete distributional information and utilizes a neural network to optimize the sampling strategy for the CF's frequency arguments. This optimization process aims to maximize the discrepancy between distributions to enhance distance estimation accuracy. Furthermore, the authors minimize the distinction between real and synthetic data under this optimized NCFD measure. Their novel method, named Neural Characteristic Function Matching (\mymethod{}), aligns the phase and amplitude of neural features in the complex plane for both real and synthetic data. This alignment helps achieve a balance between realism and diversity in synthetic samples. Experimental results demonstrate significant performance improvements over state-of-the-art methods on datasets with varying resolutions. Notably, there is a remarkable 20.5% accuracy boost observed on ImageSquawk dataset. Additionally, the proposed method reduces GPU memory usage by over 300 times and achieves processing speeds that are 20 times faster compared to existing techniques. Remarkably, this work marks a milestone achievement as it accomplishes lossless compression of CIFAR-100 using only 2.3 GB of memory on a single NVIDIA 2080 Ti GPU – an accomplishment not reported before in literature. Accepted by CVPR 2025 with 11 pages and 7 figures, this research showcases promising advancements in dataset distillation through Neural Characteristic Function Matching methodology developed by Shaobo Wang et al., which will be presented at Conference on Computer Vision and Pattern Recognition in 2025.

• - Dataset distillation is a popular method in deep learning for reducing data requirements.
• - Existing distance metrics in distribution matching may not accurately capture distributional differences, leading to unreliable measures of discrepancy.
• - A new approach reframes dataset distillation as a minmax optimization problem and introduces Neural Characteristic Function Discrepancy (NCFD) as a comprehensive metric for measuring distributional variances.
• - NCFD leverages the Characteristic Function (CF) to encapsulate complete distributional information and utilizes a neural network to optimize the sampling strategy for the CF's frequency arguments.
• - The proposed method, Neural Characteristic Function Matching (\mymethod{}), aligns the phase and amplitude of neural features in the complex plane for both real and synthetic data to achieve a balance between realism and diversity in synthetic samples.
• - Experimental results show significant performance improvements over state-of-the-art methods on datasets with varying resolutions, with a notable 20.5% accuracy boost observed on ImageSquawk dataset.
• - The method reduces GPU memory usage by over 300 times and achieves processing speeds that are 20 times faster compared to existing techniques.
• - The research marks a milestone achievement by accomplishing lossless compression of CIFAR-100 using only 2.3 GB of memory on a single NVIDIA 2080 Ti GPU, which has not been reported before in literature.

Summary- Dataset distillation is a way to use less data in deep learning. - Some ways of comparing data distributions may not be very accurate. - A new method called Neural Characteristic Function Discrepancy (NCFD) helps measure differences in data distributions better. - NCFD uses a special function and neural networks to improve how data is sampled. - The new method, Neural Characteristic Function Matching, improves how synthetic data looks and performs. Definitions- Dataset distillation: Using less data for deep learning tasks. - Distribution: How things are spread out or organized. - Metric: A way to measure or compare something. - Optimization: Making something work as well as possible. - Synthetic: Something made artificially, not real.

Introduction

Dataset distillation has emerged as a popular technique in the field of deep learning for reducing data requirements. Among the various methods used, distribution matching-based approaches have gained attention for their efficient computational performance and strong results. However, existing distance metrics employed in distribution matching often fall short in accurately capturing distributional differences, leading to unreliable measures of discrepancy. In this research paper titled "Neural Characteristic Function Matching: A Comprehensive Metric for Distribution Matching", Shaobo Wang et al. propose a novel approach to dataset distillation that reframes it as a minmax optimization problem. The authors introduce Neural Characteristic Function Discrepancy (NCFD), a comprehensive metric grounded in theory for measuring distributional variances.

The Problem with Existing Distance Metrics

The authors highlight the limitations of existing distance metrics used in distribution matching techniques. These metrics are based on statistical moments or kernel functions and do not take into account the entire distributional information. This leads to inaccurate measures of discrepancy between distributions. Moreover, these metrics fail to capture complex relationships between different features and cannot handle high-dimensional data efficiently. As a result, they are unable to provide reliable estimates of distance between distributions.

Introducing Neural Characteristic Function Discrepancy (NCFD)

To address these issues, the authors propose NCFD - a new metric that leverages the Characteristic Function (CF) to encapsulate complete distributional information. The CF is defined as the Fourier transform of probability density function (PDF) and contains both phase and amplitude components. The authors use a neural network to optimize the sampling strategy for frequency arguments of CFs from real and synthetic datasets. This optimization process aims to maximize the discrepancy between distributions, thereby enhancing distance estimation accuracy.

Minimizing Distinction Between Real and Synthetic Data

In addition to maximizing the discrepancy between distributions, the authors also minimize the distinction between real and synthetic data under the optimized NCFD measure. This is achieved through their proposed method - Neural Characteristic Function Matching (\mymethod{}). This approach aligns the phase and amplitude of neural features in the complex plane for both real and synthetic data. By doing so, it strikes a balance between realism and diversity in synthetic samples.

Experimental Results

The proposed \mymethod{} was evaluated on datasets with varying resolutions, including CIFAR-10, CIFAR-100, ImageNet, and ImageSquawk. The results demonstrate significant performance improvements over state-of-the-art methods. Notably, there was a remarkable 20.5% accuracy boost observed on ImageSquawk dataset using \mymethod{}. Additionally, this method reduced GPU memory usage by over 300 times and achieved processing speeds that were 20 times faster compared to existing techniques. One of the most impressive achievements of this research is lossless compression of CIFAR-100 using only 2.3 GB of memory on a single NVIDIA 2080 Ti GPU – something that has not been reported before in literature.

Conclusion

In conclusion, Shaobo Wang et al.'s research paper presents a novel approach to dataset distillation through Neural Characteristic Function Matching methodology. Their work addresses key limitations of existing distance metrics used in distribution matching techniques and showcases promising advancements in this field. Accepted by CVPR 2025 with 11 pages and 7 figures, this research marks a milestone achievement that will be presented at Conference on Computer Vision and Pattern Recognition in 2025. With its efficient computational performance and strong results on various datasets, \mymethod{} has potential applications in various fields such as computer vision, natural language processing (NLP), speech recognition, etc. Overall, this paper contributes significantly to the field of deep learning and opens up new avenues for future research in dataset distillation.