# A Survey on Sampling and Profiling over Big Data (Technical Report)

**Authors:** Zhicheng Liu, Aoqian Zhang  
**arXiv:** [2005.05079](https://arxiv.org/abs/2005.05079)  
**Date:** 2020-05-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2005.05079v1/

## Key points

- The exponential growth of data due to internet technology and computer science has brought opportunities and challenges.
- Big data allows for analysis of vast amounts of information to uncover hidden insights and potential value.
- The 5V characteristic of big data, particularly Volume, poses significant storage and processing challenges for traditional data mining algorithms, machine learning algorithms, and data profiling tasks.
- Sampling methods have been widely studied and used in the context of big data to address these challenges by reducing the amount of data while speeding up processing times.
- This paper focuses on researching sampling and profiling in a big data context by investigating the application of sampling in different categories of data profiling tasks.
- Sampled results are close to or even exceed those obtained from analyzing the full amount of data, making sampling technology an important tool in handling big data today with reason to believe that it will become an indispensable step in future big-data processing.
- Determining sample size methods combined with big-data processing frameworks can help researchers overcome computational limitations associated with large datasets while still providing accurate results for various types of analyses.

## Summary

The exponential growth of data due to the development of internet technology and computer science has brought both opportunities and challenges. Big data allows us to analyze and mine vast amounts of information to uncover hidden insights and potential value. However, the 5V characteristic of big data, particularly Volume, poses significant storage and processing challenges for traditional data mining algorithms, machine learning algorithms, and data profiling tasks. The large amount of data requires substantial hardware resources and is time-consuming. To address these challenges, sampling methods have been widely studied and used in the context of big data. Sampling technology effectively reduces the amount of data while speeding up processing times. This paper focuses on researching sampling and profiling in a big data context by investigating the application of sampling in different categories of data profiling tasks. Data profiling is an activity that finds metadata about a dataset with many use cases such as performing anomaly detection or repairing relational, graph, or time series data. However, it is computationally expensive for large datasets. From experimental results presented in this survey paper by Zhicheng Liu and Aoqian Zhang titled "A Survey on Sampling and Profiling over Big Data," sampled results are close to or even exceed those obtained from analyzing the full amount of data. Therefore, sampling technology plays an important role in handling big data today with reason to believe that it will become an indispensable step in future big-data processing. By determining sample size methods combined with big-data processing frameworks can help researchers overcome computational limitations associated with large datasets while still providing accurate results for various types of analyses.