# Improve3C: Data Cleaning on Consistency and Completeness with Currency

**Authors:** Xiaoou Ding, Hongzhi Wang, Jiaxuan Su, Jianzhong Li, Hong Gao  
**arXiv:** [1808.00024](https://arxiv.org/abs/1808.00024)  
**Date:** 2018-07-31  
**Summary page:** https://www.summarizepaper.com/en/paper/1808.00024v1/

## Key points

- Ensuring data quality is crucial in the field of big data management
- Exponential growth of data from various sources brings forth challenges in maintaining data quality
- The paper focuses on improving completeness, consistency, and currency of data through a 4-step framework called Improve3C
- Improve3C aims to detect and improve the quality of incomplete and inconsistent data without timestamps by computing a relative currency order among records based on given criteria
- Inconsistent repair is prioritized before incomplete repair for enhanced effectiveness and efficiency
- A currency-related consistency distance metric is introduced to accurately measure similarity between dirty records and clean ones
- Currency orders are incorporated as an important feature in training models for incompleteness repair
- Solution algorithms are provided with examples to illustrate their application
- Experiments using real-life and synthetic datasets validate the proposed method's effectiveness in cleaning dirty data with multiple quality problems
- The paper highlights the significance of data quality in big data management and presents a comprehensive framework for addressing completeness, consistency, and currency issues.

## Summary

In the field of big data management, ensuring  is crucial due to the exponential growth of data from various sources. However, this growth also brings forth multiple challenges in maintaining the quality of data. In response to these challenges, this paper focuses on improving the completeness, consistency, and currency of data through a 4-step framework called . This framework aims to detect and improve the quality of incomplete and inconsistent data that lacks timestamps by computing a relative currency order among records based on given . This allows for effective repair of inconsistent and incomplete data by considering their temporal impact. To enhance both effectiveness and efficiency, the authors prioritize inconsistent repair before incomplete repair. They also introduce a currency-related consistency distance metric to accurately measure the similarity between dirty records (inconsistent or incomplete) and clean ones. Additionally, they incorporate currency orders as an important feature in training models for incompleteness repair. The paper provides detailed explanations of solution algorithms along with examples to illustrate their application. To validate their proposed method, the authors conduct experiments using real-life and synthetic datasets. The results demonstrate that Improve3C effectively improves the performance of cleaning dirty data with multiple quality problems that are challenging for existing approaches. Overall, this paper highlights the significance of  in big data management and presents a comprehensive framework for addressing issues related to completeness, consistency, and currency. The proposed method offers an efficient solution for cleaning dirty data by considering temporal impact and incorporating currency orders as a key feature in repairing incomplete records.