# DataCI: A Platform for Data-Centric AI on Streaming Data

**Authors:** Huaizheng Zhang, Yizheng Huang, Yuanming Li  
**arXiv:** [2306.15538](https://arxiv.org/abs/2306.15538)  
**Date:** 2023-06-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.15538v2/

## Key points

- DataCI is an open-source platform for data-centric AI in dynamic streaming data settings
- It provides infrastructure and APIs for seamless streaming dataset management, pipeline development, and evaluation
- Versioning control function tracks pipeline lineage and graphical interface enhances user experience
- User experience investigation includes a playground with data selection, pipeline launching, and experiment details
- Quantitative analysis simulates a real-world case using Yelp dataset in streaming mode
- New pipeline versions are continuously developed using the latest data, but version 8 fails to outperform version 7
- Using older versions without frequent updates leads to significant drops in online performance
- DataCI addresses shortcomings of existing tools by streamlining streaming data management and method deployment
- Preliminary studies demonstrate its potential to revolutionize data-centric AI in dynamic contexts
- Further exploration is needed to determine upgrade frequency and better metrics for measuring pipeline performance in streaming scenarios
- DataCI offers an efficient platform for developing and evaluating data-centric AI models in streaming data settings

## Summary

DataCI is an open-source platform that aims to revolutionize data-centric AI in dynamic streaming data settings. It provides a comprehensive infrastructure with rich APIs for seamless streaming dataset management, data-centric pipeline development, and evaluation on streaming scenarios. The platform also offers a carefully designed versioning control function to track the lineage of pipelines and an intuitive graphical interface for an enhanced user experience. To demonstrate the effectiveness and usability of DataCI, two perspectives are considered: user experience investigation and quantitative analysis. In terms of user experience, DataCI prioritizes users' satisfaction by providing a playground where they can interactively try out the system. The playground consists of three sections: data selection from Streaming Data Sink and pre-defined pipelines from Pipeline Registry, manual pipeline launching with visualization through directed acyclic graphs (DAGs), and presentation of experiment running details for reference. For quantitative analysis, a real-world case is simulated using Yelp dataset in a streaming mode. Starting from pipeline version 5 (v5), a new pipeline version 6 (v6) is developed and deployed after passing an A/B test. Subsequent versions are continuously developed using the latest data from Streaming Data Sink. However, version 8 (v8) fails to outperform version 7 (v7). Additionally, it is observed that if v6 is used without frequent updates, online performance drops significantly. This preliminary study highlights the necessity of a system like DataCI for quick building and evaluating data-centric pipelines on streaming data due to frequent changes in data distributions. In conclusion, DataCI addresses the shortcomings of existing tools in streaming data environments by streamlining streaming data management and method deployment through its modular features and intuitive interface. Preliminary studies demonstrate its potential to revolutionize data-centric AI in dynamic contexts. Further exploration is needed to determine upgrade frequency and identify better metrics for measuring pipeline performance in streaming scenarios. Overall, DataCI offers researchers and practitioners an efficient platform for developing and evaluating data-centric AI models in streaming data settings, ultimately advancing the field of data-centric AI.