# Tracing and Visualizing Human-ML/AI Collaborative Processes through Artifacts of Data Work

**Authors:** Jennifer Rogers and, Anamaria Crisan  
**arXiv:** [2304.02699](https://arxiv.org/abs/2304.02699)  
**Date:** 2023-04-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.02699v1/

## Key points

- The field of data work has become complex, requiring a range of skills that are often inaccessible to non-technical experts.
- Automated Machine Learning (AutoML) technology automates certain aspects of data work, such as model selection and data preparation.
- AutoML still requires human intervention to be functional, resulting in a complex and collaborative process that can be difficult to trace.
- Researchers have constructed a taxonomy of data work artifacts that captures both the human and machine-generated processes involved in AutoML.
- The taxonomy was developed through an extensive literature review spanning multiple fields.
- The resulting taxonomy is concise, robust, comprehensive, extensible and explanatory.
- It comprises multiple interrelated phases that leverage statistical and computational techniques for data preparation, analysis, deployment and communication.
- To operationalize the taxonomy further, the researchers developed AutoMLTrace - a visual interactive sketch showing both the context and temporality of human-ML/AI collaboration in data work.
- They demonstrated its utility via a usage scenario with an enterprise software development team.

## Summary

The field of data work has become increasingly complex, requiring a range of skills that are often inaccessible to non-technical experts. Automated Machine Learning (AutoML) technology has emerged as a solution to this problem by automating certain aspects of data work, such as model selection and data preparation. However, AutoML still requires human intervention to be functional, resulting in a complex and collaborative process that can be difficult to trace. To address this issue, researchers have constructed a taxonomy of data work artifacts that captures both the human and machine-generated processes involved in AutoML. The taxonomy was developed through an extensive literature review spanning Machine Learning, Human Computer Interaction, Computer Supported Collaborative Work, Information Visualization, and Visual Analytics. The research team gathered an initial set of 13 papers before identifying a systematic set of published research and pre-prints on AutoML. They then conducted eight iterations to develop the taxonomy by reading the literature sources, extracting artifacts that met the definition of the meta characteristic, classifying those items, and grouping them according to an evolving set of artifact properties. The resulting taxonomy is concise, robust, comprehensive, extensible and explanatory. It comprises multiple interrelated phases that leverage statistical and computational techniques for data preparation, analysis, deployment and communication. The taxonomy also includes artifact identification and classification methods for capturing artifacts with their properties and dependencies. To operationalize the taxonomy further, the researchers developed AutoMLTrace - a visual interactive sketch showing both the context and temporality of human-ML/AI collaboration in data work. They demonstrated its utility via a usage scenario with an enterprise software development team. Finally, reflecting on their approach to developing the taxonomy through a broad literature review but assessing its utility primarily through a collaboration with one team; they acknowledge more work is required to assess its generality but remain optimistic about its potential for developing data visualization tools that interrogate sociotechnical relationships in automated data work.