The field of data work has become increasingly complex, requiring a range of skills that are often inaccessible to non-technical experts. Automated Machine Learning (AutoML) technology has emerged as a solution to this problem by automating certain aspects of data work, such as model selection and data preparation. However, AutoML still requires human intervention to be functional, resulting in a complex and collaborative process that can be difficult to trace. To address this issue, researchers have constructed a taxonomy of data work artifacts that captures both the human and machine-generated processes involved in AutoML. The taxonomy was developed through an extensive literature review spanning Machine Learning, Human Computer Interaction, Computer Supported Collaborative Work, Information Visualization, and Visual Analytics. The research team gathered an initial set of 13 papers before identifying a systematic set of published research and pre-prints on AutoML. They then conducted eight iterations to develop the taxonomy by reading the literature sources, extracting artifacts that met the definition of the meta characteristic, classifying those items, and grouping them according to an evolving set of artifact properties. The resulting taxonomy is concise, robust, comprehensive, extensible and explanatory. It comprises multiple interrelated phases that leverage statistical and computational techniques for data preparation, analysis, deployment and communication. The taxonomy also includes artifact identification and classification methods for capturing artifacts with their properties and dependencies. To operationalize the taxonomy further, the researchers developed AutoMLTrace - a visual interactive sketch showing both the context and temporality of human-ML/AI collaboration in data work. They demonstrated its utility via a usage scenario with an enterprise software development team. Finally, reflecting on their approach to developing the taxonomy through a broad literature review but assessing its utility primarily through a collaboration with one team; they acknowledge more work is required to assess its generality but remain optimistic about its potential for developing data visualization tools that interrogate sociotechnical relationships in automated data work.
- The field of data work has become complex, requiring a range of skills that are often inaccessible to non-technical experts.
- Automated Machine Learning (AutoML) technology automates certain aspects of data work, such as model selection and data preparation.
- AutoML still requires human intervention to be functional, resulting in a complex and collaborative process that can be difficult to trace.
- Researchers have constructed a taxonomy of data work artifacts that captures both the human and machine-generated processes involved in AutoML.
- The taxonomy was developed through an extensive literature review spanning multiple fields.
- The resulting taxonomy is concise, robust, comprehensive, extensible and explanatory.
- It comprises multiple interrelated phases that leverage statistical and computational techniques for data preparation, analysis, deployment and communication.
- To operationalize the taxonomy further, the researchers developed AutoMLTrace - a visual interactive sketch showing both the context and temporality of human-ML/AI collaboration in data work.
- They demonstrated its utility via a usage scenario with an enterprise software development team.
Data work is a complicated job that needs special skills. Automated Machine Learning (AutoML) technology helps with some parts of data work, but people still need to help for it to work well. Researchers made a list of all the things people and machines do in AutoML called a taxonomy. They did this by reading lots of books and papers about it. The taxonomy has different steps that use math and computers to prepare, analyze, deploy, and communicate data. The researchers also made a picture called AutoMLTrace to show how people and machines work together in data work.
Exploring Automated Machine Learning (AutoML) Through a Taxonomy of Data Work Artifacts
Data work has become increasingly complex, requiring a range of skills that are often inaccessible to non-technical experts. To address this issue, researchers have developed an automated machine learning (AutoML) technology that automates certain aspects of data work such as model selection and data preparation. However, AutoML still requires human intervention to be functional, resulting in a complex and collaborative process that can be difficult to trace. To better understand the sociotechnical relationships involved in AutoML, the research team constructed a taxonomy of data work artifacts that captures both the human and machine-generated processes involved in AutoML.
Developing the Taxonomy
The research team conducted an extensive literature review spanning Machine Learning, Human Computer Interaction, Computer Supported Collaborative Work, Information Visualization and Visual Analytics. From this review they gathered an initial set of 13 papers before identifying a systematic set of published research and pre-prints on AutoML. They then conducted eight iterations to develop the taxonomy by reading these sources for artifacts that met their definition of meta characteristics; classifying those items; and grouping them according to an evolving set of artifact properties.
The resulting taxonomy is concise yet robust with comprehensive coverage across multiple interrelated phases including statistical techniques for data preparation; analysis; deployment; communication; artifact identification/classification methods for capturing artifacts with their properties/dependencies; plus visual interactive sketches showing context/temporality for human-machine collaboration in data work.
Demonstrating Utility Through Usage Scenario
To operationalize the taxonomy further, the researchers developed AutoMLTrace - a visual interactive sketch showing both the context & temporality of human-machine collaboration in data work - which they demonstrated via usage scenario with enterprise software development team. The results were promising but more assessment is needed to assess its generality before it can be used as tool for developing visualization tools interrogating sociotechnical relationships in automated data work.
Conclusion
In conclusion, this paper presents a detailed taxonomy outlining both human & machine generated processes involved in Automated Machine Learning (AutoML). The authors demonstrate its utility through usage scenarios while acknowledging more assessment is needed before it can be used as tool for developing visualization tools interrogating sociotechnical relationships in automated data work
