# Identifying Dwarfs Workloads in Big Data Analytics

**Authors:** Wanling Gao, Chunjie Luo, Jianfeng Zhan, Hainan Ye, Xiwen He, Lei Wang, Yuqing Zhu, Xinhui Tian  
**arXiv:** [1505.06872](https://arxiv.org/abs/1505.06872)  
**Date:** 2015-05-26  
**Summary page:** https://www.summarizepaper.com/en/paper/1505.06872v1/

## Key points

- Benchmarking big data systems is important but challenging due to the wide scope and complexity of big data computing.
- The authors propose using a benchmark suite composed of "big data dwarfs" to represent the diversity of big data analytics workloads.
- Big data dwarfs are abstractions that capture frequently occurring operations in big data computing.
- Each dwarf represents one unit of computation, and big data workloads can be decomposed into one or more dwarfs.
- Using dwarf workloads instead of vast real workloads is more cost-efficient and representative for evaluating big data systems.
- The authors investigate six important application domains in big data analytics: search engine, social network, e-commerce, multimedia, bioinformatics, and astronomy.
- They analyze forty representative algorithms within these domains and identify eight dwarfs workloads in addition to OLAP (Online Analytical Processing).
- The identified dwarfs include linear algebra, sampling, logic operations, transform operations, set operations, graph operations, statistic operations, and sort.
- This research contributes to improving the efficiency and accuracy of benchmarking for assessing big data systems.

## Summary

The paper titled "Identifying Dwarfs Workloads in Big Data Analytics" explores the challenges and importance of benchmarking big data systems. Benchmarking is essential for evaluating the performance of these systems, but the wide scope and complexity of big data computing make it a difficult task. The authors propose using a benchmark suite composed of a minimum set of units of computation called "big data dwarfs" to represent the diversity of big data analytics workloads. Big data dwarfs are abstractions that capture frequently occurring operations in big data computing. Each dwarf represents one unit of computation, and big data workloads can be decomposed into one or more dwarfs. The authors argue that using dwarf workloads instead of vast real workloads is more cost-efficient and representative for evaluating big data systems. To identify the most important or emerging application domains in big data analytics, the authors thoroughly investigate six domains: search engine, social network, e-commerce, multimedia, bioinformatics, and astronomy. They analyze forty representative algorithms within these domains and single out eight dwarfs workloads in big data analytics other than OLAP (Online Analytical Processing). These dwarfs include linear algebra, sampling, logic operations, transform operations, set operations, graph operations, statistic operations and sort. In conclusion, this paper provides valuable insights into constructing a benchmark suite for evaluating big data systems by using a minimum set of units of computation called big data dwarfs. By investigating six important application domains and analyzing representative algorithms within those domains, the authors identify eight dwarfs workloads that can effectively represent diverse big data analytics tasks. This research contributes to improving the efficiency and accuracy of benchmarking for assessing booming big data systems.