# Pattern Discovery and Validation Using Scientific Research Methods

**Authors:** Dirk Riehle, Nikolay Harutyunyan, Ann Barcomb  
**arXiv:** [2107.06065](https://arxiv.org/abs/2107.06065)  
**Date:** 2021-07-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2107.06065v1/

## Key points

- Challenges of pattern discovery and validation in BOM management in software supply chains
- Need for tools to improve BOM management due to complexity of software supply chains
- Suggestion to use machine-readable and standard format for BOMs, such as SPDX
- Evaluation of proposed approach through member checking and case study research
- Member checking involves obtaining feedback from domain experts to confirm accuracy and completeness of identified patterns
- Delphi method used in exploratory study involving multiple rounds of questioning and commenting by panel of experts to define patterns with high confidence
- Member checking increases confidence but does not provide full validation due to potential biases
- Full-fledged case studies conducted using case study research methodology to evaluate effectiveness of handbook method within companies that were not experts
- Overall, member checking and case studies overwhelmingly confirmed usefulness and validity of handbook method for pattern discovery in BOM management
- Future work includes subjecting the method to more rigorous testing, theory building, and evaluation
- Acknowledgments given to individuals who contributed through workshops and extensive suggestions from reviewers
- Suggestion for independent researchers to apply this method in different contexts for further validation beyond what has been presented so far

## Summary

This article discusses the challenges of pattern discovery and validation in the context of managing Bill of Materials (BOMs) in software supply chains. It emphasizes the need for tools to improve BOM management due to the complexity of software supply chains. The article suggests using a machine-readable and standard format for BOMs, such as Software Package Data Exchange (SPDX), which enables the documentation and exchange of data and metadata for open source components. The evaluation of the proposed approach is conducted through member checking and case study research. Member checking involves obtaining feedback from domain experts who confirmed the accuracy and completeness of the identified patterns. In one exploratory study, the Delphi method was used, which involved multiple rounds of questioning and commenting by a panel of experts to define patterns with high confidence. The article acknowledges that member checking increases confidence but does not provide full validation due to potential biases. To address this, several full-fledged case studies were conducted using case study research methodology. These case studies involved observing the use of the handbook method within companies that were not experts in order to evaluate its effectiveness. While some areas for improvement were identified, overall, both member checking and case studies overwhelmingly confirmed the usefulness and validity of the handbook method for pattern discovery in BOM management. The article concludes by highlighting future work that includes subjecting the method to more rigorous testing, theory building, and evaluation. The authors aim to provide a rigorous method for pattern discovery in order to support practitioners in solving problems within their domains. Acknowledgments are given to individuals who contributed to improving this article through workshops and extensive suggestions from reviewers. In terms of further validation, it is suggested that independent researchers apply this method in different contexts to assess its effectiveness beyond what has been presented so far. The success observed in exploratory case studies indicates that the method worked as intended; however, more research is needed to confirm its transferability beyond these specific studies. Overall, this article provides insights into the challenges of pattern discovery and validation in BOM management and presents a specific approach -the handbook method- which has been evaluated through exploratory studies and case studies. The findings suggest that using scientific research methods can improve certainty and quality when proposing patterns within this domain.