# The Consequences of the Framing of Machine Learning Risk Prediction Models: Evaluation of Sepsis in General Wards

**Authors:** Simon Meyer Lauritsen, Bo Thiesson, Marianne Johansson Jørgensen, Anders Hammerich Riis, Ulrick Skipper Espelund, Jesper Bo Weile, Jeppe Lange  
**arXiv:** [2101.10790](https://arxiv.org/abs/2101.10790)  
**Date:** 2021-01-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2101.10790v1/

## Key points

- Study aims to evaluate consequences of framing machine learning risk prediction models in sepsis evaluation
- Researchers analyze different framing approaches and their impact on model performance using healthcare data from 221,283 citizens in Denmark
- Four models have similar population-level performance with AUC ranging from 0.73 to 0.82
- Significant variation in mean average precision among different framing approaches (0.007 to 0.385)
- On-clinical-demand framing has lowest percentage of missing values for vital sign parameters and better ability to learn temporal dependencies compared to other models
- Different framings can lead to opposing interpretations of certain variables when predicting sepsis (e.g., SpO2)
- Importance of considering and reporting on model framing when developing clinical risk prediction models beyond sepsis prediction alone
- Proper problem framing crucial for successful development and clinical implementation of AI technology in healthcare settings

## Summary

The study aims to evaluate the consequences of framing machine learning risk prediction models in the context of sepsis evaluation in general wards. The researchers analyze how different framing approaches affect model performance and learning using structured secondary healthcare data from 221,283 citizens aged 18 or older in four Danish municipalities. The results show that the four models have similar population-level performance, with a mean area under the receiver operating characteristic curve ranging from 0.73 to 0.82. However, there is significant variation in mean average precision, ranging from 0.007 to 0.385, among the different framing approaches. Additionally, the percentage of missing values differs between framing approaches. Among the various framing approaches, the on-clinical-demand framing stands out as it has the lowest percentage of missing values for vital sign parameters and demonstrates better ability to learn temporal dependencies compared to other models. Furthermore, this research highlights how different framings can lead to opposing interpretations of certain variables (e.g., SpO2) when predicting sepsis and underscores the importance of considering and reporting on model framing when developing clinical risk prediction models beyond sepsis prediction alone. The authors emphasize that proper problem framing is crucial for successful development and clinical implementation of AI technology in healthcare settings and caution clinicians and AI developers about the profound consequences of their choices when selecting a particular frame for their model.