# Towards Safe Propofol Dosing during General Anesthesia Using Deep Offline Reinforcement Learning

**Authors:** Xiuding Cai, Jiao Chen, Yaoyao Zhu, Beiming Wang, Yu Yao  
**arXiv:** [2303.10180](https://arxiv.org/abs/2303.10180)  
**Date:** 2023-03-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.10180v1/

## Key points

- Automated anesthesia has the potential to revolutionize anesthetic administration
- Policy Constraint Q-Learning (PCQL) is a data-driven reinforcement learning algorithm that can learn anesthesia strategies on real clinical datasets
- PCQL incorporates Conservative Q-Learning and adds a policy constraint term to keep the policy distribution of the agent consistent with that of the anesthesiologist, ensuring safer decisions made by the agent in anesthesia scenarios
- The effectiveness of PCQL was validated through extensive experiments on a real clinical anesthesia dataset
- PCQL is predicted to achieve higher gains than baseline approaches while maintaining good agreement with reference doses given by anesthesiologists, using less total dose, and being more responsive to patients' vital signs
- SHapley Additive exPlanations (SHAP) was used to analyze contributing components for transparency and interpretability of model predictions
- Overall, PCQL represents a promising step towards safe propofol dosing during general anesthesia using deep offline reinforcement learning on real clinical datasets

## Summary

The field of automated anesthesia has the potential to revolutionize anesthetic administration by enabling more precise and personalized care while freeing up anesthesiologists from repetitive tasks. In response to this challenge, a team of researchers proposed Policy Constraint Q-Learning (PCQL), a data-driven reinforcement learning algorithm that can learn anesthesia strategies on real clinical datasets. The PCQL approach incorporates Conservative Q-Learning to alleviate the problem of Q function overestimation in an offline context and adds a policy constraint term to keep the policy distribution of the agent consistent with that of the anesthesiologist. This ensures safer decisions made by the agent in anesthesia scenarios. The effectiveness of PCQL was validated through extensive experiments on a real clinical anesthesia dataset. The results showed that PCQL is predicted to achieve higher gains than baseline approaches while maintaining good agreement with reference doses given by anesthesiologists, using less total dose, and being more responsive to patients' vital signs. Additionally, confidence intervals were investigated which were able to cover most of the clinical decisions made by anesthesiologists. To increase transparency and interpretability of the model predictions, SHapley Additive exPlanations (SHAP) was used to analyze contributing components. This method provides insight into how each input feature contributes to model predictions and can help identify areas for improvement or further investigation. Overall, PCQL represents a promising step towards safe propofol dosing during general anesthesia using deep offline reinforcement learning on real clinical datasets. By reducing reliance on manual intervention and improving precision and personalization in anesthesia administration, this technology could have significant implications for patient outcomes in surgical care.