# BRATsynthetic: Text De-identification using a Markov Chain Replacement Strategy for Surrogate Personal Identifying Information

**Authors:** John D. Osborne, Tobias O'Leary, Akhil Nadimpalli, Salma M. Aly., Richard E. Kennedy  
**arXiv:** [2210.16125](https://arxiv.org/abs/2210.16125)  
**Date:** 2022-10-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.16125v1/

## Key points

- Objective: Implement and evaluate different personal health identifying information (PHI) substitution strategies to quantify privacy preserving benefits.
- Privacy considerations and legal mandates limit clinical text availability for machine learning model training.
- Synthetic generation of clinical notes has potential, but more research is needed.
- De-identification is an alternative approach, but existing software may not effectively replace identified PHI with surrogate text.
- Implement and assess three "Hiding in Plain Sight" (HIPS) strategies for PHI replacement: Consistent, Random, and Markov model-based.
- Evaluate privacy preserving benefits using false negative error rates (FNER).
- Markov chain strategy significantly reduces PHI leakage compared to Consistent strategy on diverse set of notes from University of Alabama at Birmingham (UAB).
- Markov chain strategy outperforms Consistent and Random strategies on MIMIC corpus of discharge summaries and synthetic clinical PHI distributions.
- Document-level PHI leakage reduced from 27.1% to 0.1% at 0.1% FNER, and from 94.2% to 57.7% at 5% FNER using Markov chain strategy.
- Markov chain surrogate generation strategy substantially reduces risk of inadvertent PHI release across different assumed FNERs.
- Implementation called "BRATsynthetic" released on Github for clinical informatics community use.
- Conclusion: Markov chain replacement strategy allows for release of larger de-identified corpora at same risk level compared to consistent HIPS strategy.

## Summary

Objective: The objective of this study is to implement and evaluate different personal health identifying information (PHI) substitution strategies in order to quantify their privacy preserving benefits. Background and Significance: Privacy considerations and legal mandates create a scarcity of clinical text for machine learning model training and evaluation. Synthetic generation of clinical notes has shown promise but more research is needed. An alternative approach is to use real notes and remove or replace only personal identifying information (PII), including PHI, through a process called de-identification. However, existing de-identification software may not replace identified PHI with surrogate text, making it difficult to train machine learning algorithms effectively. This challenge may lead to PHI data leaks if the de-identification process is skipped. Methods: In this study, we implement and assess three different "Hiding in Plain Sight" (HIPS) strategies for PHI replacement: a standard Consistent replacement strategy, a Random replacement strategy, and a novel Markov model-based strategy. We evaluate the privacy preserving benefits of these strategies on synthetic PHI distributions and real clinical corpora from two different institutions using false negative error rates (FNER). Results: Using FNER ranging from 0.1% to 5%, we observe that the Markov chain strategy significantly reduces PHI leakage compared to the Consistent strategy on a diverse set of notes from the University of Alabama at Birmingham (UAB). The Markov chain strategy also outperforms the Consistent and Random strategies on a MIMIC corpus of discharge summaries as well as on various synthetic clinical PHI distributions. Specifically, at 0.1% FNER document-level PHI leakage could be reduced from 27.1% to 0.1%, while at 5% FNER it could be reduced from 94.2% to 57.7%. Discussion: Our findings demonstrate that the Markov chain surrogate generation strategy substantially reduces the risk of inadvertent PHI release across different assumed PHI FNERs. We also release our implementation called "BRATsynthetic" on Github for the clinical informatics community to use. Conclusion: The Markov chain replacement strategy allows for the release of larger de-identified corpora at the same risk level compared to using a consistent HIPS strategy.