# Exploring Explainable AI in the Financial Sector: Perspectives of Banks and Supervisory Authorities

**Authors:** Ouren Kuiper, Martin van den Berg, Joost van der Burgt, Stefan Leijnen  
**arXiv:** [2111.02244](https://arxiv.org/abs/2111.02244)  
**Date:** 2021-11-03  
**Summary page:** https://www.summarizepaper.com/en/paper/2111.02244v1/

## Key points

- The study investigated the perspectives of supervisory authorities and regulated entities on the application of explainable artificial intelligence (xAI) in the financial sector.
- The research focused on three use cases: consumer credit, credit risk, and anti-money laundering.
- Semi-structured interviews were conducted at three banks and two supervisory authorities in the Netherlands to gather data.
- Regarding consumer credit, one bank used an AI system based on logistic regression to assess mortgages with traffic-light colors for middle office employees. Another bank supplemented its traditional loan approval system for consumer credit with an AI system that uses transactional data resulting in fewer defaults on loans.
- One supervisory authority monitors whether lenders comply with lending standards limiting loan amounts depending on their financial situation. Regardless of what an AI system indicates, banks must conform to these standards as they ensure that lending consumers are protected; however, within these standards banks might use AI systems to find cases their traditional systems would not give credit but determine as profitable for them which might not always be good for consumers.
- Overall findings indicate that there is a disparity between supervisory authorities and banks regarding desired scope of explainability of AI systems for investigated use cases; thus clear differentiation between technical AI (model) explainability requirements and explainability requirements of broader AI system in relation to applicable laws and regulations could benefit financial sector.

## Summary

This study aimed to investigate the perspectives of supervisory authorities and regulated entities on the application of explainable artificial intelligence (xAI) in the financial sector. The research focused on three use cases: consumer credit, credit risk, and anti-money laundering. Semi-structured interviews were conducted at three banks and two supervisory authorities in the Netherlands to gather data. The data analysis involved creating a list of main findings and conclusions per interview which were verified and supplemented by the interviewees. An overview of the main conclusions was developed by analyzing all interview reports. These conclusions were discussed in a plenary session with participants from both supervisory authorities and banks to refine the final conclusions. Regarding consumer credit, one bank used an AI system based on logistic regression to assess mortgages with traffic-light colors for middle office employees. The primary users of this AI system were not given detailed insight into its functioning and results to prevent potential gaming of the system; however, due to its relative interpretability, explainability to other stakeholders was not considered challenging beyond previous systems. Another bank supplemented its traditional loan approval system for consumer credit with an AI system that uses transactional data resulting in fewer defaults on loans. Model developers are essential stakeholders regarding explainability as it would be possible from a technological point of view to explain the model to customers; however, this requires understanding which type of narratives would be comprehensible by different consumer groups. One supervisory authority monitors whether lenders comply with lending standards limiting loan amounts depending on their financial situation. Regardless of what an AI system indicates, banks must conform to these standards as they ensure that lending consumers are protected; however, within these standards banks might use AI systems to find cases their traditional systems would not give credit but determine as profitable for them which might not always be good for consumers. Overall findings indicate that there is a disparity between supervisory authorities and banks regarding desired scope of explainability of AI systems for investigated use cases; thus clear differentiation between technical AI (model) explainability requirements and explainability requirements of broader AI system in relation to applicable laws and regulations could benefit financial sector.