Please Stop Explaining Black Box Models for High Stakes Decisions

AI-generated keywords: Interpretability Accuracy Deep Learning Preprocessing Structured Data

AI-generated Key Points

  • Black box models are increasingly used for high-stakes decision-making in society.
  • Trying to explain these models after the fact can perpetuate bad practices and potentially cause harm to society.
  • Recent work on explainable ML contains critical misconceptions that can negatively impact the widespread use of machine learning models in society.
  • There is not necessarily a tradeoff between accuracy and interpretability, particularly when data naturally have a good representation.
  • Deep learning models tend to be black boxes because they are highly recursive, but complicated black boxes are not necessary for top predictive performance.
  • There tends to be little difference in performance between more complex classifiers and much simpler ones after preprocessing in data science problems where structured data with meaningful features are constructed as part of the process.
  • The way forward is to design inherently interpretable models rather than relying on explanations after the fact.
  • This will allow policy-makers to demand more effort in ensuring safety and trust in machine learning models for high-stakes decisions while avoiding unnecessary tradeoffs between accuracy and interpretability.
Also access our AI generated: Comprehensive summary, Lay summary, Blog-like article; or ask questions about this paper to our AI assistant.

Authors: Cynthia Rudin

NIPS 2018 Workshop on Critiquing and Correcting Trends in Machine Learning, Longer version. Expands also on NSF Statistics at a Crossroads Webinar
License: CC BY-SA 4.0

Abstract: There are black box models now being used for high stakes decision-making throughout society. The practice of trying to explain black box models, rather than creating models that are interpretable in the first place, is likely to perpetuate bad practices and can potentially cause catastrophic harm to society. There is a way forward -- it is to design models that are inherently interpretable.

Submitted to arXiv on 26 Nov. 2018

Ask questions about this paper to our AI assistant

You can also chat with multiple papers at once here.

AI assistant instructions?

Results of the summarizing process for the arXiv paper: 1811.10154v1

The use of black box models for high-stakes decision-making in society has become increasingly prevalent. However, the practice of trying to explain these models after the fact rather than creating inherently interpretable models can perpetuate bad practices and potentially cause catastrophic harm to society. While there is a vast amount of literature on interpretability, much recent work on explainable ML contains critical misconceptions that can have a lasting negative impact on the widespread use of machine learning models in society. One such misconception is that there is necessarily a tradeoff between accuracy and interpretability, which is often not true, particularly when data naturally have a good representation. Deep learning models tend to be black boxes because they are highly recursive, but this does not mean that complicated black boxes are necessary for top predictive performance. In fact, there tends to be little difference in performance between more complex classifiers and much simpler ones after preprocessing in data science problems where structured data with meaningful features are constructed as part of the process. The way forward is to design models that are inherently interpretable rather than relying on explanations after the fact. This will allow policy-makers to demand more effort in ensuring safety and trust in machine learning models for high-stakes decisions while avoiding any unnecessary tradeoffs between accuracy and interpretability.
Created on 06 Apr. 2023

Assess the quality of the AI-generated content by voting

Score: 1

Why do we need votes?

Votes are used to determine whether we need to re-run our summarizing tools. If the count reaches -10, our tools can be restarted.

Similar papers summarized with our AI tools

Navigate through even more similar papers through a

tree representation

Look for similar papers (in beta version)

By clicking on the button above, our algorithm will scan all papers in our database to find the closest based on the contents of the full papers and not just on metadata. Please note that it only works for papers that we have generated summaries for and you can rerun it from time to time to get a more accurate result while our database grows.

Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.