Résultats de recherche pour « Counterfactual explanations »

Artificial intelligence has become increasingly important in quantitative portfolio management, enabling deep learning models to capture complex nonlinear relationships between macroeconomic conditions and asset allocation decisions. Despite their predictive capability, these models often operate as black boxes, limiting transparency and reducing practitioner trust in high-stakes financial decision-making. Counterfactual explanations provide an attractive mechanism for explaining model behavior by identifying minimal input modifications that alter model predictions. However, existing counterfactual generation methods typically optimize over unrestricted feature spaces, frequently producing economically implausible market scenarios that lack practical value for institutional portfolio management. This paper proposes a financially constrained counterfactual explanation framework for AI-driven portfolio allocation. Counterfactual generation is formulated as a constrained optimization problem over economically actionable macro-financial variables while enforcing historical feasibility throughout the optimization process. Unlike unconstrained optimization, the proposed framework restricts interventions to the Federal Funds Rate, Yield Curve Slope, and market volatility, generating economically meaningful explanations that remain consistent with historically observed market behavior. Experiments conducted on 934 out-of-sample historical market observations demonstrate that the proposed framework substantially reduces intervention dimensionality relative to unconstrained counterfactual generation. Compared with an unconstrained gradientbased optimizer, the proposed approach reduces the average number of modified variables from 9.73 to 3.00, while the counterfactual success rate decreases from 57.17% to 19.27%. An ablation study further demonstrates that jointly optimizing multiple macro-financial variables substantially improves feasible recourse relative to singlevariable interventions. The results highlight the trade-off between optimization flexibility and financial realism and demonstrate the importance of incorporating domain constraints into counterfactual explanations for quantitative portfolio management.

Decision support systems based on machine learning models should be able to help users identify opportunities and threats. Popular model-agnostic explanation models can identify factors that support various predictions, answering questions like “What factors affect sales?” or “Why did sales decline?”, but do not highlight what a person should or could do to get a more desirable outcome. Counterfactual explanation approaches address intervention, and some even consider feasibility, but there has been no evaluation of their suitability for real-time applications, such as question answering. Here we address this gap by introducing a novel model-agnostic method that provides specific, feasible changes that would impact the outcomes of a complex black-box AI model for a given instance and assess its real-world utility by measuring its real-time performance and ability to find achievable changes. The method uses the instance of concern to generate high-precision explanations and then applies a secondary method to find minimally contrastive and maximally probable high-precision counterfactual explanations, while limiting contrasted features to changes that are achievable. We demonstrate that using this method, it is possible to find such explanations quickly enough for use in real-time systems. High-precision achievable minimally contrastive explanations would be useful in applications where people seek remedial actions or question how effective a proposed remedy is likely to be.

Abstract This paper studies how counterfactual explanations can be used to assess the fairness of a model. Using machine learning for high-stakes decisions is a threat to fairness as these models can amplify bias present in the dataset, and there is no consensus on a universal metric to detect this. The appropriate metric and method to tackle the bias in a dataset will be case-dependent, and it requires insight into the nature of the bias first. We aim to provide this insight by integrating explainable AI (XAI) research with the fairness domain. More specifically, apart from being able to use (Predictive) Counterfactual Explanations to detect explicit bias when the model is directly using the sensitive attribute, we show that it can also be used to detect implicit bias when the model does not use the sensitive attribute directly but does use other correlated attributes leading to a substantial disadvantage for a protected group. We call this metric $PreCoF$, or Predictive Counterfactual Fairness. Our experimental results show that our metric succeeds in detecting occurrences of implicit bias in the model by assessing which attributes are more present in the explanations of the protected group compared to the unprotected group. These results could help policymakers decide on whether this discrimination is justified or not.

Abstract Artificial intelligence (AI) based models can detect cardiac diseases from 12-lead electrocardiograms (ECGs). However clinical adoption of AI remains limited, partly by a lack of interpretability. Existing explainable AI (XAI) techniques identify which regions contributed to a prediction but fail to specify what modification would have changed that outcome, an aspect that is central to the kind of differential diagnosis reasoning clinicians routinely carry out. This paper introduce InterGRACE, a framework that generates counterfactual ECGs by guiding adversarial perturbations with Integrated Gradients (IG) attributions, concentrating changes on the features a classifier itself deems most decision-relevant while penalizing large or spatially rough deviations from the original signal. Here InterGRACE is applied to a myocardial infarction (MI) versus non-MI classification task on the public PTB-XL dataset, using two independently trained classifiers (Model 1 and Model 2) to examine whether the resulting counterfactuals reflect shared and clinically meaningful features rather than artifacts of a single model. IG-guided counterfactuals reliably flipped Model 1's own predictions with a success rate of 67.4%, and a subset of these counterfactuals also flips Model 2's predictions with a success rate of 58.8%, indicating partial cross-model transferability. Lead-wise inspection shows that the perturbations concentrate in the ST segment and T wave, rather than acting uniformly across the waveform. This paper proposes that attribution-guided counterfactual generation can provide more transparent and clinically meaningful explanations for ECG-based deep learning models, while also emphasizing the necessity of physiologically constrained generative methods before deploying such tools in clinical decision support.

An outstanding challenge in deep learning is its lack of interpretability. The inability of both designers and users of neural networks to explain why a prediction is made is a major drawback. This not only dissuades chemists from using deep learning predictions, but also has led to neural networks learning spurious correlations that are difficult to notice. Counterfactuals are a category of prediction explanations that provide a rationale behind the prediction value. Counterfactuals have satisfying properties like being in the same space as the features and require no deep learning expertise to understand. Counterfactuals are generated via optimization, which makes them complex to use when the features are chemical structures. Here we present an algorithm built on the "Superfast Traversal, Optimization, Novelty, Exploration and Discovery" method (Nigam et. al, 2021) that is efficient and model-agnostic for generating counterfactuals. We show this method is applicable in random forest models, sequence models, and graph neural networks in both classification and regression. Our method requires no additional training, data, or model gradients.

An outstanding challenge in deep learning in chemistry is its lack of interpretability. The inability of explaining why a neural network makes a prediction is a major barrier to deployment of AI models. This not only dissuades chemists from using deep learning predictions, but also has led to neural networks learning spurious correlations that are difficult to notice. Counterfactuals are a category of explanations that provide a rationale behind a model prediction with satisfying properties like providing chemical structure insights. Yet, counterfactuals are have been previously limited to specific model architectures or required reinforcement learning as a separate process. In this work, we show a universal model-agnostic approach that can explain any black-box model prediction. We demonstrate this method on random forest models, sequence models, and graph neural networks in both classification and regression.

Graph Neural Networks (GNNs) achieve strong predictive performance on graph-structured data across domains such as chemistry, biology, and network analysis, yet they provide no intrinsic explanation of their predictions. This limits their adoption in high-stakes and safety-critical settings.Counterfactual explanations address this by revealing the minimal structural modifications that would change a model's prediction. On graphs, however, such a modification is hard to produce.The search space is discrete and combinatorial, and a valid answer must respect categorical node and edge types together with domain rules such as chemical valency in the case of molecular graphs. Existing explainers give up one of two things. Either edits are not held on the data manifold, or the search does not span the full edit space. We propose Graph Diffusion Counterfactual Explanation via Inversion (GDCE-I), which gives up neither. Adiscrete denoising diffusion model with a novel discrete inversion scheme enables distribution-aware edits leveraging the whole domain edit space. We further address the incomplete and inconsistent evaluation of graph counterfactuals by deriving a framework of explanation desiderata and applying it to every method under one shared protocol. Across four benchmarks, GDCE-I outperforms related work by a large margin on the defined framework. For the molecular domain, we further qualitatively show that GDCE-I attains interpretable in-distribution solutions.