AI summary · arXiv: 2207.14160v2 · cs.SE
Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmark
The essentials
- Explanation as a legal right has gained attention in recent years
- Importance of going beyond accuracy metrics to improve AI models by debugging learned patterns and demystifying AI behavior
- Challenges arising from the widespread use of explainable AI (XAI)
- Increase in published XAI algorithms, making it difficult for practitioners to choose the most suitable tool
- Potential misuse and misinterpretation of XAI algorithms by data scientists
- Proposal of a benchmark led by Mohamed Karim Belaid, Eyke Hüllermeier, Maximilian Rabus, and Ralf Krestel to address issues and ensure proper comparison and utilization of XAI
- Benchmark aims to unify exclusive functional testing methods for XAI algorithms
- Development of a selection protocol to identify non-redundant functional tests tailored to meet specific end-user requirements for explaining a model
- Hierarchical scoring system with three levels targeting different end-user groups: researchers, practitioners, and laymen in XAI
- Grouping tests into five categories: fidelity, fragility, stability, simplicity, and stress tests
- Inclusion of an aggregated comprehensibility score simplifying interpreting algorithm outputs into one easy-to-compare value
- Interactive user interface aiming to recommend suitable solutions for various machine learning tasks along with their current limitations
Key points written by AI: they may contain errors, check them against the paper.
Ask questions about this paper to our AI assistant
How to chat with this paper? Illustrated guide
You can also chat with multiple papers at once here.
Keep this conversation — a free account saves your questions and answers, your collections of papers and your alerts.
Create a free accountTools for this paper
Advanced tools Premium
For researchers who follow a field closely: citation alerts, journal-club slides, version comparison and Zotero export. Discover Premium
Papers around this one
See them on a mapRanked by how close their title and abstract are to this paper’s. Select some to read them together.
Closest papers
-
Model Share AI: An Integrated Toolkit for Collaborative Machine Learning Model Development, Provenance Tracking, and Deployment in PythonHeinrich Peters, Michael Parrott · 2023 · Software Engineering · AI summary available
Abstract
Machine learning (ML) has the potential to revolutionize a wide range of research areas and industries, but many ML projects never progress past the proof-of-concept stage. To address this issue, we introduce Model Share AI (AIMS), an easy-to-use MLOps platform designed to streamline collaborative model development, model provenance tracking, and model deployment, as well as a host of other functions aiming to maximize the real-world impact of ML research. AIMS features collaborative project spaces and a standardized model evaluation process that ranks model submissions based on their performance on unseen evaluation data, enabling collaborative model development and crowd-sourcing. Model pe…
-
An AI System Evaluation Framework for Advancing AI Safety: Terminology, Taxonomy, Lifecycle MappingBoming Xia, Qinghua Lu, Liming Zhu et al. · 2024 · Software Engineering · AI summary available
Abstract
The advent of advanced AI underscores the urgent need for comprehensive safety evaluations, necessitating collaboration across communities (i.e., AI, software engineering, and governance). However, divergent practices and terminologies across these communities, combined with the complexity of AI systems-of which models are only a part-and environmental affordances (e.g., access to tools), obstruct effective communication and comprehensive evaluation. This paper proposes a framework for AI system evaluation comprising three components: 1) harmonised terminology to facilitate communication across communities involved in AI safety evaluation; 2) a taxonomy identifying essential elements for AI …
-
Evaluating and Explaining Large Language Models for Code Using Syntactic StructuresDavid N Palacio, Alejandro Velasco, Daniel Rodriguez-Cardenas et al. · 2023 · Software Engineering · AI summary available
Abstract
Large Language Models (LLMs) for code are a family of high-parameter, transformer-based neural networks pre-trained on massive datasets of both natural and programming languages. These models are rapidly being employed in commercial AI-based developer tools, such as GitHub CoPilot. However, measuring and explaining their effectiveness on programming tasks is a challenging proposition, given their size and complexity. The methods for evaluating and explaining LLMs for code are inextricably linked. That is, in order to explain a model's predictions, they must be reliably mapped to fine-grained, understandable concepts. Once this mapping is achieved, new methods for detailed model evaluations a…
-
Empirical Study on the Software Engineering Practices in Open Source ML Package RepositoriesMinke Xiu, Ellis E. Eghan, Zhen Ming et al. · 2020 · Software Engineering · AI summary available
Abstract
Recent advances in Artificial Intelligence (AI), especially in Machine Learning (ML), have introduced various practical applications (e.g., virtual personal assistants and autonomous cars) that enhance the experience of everyday users. However, modern ML technologies like Deep Learning require considerable technical expertise and resources to develop, train and deploy such models, making effective reuse of the ML models a necessity. Such discovery and reuse by practitioners and researchers are being addressed by public ML package repositories, which bundle up pre-trained models into packages for publication. Since such repositories are a recent phenomenon, there is no empirical data on their…
-
The AI Agent IndexStephen Casper, Luke Bailey, Rosco Hunter et al. · 2025 · Software Engineering · AI summary available
Abstract
Leading AI developers and startups are increasingly deploying agentic AI systems that can plan and execute complex tasks with limited human involvement. However, there is currently no structured framework for documenting the technical components, intended uses, and safety features of agentic systems. To fill this gap, we introduce the AI Agent Index, the first public database to document information about currently deployed agentic AI systems. For each system that meets the criteria for inclusion in the index, we document the system's components (e.g., base model, reasoning implementation, tool use), application domains (e.g., computer use, software engineering), and risk management practice…
-
Can AI Serve as a Substitute for Human Subjects in Software Engineering Research?Marco A. Gerosa, Bianca Trinkenreich, Igor Steinmacher et al. · 2023 · Software Engineering · AI summary available
Abstract
Research within sociotechnical domains, such as Software Engineering, fundamentally requires a thorough consideration of the human perspective. However, traditional qualitative data collection methods suffer from challenges related to scale, labor intensity, and the increasing difficulty of participant recruitment. This vision paper proposes a novel approach to qualitative data collection in software engineering research by harnessing the capabilities of artificial intelligence (AI), especially large language models (LLMs) like ChatGPT. We explore the potential of AI-generated synthetic text as an alternative source of qualitative data, by discussing how LLMs can replicate human responses an…
-
BinaryAI: Binary Software Composition Analysis via Intelligent Binary Source Code MatchingLing Jiang, Junwen An, Huihui Huang et al. · 2024 · Software Engineering · AI summary available
Abstract
While third-party libraries are extensively reused to enhance productivity during software development, they can also introduce potential security risks such as vulnerability propagation. Software composition analysis, proposed to identify reused TPLs for reducing such risks, has become an essential procedure within modern DevSecOps. As one of the mainstream SCA techniques, binary-to-source SCA identifies the third-party source projects contained in binary files via binary source code matching, which is a major challenge in reverse engineering since binary and source code exhibit substantial disparities after compilation. The existing binary-to-source SCA techniques leverage basic syntactic …
-
GitHub Copilot AI pair programmer: Asset or Liability?Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam et al. · 2022 · Software Engineering · AI summary available
Abstract
Automatic program synthesis is a long-lasting dream in software engineering. Recently, a promising Deep Learning (DL) based solution, called Copilot, has been proposed by Open AI and Microsoft as an industrial product. Although some studies evaluate the correctness of Copilot solutions and report its issues, more empirical evaluations are necessary to understand how developers can benefit from it effectively. In this paper, we study the capabilities of Copilot in two different programming tasks: (1) generating (and reproducing) correct and efficient solutions for fundamental algorithmic problems, and (2) comparing Copilot's proposed solutions with those of human programmers on a set of progr…
Go further
How are these papers found?
We compare the title and abstract of this paper with those of every paper summarized on this site, and with the papers Semantic Scholar recommends, those it cites and those citing it (for a much-cited paper, its most recent citations and, from OpenAlex, the most cited ones). Among the prior work and the follow-ups, the papers that use each other in depth, a method it uses and the much-cited follow-ups come a little higher. The levels (very close, close, related) come from the similarity of the texts, not from a judgment of quality; * marks a paper without an abstract, compared by its title. The sentence under a paper is written by an AI from the two abstracts: check it in the papers. A paper marked not relevant by three readers leaves the list.
For now only among the papers summarized on this site: search again to add the prior work, the follow-ups and Semantic Scholar’s recommendations.
The previous summary was created more than a year ago and can be re-run (if necessary) by clicking on the Run button below.
Disclaimer: The AI-based summarization tool and virtual assistant provided on this website may not always provide accurate and complete summaries or responses. We encourage you to carefully review and evaluate the generated content to ensure its quality and relevance to your needs.