CS Paper Radar

Interpretable and Fair Generalized Additive Neural Networks via Multi-objective Learning

Ziming Wang, Changwu Huang, Ke Tang, Yew-Soon Ong, Xin Yao

cs.LG ·submitted 2026-09-05 07:29 UTC · v1
published

Abstract

Interpretability and fairness are two of the most emphasized dimensions in trustworthy artificial intelligence (AI). Various explainable AI methods have been introduced to improve interpretability. This paper focuses on neural network (NN)-based generalized additive models (GAMs), a class of self-interpretable models. While most existing research has prioritized improving the accuracy of NN-based GAMs, their interpretability remains largely underexplored. To address this gap, this paper introduces explicit quantitative metrics for evaluating the interpretability of NN-based GAMs, empirically examines their effectiveness, and explores strategies for improving interpretability within these models. In addition, the simultaneous and explicit optimization of both interpretability and fairness, along with their trade-offs and the underlying reasons, remains underexplored. To address this, we propose a multi-objective neural basis model (MONBM) framework based on multi-objective evolutionary learning to consider accuracy, interpretability, and fairness simultaneously. A partial retraining strategy is further developed to facilitate the practical application of evolutionary multi-objective optimization to deep model architectures. Based on MONBM, this paper reveals the complex relationships between these dimensions and the reasons behind these intricate relationships. This analysis demonstrates how multi-objective optimization can be combined with self-interpretable models to reveal relationships among trustworthiness objectives. In addition, MONBM obtains a set of models with different trade-offs between dimensions, and the competitiveness of the approach is validated by comparing it with state-of-the-art methods.

Comment
Published in Neural Networks
Journal ref
Neural Networks (2026), Article 109520
DOI
10.1016/j.neunet.2026.109520

Closest papers

Nearest neighbours by meaning across the whole index.

ONAM: Fitting Orthogonal Neural Additive Models for Post-Hoc Interpretability of Black-Box Predictions

cs.LG stat.MLstat.ME ·2026-11-16 ·David Köhler, Matthias Schmid
similarity 0.59
Abstract

Modern machine learning models, such as artificial neural networks, achieve high predictive performance through increasingly complex black-box architectures, limiting their use in settings where transparency, interpretability, and fairness are required. Here we present the R package ONAM, which provides a novel and practical implementation of functional decomposition, a technique to facilitate the interpretability of black-box models through structured effect representations. ONAM implements an algorithmic framework for the post-hoc decomposition of predictions into components associated with different input features. The package is designed for usability and efficiency, including functionalities for visualization as well as quantitative measures to assess interpretability. As a model-agnostic method, it can be applied to a wide range of prediction models. We describe the mathematical foundations, algorithmic design, and software architecture of ONAM. A central result of the proposed methodology is that the resulting decomposition satisfies the condition of hierarchical orthogonality, thereby guaranteeing that all components are identifiable and uniquely defined.

0.25

A Synthetic Ground-Truth Framework for the Evaluation of Explainable AI Methods

cs.AI ·2026-09-24 ·Miquel Miró-Nicolau, Francesco Spinnato, Riccardo Guidotti
similarity 0.59
Abstract

Evaluating explainable Artificial Intelligence (XAI) methods is a challenging task due to the lack of reliable evaluation procedures and, in particular, the absence of ground truth explanations. In the literature, existing evaluation approaches typically assess explanations by measuring their fidelity with respect to the predictions of a black-box model. However, such evaluation strategies only quantify the degree to which an explanation reproduces the model's output, without ensuring that the explanation correctly reflects the underlying decision process. As a consequence, different explanations may achieve similar fidelity scores while providing inconsistent or misleading interpretations of the model behavior. In this paper, we propose a framework for the evaluation of XAI methods based on synthetic ground truth. The proposed approach relies on controlled interventions to generate synthetic datasets in which the importance of input components can be determined by design. This enables the construction of ground truth explanations that are directly aligned with the behavior of the model under analysis. The framework is instantiated across three data domains, namely binary images, tabular data, and time series, allowing a comprehensive assessment of explanation methods in heterogeneous settings. Experimental results obtained by evaluating nine widely used XAI methods show significant limitations in current techniques and highlight the importance of synthetic, intervention-based benchmarks for a reliable assessment of explanation quality.

0.14

How a Model's Architecture and Performance Influences Explainability: A Study of Grad-CAM and CRAFT in CNN-Based Image Classification

cs.CV cs.AI ·2026-11-09 ·Kilian Göller, Haadia Amjad, Steffen Seitz, Carsten Knoll, Ronald Tetzlaff
top venuehot direction similarity 0.58
Abstract · Accepted at the 4th World Conference on eXplainable Artificial Intelligence (xAI 2026), Fortaleza, Brazil, July 1-3, 2026. To appear in Spri…

Deep learning models, particularly convolutional neural networks (CNNs), are widely adopted in computer vision, yet their opaque decision-making raises concerns in high-risk applications like healthcare and autonomous systems, where trust and accountability are critical. Therefore, robust explainable AI (XAI) methods are desirable to provide transparent insights into model decisions. To ensure these methods are reliable and effective across diverse scenarios, it is important to understand how model performance and architectural complexity influence explanation quality. Our study takes an initial step toward systematically investigating the impact of these two factors on key XAI evaluation metrics. We constructed a pipeline using 10 subsets of ImageNet-1K, each containing 80 unique classes, on each of which we trained three CNN architectures of increasing structural complexity: VGG16, ResNet50, and ConvNeXt-Tiny. For each individual model, we analyzed 10 training checkpoints representing increasing levels of model accuracy using two XAI methods: Grad-CAM (feature-attribution-based) and CRAFT (concept-based). We assessed these methods across four evaluation categories: Faithfulness, Correctness, Sensitivity, and Simplicity. Our findings reveal that as training progresses, the complex architectures (ResNet50 and ConvNeXt-Tiny) exhibit improved Faithfulness and lower Sensitivity, indicating more robust explanations. In contrast, VGG16's simpler architecture yields simpler but less faithful and correct explanations.

0.39

XAI-Arena: Can LLMs Assess the Quality of XAI Explanations?

cs.AI ·2026-09-08 ·Yanfei Hu Fleischhauer, Alona Zharova, Nadja Klein, Stefan Feuerriegel
benchmark similarity 0.58
Abstract

Evaluating the quality of explanations produced by explainable AI (XAI) methods remains challenging because existing approaches often rely on subjective human judgment, limiting reproducibility, scalability, and comparability between studies. We examine whether LLMs can serve as a reproducible and scalable mechanism to make comparative assessments of the quality of XAI explanations. We introduce XAI-Arena, an LLM-as-a-judge framework for scalable, reproducible, multidimensional, and stakeholder-sensitive evaluation of XAI explanation quality. XAI-Arena then allows us to compare XAI explanations along various dimensions, namely, perceived simplicity, clarity, task adequacy, trust calibration, actionability, transparency, faithfulness, and overall interpretability. We then benchmark XAI explanation methods across various datasets, machine learning models, and stakeholder personas. Human validation shows a strong positive association between LLM-generated and human ratings (Spearman's rho=.693, p<.001). Together, LLM-based evaluations can capture systematic differences in XAI explanation quality and provide a scalable and reproducible framework for comparative assessment of XAI explanations.

0.16

NeuroRule: Making Black-Box Neural Networks Explainable through Rule-set Evolution

cs.NE cs.LG ·2026-09-22 ·Tapaswini Kodavanti, Hormoz Shahrzad, Risto Miikkulainen
similarity 0.56
Abstract

High-capacity neural network models have achieved state-of-the-art performance across diverse classification tasks, yet they frequently operate as black-box models, lacking the transparency necessary for critical decision-making. Such opacity creates a persistent trade-off between performance and explainability. This paper proposes a solution to address this gap: the NeuroRule knowledge distillation framework that results in explainable rule-sets from neural network models. NeuroRule adapts the EVOTER rule-set evolution infrastructure to treat neural networks as targets for the evolution process, distilling their performance into concise sets of propositional logic expressions. There are three primary contributions: (1) an evolutionary method for distilling black-box neural network models into explicit rule-set models; (2) a method for making rule sets more explainable by including a conciseness objective to evolution; and (3) a demonstration that the distillation is viable even without access to the original neural network training data. The paper thus establishes that black-box neural network models can be made explainable and therefore useful in real-world applications where trustworthiness is paramount.

0.21

How to train your model organism

cs.LG ·2026-10-07 ·Xilin Wang, David Bau, Byron C. Wallace
position similarity 0.56
Abstract

Model organisms of alignment-relevant behaviors (e.g., backdoors, sycophancy, spurious correlations) have emerged as a key tool for evaluating whitebox interpretability techniques. We argue that the prevailing practice of training model organisms to a single objective of installing the target behavior is insufficient and propose validating model organisms with respect to three objectives with associated metrics: target-behavior installation, general-capability preservation (i.e., parametric knowledge, chat quality), and output naturalness (i.e., CoT and activations). We re-visit two publicly released organism suites using this validation framework and show that (1) chat quality and CoT naturalness degrade substantially across training recipes, and (2) validation metrics predict how well interpretability methods recover the installed behavior, e.g., a logit lens readout covaries with an organism's general capabilities. We introduce a multi-objective training approach based on model merging to train more realistic model organisms. Finally, on a new suite of model organisms targeting demographic biases in clinical reasoning, we compare training recipes and find that DPO training stays closer to the base model than supervised finetuning, and the proposed model optimization approach better preserves capabilities and naturalness. Auditing this suite with an investigator agent, we again observe validation metrics tracking bias recovery. In sum, training methods shape the interpretability conclusions an organism supports, and we argue that one should consider multiple objectives to draw generalizable conclusions about interpretability methods using (realistic) model organisms.

0.24

WisdomNet: trustable machine learning toward error-free classification

cs.LG cs.AI ·2026-10-11 ·Truong X. Tran, Ramazan S. Aygun
publishedhighly cited similarity 0.56
Abstract

Misclassification is a critical problem in many machine learning applications. Since even the classifier models with high accuracy (e.g., > 95%) still introduce some misclassification error, it may not be possible to rely on the output of a classifier. In this paper, we introduce trustable learning, which prompts the learning model to yield only the true output, thus avoiding misclassifications. Whenever the model cannot decide the output accurately, the learning model should indicate that there could be a misclassification error if it is forced to classify, and hence, it should reject to make a decision or defer it to a human expert. Therefore, we develop a methodology for trustable learning and apply it to artificial neural networks and show that it is possible to develop a classifier with 0% misclassification error. We propose a novel neural network architecture named WisdomNet that could provide zero prediction error by introducing an additional neuron named as conjugate neuron that would indicate whether the network is able to classify the data correctly or not. The WisdomNet architecture can be applied to any previously built model, and we have evaluated WisdomNet with several network architectures such as multilayer perceptron, convolutional neural network, and deep network on different data sets. The results show that the WisdomNet is able to reduce the classification error rate to 0%, while labeling the data is difficult to classify as `reject' at a low percentage of within around 10%.

0.59

Architecture-aware Robustness Evaluation of Explainable Deep Learning for Breast Cancer Diagnosis

cs.CV ·2026-09-18 ·Balenthira Thanusanth, Selvarajah Thuseethan, Roshan G. Ragel, Bimali S. Weerakoon, Ayesh Jayasinghe +1 more
code/data similarity 0.56
Abstract · 12 pages, 7 figures

Explainable Artificial Intelligence (XAI) has become essential in medical image analysis to ensure transparency of deep learning (DL)-based diagnostic systems. However, selecting appropriate XAI techniques for breast cancer recognition remains largely ad hoc, with limited systematic evaluation across different DL architectures. This study presents a systematic architecture-aware evaluation protocol to assess the effectiveness of nine widely used XAI techniques across four categories of DL models: very deep, lightweight, transformer-based and hybrid neural networks. The evaluation is conducted on a breast ultrasound dataset comprising 780 images using clinically aligned spatial metrics, including Pointing Game, Intersection over Union and Mean Coverage, to quantify agreement between generated explanations and expert-annotated lesion regions. Results indicate that explanation quality is primarily influenced by the interaction between model architecture and XAI method, rather than any single technique consistently outperforming others. Hybrid architectures produce more spatially coherent explanations, while lightweight and transformer-based models exhibit greater variability across methods. The findings show that no single technique generalises across architectures and evaluation criteria, emphasising the need for joint selection of DL models and XAI techniques. Explainability depends on both model design and explanation strategy and should not be considered independently. This work provides a structured evaluation protocol and practical guidance for selecting XAI techniques in breast cancer diagnosis, supporting more transparent clinical decision-support systems. \textcolor{blue}{Code is publicly available at https://github.com/Nishan-Charlie/Explainable-AI}

0.29