Open-source alternatives to Explainx

⬛ Python Individual Conditional Expectation Plot Toolbox

.. image:: sphinx/source/images/GammaFacetLogoRGB_LB.svg

The official implementation of "The Shapley Value of Classifiers in Ensemble Games" (CIKM 2021).

WeightWatcher (WW) is an open-source, diagnostic tool for analyzing Deep Neural Networks (DNN), without needing access to training or even test data. It is based on theoretical research into Why Deep Learning Works, based on our Theory of Heavy-Tailed Self-Regularization (HT-SR). It uses ideas…

Python implementation of the rulefit algorithm

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).

Explainable outlier/anomaly detection based on smart decision tree grouping, similar in spirit to the GritBot software developed by RuleQuest research. Written in C++ with interfaces for R and Python (additional Ruby wrapper can be found here). Supports columns of types numeric, categorical,…

🦊 Xplique (pronounced \ɛks.plik\ ) is a Python toolkit dedicated to explainability. The goal of this library is to gather the state of the art of Explainable AI to help you understand your complex neural network models. Originally built for Tensorflow's model it also works for PyTorch models…

title: 'DIANNA: Deep Insight And Neural Network Analysis' tags: - Python - explainable AI - deep neural networks - ONNX - benchmark sets authors: - name: Elena Ranguelova^co-first author # note this makes a footnote saying 'co-first author' orcid: 0000-0002-9834-1756 affiliation: 1 - name:…

A Tree based feature selection tool which combines both the Boruta feature selection algorithm with shapley values.

XAI - An eXplainability toolbox for machine learning

NeuroX provide all the necessary tooling to perform Interpretation and Analysis of (Deep) Neural Networks centered around Probing. Specifically, the toolkit provides:

TorchCAM: class activation explorer

effector an eXplainable AI package for tabular data. It:

Interpretability and explainability of data and machine learning models

This repo contains the code for our talk "Demystifying the neural network black box". Slides are available on Speaker Deck. This code has not been maintained for over a year. It's archived on 2024-12-18.

tensorflow implementation of Grad-CAM (CNN visualization)

Interpret is an interpretable machine learning library and glassbox model framework. It provides toolkits for training inherently transparent models and applying post-hoc explanation techniques to make machine learning predictions human-understandable. The framework distinguishes itself by integrating differential privacy into the training of interpretable models to prevent sensitive data from leaking through explanations. It also features a visualization tool for rendering interactive decision paths and model behavior. The library covers model explainability through feature importance calcu

Interpret Community SDK

This repository is the official implementation of "Hyperdimensional Probe: Decoding LLM Representations via Vector Symbolic Architectures". This work combines symbolic representations and neural probing to introduce Hyperdimensional Probe, a new paradigm for decoding LLM vector space into…

https://doi.org/10.48550/arXiv.2404.03623)

Gradient class activation maps are a visualization technique for deep learning networks.

This project is a computer vision explainable AI library and framework for PyTorch, providing a suite of tools to visualize and audit the internal decision-making processes of deep neural networks. It serves as a neural network attribution tool and debugging utility to identify which image regions drive model predictions. The library is distinguished by its support for both gradient-based and gradient-free attribution methods, allowing for the generation of visual heatmaps and attribution maps without requiring modifications to the original model source code. It further differentiates itself

GIM (Gradient Interaction Modifications) is a state-of-the-art feature attribution method and circuit discovery method. It currently leads the leaderboard for the Mechanistic Interpretability Benchmark, while being as fast as gradients.

Leave One Feature Out Importance