Open-source alternatives to Lucid

TreeInterpreter

A library for debugging/inspecting machine learning classifiers and explaining their predictions

⬛ Python Individual Conditional Expectation Plot Toolbox

This project is an agnostic model interpretability framework and explainability tool designed to provide local interpretable explanations for individual predictions. It functions as a local surrogate model that approximates the behavior of any machine learning classifier or regression model to identify the most influential features for a specific instance. The framework is designed to be model-agnostic, meaning it can explain predictions across tabular, text, and image data regardless of the underlying architecture. It employs local linear approximations and feature importance visualization t

SHAP is a machine learning explainer that uses a game-theoretic framework to estimate the contribution of each feature to a model prediction. It provides a set of tools for quantifying how individual input features push a specific output away from a baseline value. The project includes specialized explainers for different architectures, including high-speed implementations for decision trees and ensemble models, linearization algorithms for deep learning networks, and covariance integration for linear models. It also features a model-agnostic interpretability tool that uses a kernel method to

Interpret is an interpretable machine learning library and glassbox model framework. It provides toolkits for training inherently transparent models and applying post-hoc explanation techniques to make machine learning predictions human-understandable. The framework distinguishes itself by integrating differential privacy into the training of interpretable models to prevent sensitive data from leaking through explanations. It also features a visualization tool for rendering interactive decision paths and model behavior. The library covers model explainability through feature importance calcu

This is a PyTorch CNN visualization toolkit designed for neural network interpretability. It provides a set of tools to explain model decisions and analyze the internal behavior of convolutional neural networks through the visualization of activations, gradients, and filters. The project implements specialized techniques for synthesizing representative images, including Deep Dream optimizations to amplify patterns and class-specific image generation via input optimization. It also features a saliency map generator that produces gradient-based heatmaps to identify the specific image regions in

This project is a comprehensive educational resource and technical manual focused on interpretable machine learning and explainable AI. It serves as a textbook and reference for implementing techniques that make complex machine learning models transparent and understandable to humans. The resource provides guidance on both building inherently transparent models, such as decision trees and sparse linear models, and applying post-hoc explanation methods to black-box systems. It details specific methodologies for quantifying feature importance, generating rationales for individual predictions, a

Tensorpack is a high-level TensorFlow neural network framework and research library designed for building and training deep learning models. It provides a collection of reproducible neural network architectures for computer vision, generative tasks, reinforcement learning, and natural language processing. The project distinguishes itself through a specialized deep learning data pipeline that uses pure Python for parallel data loading and streaming. It includes a multi-GPU training orchestrator for distributing workloads via data-parallel strategies and a dedicated interpretability toolkit for

Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, Rory Sayres

moDel Agnostic Language for Exploration and eXplanation

A toolbox to iNNvestigate neural networks' predictions!

Contrastive Explanation (Foil Trees), developed at TNO/Utrecht University

Captum is an open-source library for explaining model predictions by attributing them to input features, neurons, and layers using gradient-based and perturbation-based methods. It provides a modular framework for implementing, evaluating, and combining a range of explanation techniques, including gradient-based attribution, perturbation-based analysis, game-theoretic Shapley value approximation, and surrogate model explanations, with support for parallelization and noise stabilization. The library distinguishes itself through its breadth of attribution methods and its support for advanced in

Interpretability and explainability of data and machine learning models

XAI - An eXplainability toolbox for machine learning

Netron is a visualizer for neural network and machine learning models. It provides a graphical interface that renders model architectures as interactive node-link diagrams, allowing users to inspect internal layers, tensors, and metadata. By performing static analysis, the tool enables the examination of model definitions without executing the underlying machine learning code. The software distinguishes itself through a schema-driven parsing engine that translates diverse proprietary model formats into a unified internal graph structure. This approach ensures interoperability, allowing users

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).

Leave One Feature Out Importance

The official implementation of "The Shapley Value of Classifiers in Ensemble Games" (CIKM 2021).

Code for "High-Precision Model-Agnostic Explanations" paper

TensorFlow.js is a JavaScript machine learning library and browser-based runtime used to build, train, and execute models. It functions as a WebGL accelerated tensor engine, providing a foundation for high-performance linear algebra operations and an automatic differentiation framework for computing gradients. The project distinguishes itself through its ability to run machine learning directly in web environments, supporting both client-side inference and browser-based training. It enables the deployment of Python-based models by converting Keras or TensorFlow models into compatible formats

This repository is a deep learning educational resource and a neural network project suite. It provides a collection of practical TensorFlow implementations and coding projects designed to demonstrate the application of various neural network architectures to real-world data. The project includes specific samples for generative adversarial networks, focusing on synthetic image generation and style translation. It also provides examples of deep learning model construction across different learning paradigms. The codebase covers a broad range of capabilities, including computer vision for imag

This project is a comprehensive machine learning educational resource and tutorial series delivered as a collection of interactive Jupyter Notebooks. It provides practical Python implementations for the end-to-end machine learning lifecycle, covering supervised and unsupervised learning, deep learning, and reinforcement learning. The resource distinguishes itself by providing detailed implementation guides for complex architectures, including transformers, generative adversarial networks, and convolutional neural networks. It also features specialized courseware for developing reinforcement l

cnn-explainer is an interactive web application and educational sandbox designed for visualizing the internal operations and layers of convolutional neural networks. It functions as a tool for understanding how these networks process image data through real-time graphics and interactive visualizations. The project includes a browser-based environment for training small convolutional neural networks on specific image classes. It also provides a model converter that transforms trained neural network files from backend framework formats into web-compatible versions for browser loading. The appl

pykan is a library for implementing Kolmogorov-Arnold Networks, replacing fixed node activation functions with learnable spline functions located on the network edges. It serves as an interpretable AI framework and symbolic regression tool designed to derive transparent mathematical rules from complex data. The project focuses on converting learned numerical functions into human-readable symbolic expressions through library matching and formula conversion. It utilizes additive-compositional topologies and learnable piecewise polynomial segments to approximate non-linear mappings. The framewo