Awesome GitHub RepositoriesModel Interpretability Frameworks

SHAP is an explainable AI toolkit that provides a game theoretic framework for interpreting machine learning model predictions. It functions as a feature attribution engine, decomposing model outputs into the sum of individual feature effects to clarify how specific input variables influence a final decision. By assigning importance values to these inputs, the library enables users to understand the logic behind complex predictive models. The project distinguishes itself through its versatility and specialized calculation methods. It operates as a model-agnostic diagnostic library, capable of

pykan is a library for implementing Kolmogorov-Arnold Networks, replacing fixed node activation functions with learnable spline functions located on the network edges. It serves as an interpretable AI framework and symbolic regression tool designed to derive transparent mathematical rules from complex data. The project focuses on converting learned numerical functions into human-readable symbolic expressions through library matching and formula conversion. It utilizes additive-compositional topologies and learnable piecewise polynomial segments to approximate non-linear mappings. The framewo

This project is an agnostic model interpretability framework and explainability tool designed to provide local interpretable explanations for individual predictions. It functions as a local surrogate model that approximates the behavior of any machine learning classifier or regression model to identify the most influential features for a specific instance. The framework is designed to be model-agnostic, meaning it can explain predictions across tabular, text, and image data regardless of the underlying architecture. It employs local linear approximations and feature importance visualization t

Captum is an open-source library for explaining model predictions by attributing them to input features, neurons, and layers using gradient-based and perturbation-based methods. It provides a modular framework for implementing, evaluating, and combining a range of explanation techniques, including gradient-based attribution, perturbation-based analysis, game-theoretic Shapley value approximation, and surrogate model explanations, with support for parallelization and noise stabilization. The library distinguishes itself through its breadth of attribution methods and its support for advanced in

Lit is a machine learning interpretability framework and model debugging tool designed to analyze model behavior and performance. It serves as an interpretability dashboard for large language models and a general performance analyzer for text, image, and tabular datasets. The project distinguishes itself through a comprehensive suite of interpretability tools, including salience map generation for feature attribution, the creation of synthetic and counterfactual examples to test robustness, and the projection of high-dimensional embeddings into visual spaces via UMAP or PCA. It further enable

Transformers-interpret es una biblioteca de diagnóstico diseñada para la interpretabilidad de modelos de machine learning basados en transformers. Funciona como un framework de atribución que cuantifica la contribución de tokens de entrada individuales a las predicciones finales de un modelo, permitiendo a los usuarios auditar patrones de decisión y depurar tareas de procesamiento de lenguaje natural (NLP). La biblioteca utiliza análisis basado en gradientes e introspección basada en hooks para rastrear cómo características de entrada específicas influyen en las salidas del modelo. Al mapear puntuaciones de atribución numérica abstractas de vuelta a unidades lingüísticas legibles por humanos, proporciona una visión clara de cómo los modelos procesan el texto. El framework admite análisis específicos, permitiendo a los usuarios explicar predicciones para clases específicas o examinar relaciones de entrada por pares. Más allá de la atribución central, la herramienta incluye capacidades de visualización que generan representaciones gráficas y tabulares de la importancia de las características. Estas salidas ayudan a verificar que los modelos se basan en datos relevantes en lugar de patrones no deseados, facilitando una comprensión más profunda del comportamiento del modelo en varias arquitecturas de transformer.