Awesome GitHub RepositoriesModel Interpretability Frameworks

SHAP is an explainable AI toolkit that provides a game theoretic framework for interpreting machine learning model predictions. It functions as a feature attribution engine, decomposing model outputs into the sum of individual feature effects to clarify how specific input variables influence a final decision. By assigning importance values to these inputs, the library enables users to understand the logic behind complex predictive models. The project distinguishes itself through its versatility and specialized calculation methods. It operates as a model-agnostic diagnostic library, capable of

pykan is a library for implementing Kolmogorov-Arnold Networks, replacing fixed node activation functions with learnable spline functions located on the network edges. It serves as an interpretable AI framework and symbolic regression tool designed to derive transparent mathematical rules from complex data. The project focuses on converting learned numerical functions into human-readable symbolic expressions through library matching and formula conversion. It utilizes additive-compositional topologies and learnable piecewise polynomial segments to approximate non-linear mappings. The framewo

This project is an agnostic model interpretability framework and explainability tool designed to provide local interpretable explanations for individual predictions. It functions as a local surrogate model that approximates the behavior of any machine learning classifier or regression model to identify the most influential features for a specific instance. The framework is designed to be model-agnostic, meaning it can explain predictions across tabular, text, and image data regardless of the underlying architecture. It employs local linear approximations and feature importance visualization t

Captum is an open-source library for explaining model predictions by attributing them to input features, neurons, and layers using gradient-based and perturbation-based methods. It provides a modular framework for implementing, evaluating, and combining a range of explanation techniques, including gradient-based attribution, perturbation-based analysis, game-theoretic Shapley value approximation, and surrogate model explanations, with support for parallelization and noise stabilization. The library distinguishes itself through its breadth of attribution methods and its support for advanced in

Lit is a machine learning interpretability framework and model debugging tool designed to analyze model behavior and performance. It serves as an interpretability dashboard for large language models and a general performance analyzer for text, image, and tabular datasets. The project distinguishes itself through a comprehensive suite of interpretability tools, including salience map generation for feature attribution, the creation of synthetic and counterfactual examples to test robustness, and the projection of high-dimensional embeddings into visual spaces via UMAP or PCA. It further enable

Transformers-interpret هي مكتبة تشخيصية مصممة لقابلية تفسير نماذج تعلم الآلة القائمة على المحولات (transformer-based). تعمل كإطار عمل للإسناد يحدد مساهمة رموز الإدخال الفردية في التنبؤات النهائية للنموذج، مما يسمح للمستخدمين بتدقيق أنماط القرار وتصحيح أخطاء مهام معالجة اللغة الطبيعية. تستخدم المكتبة التحليل القائم على التدرج والاستبطان القائم على الخطافات (hooks) لتتبع كيفية تأثير ميزات إدخال محددة على مخرجات النموذج. من خلال تعيين درجات الإسناد الرقمية المجردة مرة أخرى إلى وحدات لغوية قابلة للقراءة البشرية، فإنها توفر رؤية واضحة لكيفية معالجة النماذج للنص. يدعم إطار العمل التحليل المستهدف، مما يتيح للمستخدمين شرح التنبؤات لفئات محددة أو فحص علاقات الإدخال الزوجية. بعيداً عن الإسناد الأساسي، تتضمن الأداة قدرات تصور تولد تمثيلات رسومية وجدولية لأهمية الميزة. تساعد هذه المخرجات في التحقق من أن النماذج تعتمد على بيانات ذات صلة بدلاً من أنماط غير مقصودة، مما يسهل فهماً أعمق لسلوك النموذج عبر بنيات المحولات المختلفة.