Lit is a machine learning interpretability framework and model debugging tool designed to analyze model behavior and performance. It serves as an interpretability dashboard for large language models and a general performance analyzer for text, image, and tabular datasets.
The project distinguishes itself through a comprehensive suite of interpretability tools, including salience map generation for feature attribution, the creation of synthetic and counterfactual examples to test robustness, and the projection of high-dimensional embeddings into visual spaces via UMAP or PCA. It further enables side-by-side comparison of multiple model versions and the quantification of high-level concept importance.
The framework covers a broad capability surface including quantitative model evaluation with confusion matrices and custom metric calculation, interactive data management via slicing and filtering, and the visualization of structured predictions. It provides an extensible architecture that supports custom visualization development and the integration of remote model endpoints.
The interface can be deployed via Docker containers or embedded directly within notebook output cells.