Open source and extensible
Captum is an open source library for model interpretability research, so teams can adapt it to their own attribution and explanation workflows.
Captum
ClaimCaptum is an open source PyTorch library for model interpretability. Explain predictions, inspect internal components, and benchmark attribution methods across vision, text, and multimodal workflows.
What Captum does
Captum is an open source library for model interpretability built on PyTorch. It is designed to help users understand which features, layers, neurons, or other model components contribute to a model’s output.
The project positions itself as a tool for researchers, model developers, and application engineers who want to explain predictions, troubleshoot unexpected behavior, and benchmark interpretability algorithms across a range of PyTorch-based models.
Site documentation and tutorials show that Captum is used on vision, text, multimodal, question answering, regression, recommender, segmentation, and language model workflows, with examples centered on attribution methods such as Integrated Gradients and related techniques.
Key features
Multi-modal support
The site highlights support for interpreting models across modalities including vision, text, and multimodal inputs.
PyTorch-native workflow
Captum is built on PyTorch and is designed to work with most PyTorch model types with minimal modification to the original network.
Multiple interpretability algorithms
The tutorials show a range of attribution methods such as Integrated Gradients, DeepLIFT, GradientShap, Occlusion, LRP, GradCAM, TCAV, and LIME.
Layer- and neuron-level analysis
Documentation shows it can be used not only for final outputs, but also for layers, neurons, embeddings, and attention components.
Distributed attribution examples
The tutorial set includes examples for distributed execution with `torch.distributed` and `DataParallel`.
Common use cases
Feature attribution for predictions
Use Captum to attribute a model prediction back to input features when you want to see which inputs matter most for a specific output.
Interpreting models across modalities
Apply Captum to vision, text, or multimodal models when you need explanation workflows tailored to the modality you are working with.
Internal model analysis
Use the library to inspect layers, neurons, embeddings, and attention patterns when the question is about how the model is internally making decisions.
Debugging and explanation in production
Use Captum on production-trained models to troubleshoot unexpected outputs and to help explain results to end users in a more understandable way.
Interpretability research and benchmarking
Use the tutorial examples as a starting point for benchmarking new interpretability ideas against existing algorithms in the library.
Pros and Cons
Pros
- Open source and built specifically for interpretability on PyTorch.
- Supports multiple modalities, including vision, text, and multimodal models.
- Offers several attribution and explanation methods rather than a single algorithm.
- Includes examples that go beyond output attribution to layers, neurons, embeddings, and attention.
- Provides tutorials for practical workflows and different model types.
Cons
- The public site does not provide pricing details, and the pricing page currently resolves to a 404.
- The tutorials assume familiarity with PyTorch, so new users may need baseline PyTorch knowledge before they can follow along.
FAQ
What is Captum used for?
Captum is an open source library for model interpretability built on PyTorch. It provides attribution methods and related workflows for understanding which inputs or internal components contribute to a model’s output.
What kinds of models can Captum help interpret?
The source highlights Captum’s use with vision, text, multimodal models, question answering, semantic segmentation, regression, recommender systems, and LLM attribution.
Who is Captum for?
The introduction page says Captum is aimed at model developers, interpretability researchers, and application engineers using trained models in production.
How do you get started with Captum?
The homepage shows installation via either `conda install captum -c pytorch` or `pip install captum`, and the tutorials assume familiarity with PyTorch.
Does Captum list pricing?
The public site does not show pricing information, and the pricing URL currently returns a GitHub Pages 404 page.
Quick Facts
- Category
- Developer Tool
- Platform
- PyTorch
- Primary users
- Model developers, interpretability researchers, application engineers
- Source domain
- captum.ai
- License/status
- Open source
- Pricing
- Not listed on the public site