OpenTelemetry-native tracing
Record agent and LLM application activity as spans, including inputs, outputs, latency, token counts, and cost for individual steps.
TruLens
Freemium
TruLens is an open-source library for tracing and evaluating AI agents and other LLM applications. It helps development teams inspect agent steps, score quality with configurable metrics, and compare application versions using OpenTelemetry-based instrumentation.
TruLensとは？
TruLens is an open-source evaluation and observability library for AI agents and LLM applications. It records application activity as traces and evaluates quality at the level of an overall response or individual steps, helping teams connect an unsatisfactory answer to the retrieval, tool call, plan, or generation that caused it.
The product captures details such as inputs, outputs, latency, token usage, and cost for recorded steps. Its evaluation workflow includes metrics for agent behavior, retrieval-augmented generation, and tool use, including context relevance, groundedness, answer relevance, correctness, plan adherence, and tool selection. Evaluation results can be inspected alongside trace details rather than treated as an isolated score.
Teams can start with built-in LLM-judge metrics and adapt them with rubrics, examples, score ranges, and additional instructions. TruLens also supports custom metrics and comparison of application versions through leaderboard-style results that place quality measures alongside latency and cost, making it possible to assess changes such as revised tool arguments, reranking, policy citation, or model selection.
The documentation covers runtime and batch evaluation, evaluation of existing data, human feedback logging, ground-truth workflows, guardrails, and viewing trends and records. Instrumentation is based on OpenTelemetry, and the documentation includes guides and provider references for working with frameworks, model providers, and storage systems. TruLens is distributed under the MIT license; the supplied product pages do not provide a detailed pricing model or plan limits.
TruLensでできること
Step-level evaluation
Score agent behavior and intermediate operations with metrics such as tool selection, plan adherence, context relevance, groundedness, answer relevance, and correctness.
Explainable LLM judges
View score explanations that identify issues such as missing tool calls, unsuitable arguments, or poorly selected context instead of relying only on an aggregate number.
Configurable and custom metrics
Adapt shipped evaluations with rubrics, few-shot examples, score ranges, and additional instructions, or define custom metrics with selectors for the relevant record data.
Version and quality-cost comparison
Compare application versions using evaluation scores together with average latency and total cost to investigate quality and efficiency trade-offs.
利用シーン
“Debugging agent failures”
Inspect a failed answer through its trace to determine whether planning, retrieval, tool selection, tool arguments, or generation introduced the problem.
“Evaluating RAG pipelines”
Measure whether retrieved context is relevant and whether the final answer is grounded in that context, using RAG-focused metrics and evaluation workflows.
“Testing application changes”
Run comparable records across versions and compare quality, latency, and cost when changing prompts, tools, reranking, citations, or models.
“Monitoring deployed applications”
Use runtime evaluation, recorded traces, result views, and trends to follow application behavior after release rather than relying only on pre-launch tests.
“Aligning evaluations to a domain”
Refine judge behavior with domain-specific criteria, examples, and scoring instructions when general-purpose metrics do not capture the team's quality standard.
よくある質問
What does TruLens trace?
It traces AI-agent and LLM-application activity as spans. The documented trace view includes actions such as planning, retrieval, tool calls, and generation, along with inputs, outputs, timing, token usage, and cost where available.
Can evaluation scores be customized?
Yes. The documented metric API supports custom criteria, additional instructions, few-shot examples, score ranges, selectors, and custom metrics. This allows teams to align evaluations with domain-specific requirements.
Can TruLens evaluate an application while it runs?
The documentation includes runtime evaluation and inline evaluations, as well as evaluation on existing data and batch evaluation. The exact workflow depends on the application and instrumentation setup.
Does TruLens only evaluate final answers?
No. Its examples and metrics cover intermediate agent behavior, including plan adherence, tool selection, retrieval context, and other recorded steps, in addition to final-answer quality.
Is detailed pricing available on the supplied product pages?
No pricing tiers, limits, or commercial terms are provided in the supplied pricing-page content. The available product information identifies TruLens as open source and lists an MIT license.
クイック情報
- Category
- Developer Tool
- Primary function
- AI-agent tracing and evaluation
- Instrumentation
- OpenTelemetry-native
- License
- MIT
- Evaluation modes
- Runtime, batch, existing-data, and ground-truth workflows
- Source domain
- trulens.org
TruLensの代替品
Graphiti
www.getzep.com
Graphiti is an open-source GitHub project for building real-time knowledge graphs for AI agents. It provides a codebase that developers can inspect, extend, and contribute to through the repository’s development workflow.
OpenTrade
opentrade.io
OpenTrade is an open-source trading harness for Claude Code and Codex agents. It provides a repository-based app, agent templates, and scheduling-oriented run management for automated trading workflows.
Statewave
www.statewave.ai
Statewave is an open-source, self-hosted memory runtime for AI agents. It stores durable episodes, compiles structured memories with provenance, and returns ranked context bundles for applications that need continuity across sessions.
Firetower
usefiretower.com
Firetower is an open-source, self-hosted control plane for running coding agents on servers you own. It lets developers start agent sessions from repositories, monitor them from desktop or mobile clients, review changes, and ship branches without keeping the initiating device online.
Cognee
www.cognee.ai
Cognee is an open-source memory platform for AI agents and applications. It connects data across sources, builds connected memory using vector, graph, and relational storage, and makes that context available through SDK, cloud, or self-hosted deployments.
Steel
steel.dev
Steel is open-source browser infrastructure for AI agents and web automation. It provides cloud browser sessions through an API, with support for common browser automation tools and workflows such as scraping, research, RPA, and agent-driven web tasks.