2026 Best LLM Evaluation tools

Explore LLM evaluation tools, benchmarks, and testing platforms to measure model quality, compare outputs, detect risks, and improve AI applications.

AI Infrastructure

Explore this collection

Tools

Agnost AI preview
Agnost AI logo

Agnost AI

AI Observability

Agnost AI monitors AI agents in production by analyzing conversations, traces, user corrections, and frustration. It helps teams identify recurring failures, review suggested prompt fixes, and apply approved changes with replay-based regression coverage.

Freemium
Adaline preview
Adaline logo

Adaline

AI Observability

Observability and evaluation tools for teams improving AI agents from production data.

FastRouter.ai preview
FastRouter.ai logo

FastRouter.ai

AI Gateway And Routing

An OpenAI-compatible gateway for routing, evaluating, and governing access to 200+ AI models.

Freemium
Sourcebae preview
Sourcebae logo

Sourcebae

AI Recruiting Assistant

Expert matching, annotation workflows, and quality dashboards for AI training and evaluation.

Paid
Klavis AI preview
Klavis AI logo

Klavis AI

AI Agent Infrastructure

Klavis AI offers live environments to train and evaluate AI agents with coding and tool-use data.

Freemium
Hamming AI preview
Hamming AI logo

Hamming AI

AI Observability

Testing, red teaming, and production monitoring for teams building voice and chat agents.

Freemium
MCPJam preview
MCPJam logo

MCPJam

AI Testing Assistant

MCPJam is a testing and evaluation platform for MCP servers. It helps developers inspect servers locally, run user and model-based tests, and add behavior checks to CI/CD workflows.

Freemium
Dynamo AI preview
Dynamo AI logo

Dynamo AI

AI Observability

Enterprise AI controls for evaluating, securing, monitoring, and governing generative and agentic AI.

TruLens preview
TruLens logo

TruLens

AI Agent Infrastructure

TruLens is an open-source library for tracing and evaluating AI agents and other LLM applications. It helps development teams inspect agent steps, score quality with configurable metrics, and compare application versions using OpenTelemetry-based instrumentation.

Freemium
TensorZero preview
TensorZero logo

TensorZero

AI Gateway And Routing

TensorZero is an open-source LLMOps platform for building and operating production-grade LLM applications. Its stated scope combines an LLM gateway with observability, evaluation, optimization, and experimentation tools.

Freemium
iFixAi preview
iFixAi logo

iFixAi

LLM Evaluation

iFixAi independently audits AI agents for whether they perform their assigned work while respecting permissions, approvals, organizational boundaries, and evidence requirements. It is designed for teams that need to examine deployed or developing agents beyond conventional benchmarks and evals.

Freemium
OverallGPT preview
OverallGPT logo

OverallGPT

LLM Evaluation

OverallGPT compares AI answers to one question side by side and combines the responses into an Overall Answer.

Freemium
Jev State preview
Jev State logo

Jev State

AI Agent Builder

Jev State is a workspace for defining, testing, and regression-checking conversational decisions powered by Jev. It helps teams inspect why an agent takes a step and export runnable TypeScript and tests for an application.

Freemium
inferock-bench preview
inferock-bench logo

inferock-bench

AI Gateway And Routing

inferock-bench is a local diagnostic proxy for tracking LLM API usage, provider-reported costs, failures, and billing-integrity signals. It helps developers inspect calls to OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints using locally stored receipts.

Freemium
Oqoqo preview
Oqoqo logo

Oqoqo

LLM Evaluation

Oqoqo is an evaluation platform for running repeatable agent experiments in realistic, isolated environments. Teams use it to build private benchmarks, compare agents and models, and investigate why agent workflows succeed or fail.

Freemium
ComputeArena preview
ComputeArena logo

ComputeArena

LLM Evaluation

ComputeArena is a community benchmark directory for measuring local AI model throughput across chips, quantisations, and runtimes. It helps developers run offline benchmarks, inspect signed reports, and compare decode and prefill performance on compatible workloads.

Freemium
fez preview
fez logo

fez

AI Gateway And Routing

Fez is an experimental decision-model platform that turns workflow context into probabilities for model routing, tool selection, escalation, and support triage. It helps teams train and evaluate specialized models for repeated decisions in AI systems.

Freemium
QAgent preview
QAgent logo

QAgent

AI Testing Assistant

QAgent is an AI agent testing and quality assurance platform for developers and agile teams. It connects to an agent through a webhook or endpoint, runs automated test cases, and evaluates responses for groundedness, policy adherence, prompt compliance, and related quality dimensions.

Freemium
ALIA preview
ALIA logo

ALIA

AI Model Hubs

ALIA est une infrastructure de données IA spécialisée dans les langues, cultures et connaissances africaines. Elle permet aux organisations de collecter, annoter, valider et évaluer des données destinées à l’entraînement et à l’évaluation de modèles.

Freemium
NovaSynth preview
NovaSynth logo

NovaSynth

AI Testing Assistant

NovaSynth is a pre-production testing platform for AI agents that uses synthetic users across real SIP phone calls, LiveKit audio, and chat. It helps teams rehearse multi-turn interactions, surface failures, and validate fixes before releasing agents to customers.

Freemium