2026 Mejores herramientas de Evaluación de LLM

O

OpenTrain AI

Evaluación de LLM

Marketplace y servicio gestionado para contratar formadores de IA, etiquetadores de datos y expertos de dominio previamente evaluados, para RLHF, evaluación, red teaming, anotación y flujos de trabajo con agentes en tus herramientas actuales.

A

Agnost AI

Observabilidad de IA

Agnost AI monitors AI agents in production by analyzing conversations, traces, user corrections, and frustration. It helps teams identify recurring failures, review suggested prompt fixes, and apply approved changes with replay-based regression coverage.

F

FastRouter.ai

Gateway y enrutamiento de IA

An OpenAI-compatible gateway for routing, evaluating, and governing access to 200+ AI models.

S

Sourcebae

Asistente de reclutamiento con IA

Expert matching, annotation workflows, and quality dashboards for AI training and evaluation.

O

OverallGPT

Evaluación de LLM

Compara en paralelo las respuestas de varios modelos de IA a una misma pregunta y consulta una respuesta conjunta que sintetiza sus aportaciones.

O

Oqoqo

Evaluación de LLM

Plataforma de evaluación para ejecutar experimentos repetibles con agentes en entornos realistas y aislados. Permite crear benchmarks privados, comparar agentes y modelos e investigar por qué los flujos de trabajo tienen éxito o fallan.

I

inferock-bench

Gateway y enrutamiento de IA

inferock-bench is a local diagnostic proxy for tracking LLM API usage, provider-reported costs, failures, and billing-integrity signals. It helps developers inspect calls to OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints using locally stored receipts.

C

ComputeArena

Evaluación de LLM

Directorio comunitario de benchmarks para medir el rendimiento de modelos de IA locales en distintos chips, cuantizaciones y entornos de ejecución.

Q

QAgent

Asistentes de pruebas con IA

QAgent is an AI agent testing and quality assurance platform for developers and agile teams. It connects to an agent through a webhook or endpoint, runs automated test cases, and evaluates responses for groundedness, policy adherence, prompt compliance, and related quality dimensions.

N

NovaSynth

Asistentes de pruebas con IA

Plataforma de pruebas de preproducción para agentes de IA que simula usuarios en llamadas SIP reales, audio de LiveKit y chat para ensayar conversaciones, detectar fallos y validar correcciones antes del lanzamiento.

J

Jev State

Creador de agentes de IA

Jev State is a workspace for defining, testing, and regression-checking conversational decisions powered by Jev. It helps teams inspect why an agent takes a step and export runnable TypeScript and tests for an application.

A

ALIA

Centros de modelos de IA

ALIA est une infrastructure de données IA spécialisée dans les langues, cultures et connaissances africaines. Elle permet aux organisations de collecter, annoter, valider et évaluer des données destinées à l’entraînement et à l’évaluation de modèles.

F

fez

Gateway y enrutamiento de IA

Fez is an experimental decision-model platform that turns workflow context into probabilities for model routing, tool selection, escalation, and support triage. It helps teams train and evaluate specialized models for repeated decisions in AI systems.

I

iFixAi

Evaluación de LLM

iFixAi independently audits AI agents for whether they perform their assigned work while respecting permissions, approvals, organizational boundaries, and evidence requirements. It is designed for teams that need to examine deployed or developing agents beyond conventional benchmarks and evals.

G

Giskard

Asistentes de pruebas con IA

Giskard is an AI security and evaluation platform for testing conversational LLM agents before and after deployment. It combines automated red teaming, quality evaluation, runtime guardrails, and remediation workflows for teams responsible for reliable AI systems.

D

DeepEval

Asistentes de pruebas con IA

DeepEval is an open-source LLM evaluation framework for testing and benchmarking AI applications. It helps developers run pytest-native evaluations, score outputs and agent traces, and iterate on systems across text, image, audio, and voice workflows.

T

TensorZero

Gateway y enrutamiento de IA

TensorZero is an open-source LLMOps platform for building and operating production-grade LLM applications. Its stated scope combines an LLM gateway with observability, evaluation, optimization, and experimentation tools.

G

Galileo

Observabilidad de IA

Galileo is an AI observability and evaluation platform for testing, debugging, and governing LLM and agent systems across development and production. It helps teams turn evaluation results into production guardrails and monitor AI behavior at scale.

P

Promptfoo

Asistentes de pruebas con IA

Promptfoo is an AI security and testing platform for evaluating LLM applications, agents, models, and workflows. It helps developers and security teams find vulnerabilities, validate guardrails, map findings to security frameworks, and track remediation through development and deployment.

T

TruLens

Infraestructura para agentes de IA

TruLens is an open-source library for tracing and evaluating AI agents and other LLM applications. It helps development teams inspect agent steps, score quality with configurable metrics, and compare application versions using OpenTelemetry-based instrumentation.