2026년 최고의 LLM 평가 도구

O

OpenTrain AI

LLM 평가

OpenTrain AI는 사전 검증된 AI 트레이너, 데이터 라벨러, 도메인 전문가를 채용하는 마켓플레이스이자 관리형 서비스입니다. RLHF, 평가, 레드팀, 어노테이션, 에이전트 워크플로를 지원합니다.

A

Agnost AI

AI 옵저버빌리티

Agnost AI monitors AI agents in production by analyzing conversations, traces, user corrections, and frustration. It helps teams identify recurring failures, review suggested prompt fixes, and apply approved changes with replay-based regression coverage.

F

FastRouter.ai

AI 게이트웨이 및 라우팅

An OpenAI-compatible gateway for routing, evaluating, and governing access to 200+ AI models.

S

Sourcebae

AI 채용 어시스턴트

Expert matching, annotation workflows, and quality dashboards for AI training and evaluation.

O

OverallGPT

LLM 평가

OverallGPT compares AI answers to one question side by side and combines the responses into an Overall Answer.

O

Oqoqo

LLM 평가

Oqoqo is an evaluation platform for running repeatable agent experiments in realistic, isolated environments. Teams use it to build private benchmarks, compare agents and models, and investigate why agent workflows succeed or fail.

I

inferock-bench

AI 게이트웨이 및 라우팅

inferock-bench is a local diagnostic proxy for tracking LLM API usage, provider-reported costs, failures, and billing-integrity signals. It helps developers inspect calls to OpenAI, Anthropic, Gemini Developer API, and pinned OpenRouter endpoints using locally stored receipts.

C

ComputeArena

LLM 평가

ComputeArena is a community benchmark directory for measuring local AI model throughput across chips, quantisations, and runtimes. It helps developers run offline benchmarks, inspect signed reports, and compare decode and prefill performance on compatible workloads.

Q

QAgent

AI 테스트 어시스턴트

QAgent is an AI agent testing and quality assurance platform for developers and agile teams. It connects to an agent through a webhook or endpoint, runs automated test cases, and evaluates responses for groundedness, policy adherence, prompt compliance, and related quality dimensions.

N

NovaSynth

AI 테스트 어시스턴트

NovaSynth is a pre-production testing platform for AI agents that uses synthetic users across real SIP phone calls, LiveKit audio, and chat. It helps teams rehearse multi-turn interactions, surface failures, and validate fixes before releasing agents to customers.

J

Jev State

AI 에이전트 빌더

Jev State is a workspace for defining, testing, and regression-checking conversational decisions powered by Jev. It helps teams inspect why an agent takes a step and export runnable TypeScript and tests for an application.

A

ALIA

AI 모델 허브

ALIA est une infrastructure de données IA spécialisée dans les langues, cultures et connaissances africaines. Elle permet aux organisations de collecter, annoter, valider et évaluer des données destinées à l’entraînement et à l’évaluation de modèles.

F

fez

AI 게이트웨이 및 라우팅

Fez is an experimental decision-model platform that turns workflow context into probabilities for model routing, tool selection, escalation, and support triage. It helps teams train and evaluate specialized models for repeated decisions in AI systems.

I

iFixAi

LLM 평가

iFixAi independently audits AI agents for whether they perform their assigned work while respecting permissions, approvals, organizational boundaries, and evidence requirements. It is designed for teams that need to examine deployed or developing agents beyond conventional benchmarks and evals.

G

Giskard

AI 테스트 어시스턴트

Giskard is an AI security and evaluation platform for testing conversational LLM agents before and after deployment. It combines automated red teaming, quality evaluation, runtime guardrails, and remediation workflows for teams responsible for reliable AI systems.

D

DeepEval

AI 테스트 어시스턴트

DeepEval is an open-source LLM evaluation framework for testing and benchmarking AI applications. It helps developers run pytest-native evaluations, score outputs and agent traces, and iterate on systems across text, image, audio, and voice workflows.

T

TensorZero

AI 게이트웨이 및 라우팅

TensorZero is an open-source LLMOps platform for building and operating production-grade LLM applications. Its stated scope combines an LLM gateway with observability, evaluation, optimization, and experimentation tools.

G

Galileo

AI 옵저버빌리티

Galileo is an AI observability and evaluation platform for testing, debugging, and governing LLM and agent systems across development and production. It helps teams turn evaluation results into production guardrails and monitor AI behavior at scale.

P

Promptfoo

AI 테스트 어시스턴트

Promptfoo is an AI security and testing platform for evaluating LLM applications, agents, models, and workflows. It helps developers and security teams find vulnerabilities, validate guardrails, map findings to security frameworks, and track remediation through development and deployment.

T

TruLens

AI 에이전트 인프라

TruLens is an open-source library for tracing and evaluating AI agents and other LLM applications. It helps development teams inspect agent steps, score quality with configurable metrics, and compare application versions using OpenTelemetry-based instrumentation.