# Evidently AI

Open-source framework for evaluating and monitoring LLM applications, AI agents, and ML models.

- Findings24: <https://findings24.com/zh-CN/products/evidently-ai>
- 网站: <https://evidentlyai.com>
- 价格: Freemium
- 分类: AI 可观测性, AI 测试助手, 大语言模型评测

## 什么是 Evidently AI？

Evidently AI is an open-source framework for evaluating, testing, and monitoring LLM applications, RAG systems, AI agents, and predictive machine-learning models. It helps teams assess output quality, safety, and reliability during development and track changes after deployment. Evaluations can combine built-in metrics with custom rules, classifiers, prompts, or models, while reports and monitoring views make results available for review.

## Evidently AI 能做什么？

### Automated evaluations

Measure AI output quality, safety, and reliability, and run evaluation workflows as part of a pipeline.

### Built-in and custom metrics

Use a library of more than 100 built-in metrics or define evaluations with custom prompts, models, and rules.

### Synthetic test data

Generate realistic, edge-case, or adversarial inputs tailored to an application, including benign and hostile prompts.

### Evaluation reports

Explore evaluation results in visual reports that can be shared with others.

### Ongoing monitoring

Track evaluation results and quality checks in a live dashboard to identify drift, regressions, and emerging risks.

### Coverage for LLM and predictive ML systems

Evaluate LLM behavior, retrieval quality, and safety alongside predictive performance, data drift, and data quality.

## 使用场景

### Validate a RAG application

Assess retrieval quality and context relevance, and check generated answers for factuality or hallucinations before release.

### Test an AI agent workflow

Use edge-case and adversarial inputs to evaluate agent responses and expose risks such as unsafe output or prompt-based attacks.

### Add evaluations to a development pipeline

Run quality and safety checks during application updates, then review results in visual reports.

### Monitor predictive models in production

Track model performance, data quality, and data drift as production data changes.

## 常见问题

### What systems can Evidently evaluate?

It supports LLM-powered products such as chatbots, RAG applications, AI agents, and copilots, as well as predictive machine-learning systems.

### Can I define evaluations for my own requirements?

Yes. The framework offers more than 100 built-in metrics and supports custom evaluations using prompts, models, or rules, including combinations of rules, classifiers, and LLM-based evaluations.

### Can evaluations run as part of a pipeline?

Yes. Evidently describes running evaluations in a pipeline and reviewing the results through shareable visual reports.

### What kinds of checks are available for LLM applications?

Examples include guideline and format adherence, hallucinations and factuality, PII detection, retrieval quality and context relevance, sentiment, toxicity, tone, and trigger words.

## 快速信息

- **Product category**: Open-source AI evaluation and observability framework
- **License**: Apache 2.0
- **Supported systems**: LLM applications, RAG systems, AI agents, and predictive ML models
- **Evaluation methods**: Built-in metrics, rules, classifiers, and LLM-based evaluations
- **Website**: evidentlyai.com