Text and image moderation
The `omni-moderation-latest` model accepts text and image inputs. It does not classify audio, and image files can be up to 20 MB.
OpenAI Moderation
Freemium
OpenAI Moderation helps applications identify potentially harmful content in text and images. Developers can classify standalone inputs or receive moderation signals alongside generated responses through the OpenAI API.
OpenAI Moderationとは？
OpenAI Moderation is an OpenAI API capability for identifying potentially harmful content in text and images. Applications can submit standalone inputs to the moderation endpoint or request moderation results as part of a response-generation request. The returned signals can support application policies such as filtering content, routing items for review, or taking action on accounts that submit flagged material.
OpenAI Moderationでできること
Standalone classification
Use the moderation endpoint when an application needs to assess text or images without generating a model response.
Inline moderation for generated content
Add a moderation object to a Responses API or Chat Completions API generation request to receive moderation scores for the model input and generated output without making a separate moderation request.
Interpretable moderation results
Results can include a flagged status, moderation categories, and scores. Applications should review these results before displaying generated output or taking downstream action.
Policy-oriented workflows
Moderation results can be incorporated into application rules for filtering, human review, or intervention on accounts that submit flagged content.
利用シーン
“Pre-screen user submissions”
Classify text or images before publishing them in a community, marketplace, or other user-facing experience, then apply the application’s content policy.
“Review generated responses”
Request moderation signals alongside generated text and inspect the input and output results before showing a response to a user.
“Route borderline content for review”
Use categories, scores, and flagged statuses to send selected items to a human-review workflow instead of applying the same action to every submission.
“Moderate multimodal inputs”
Assess supported combinations of text and images when an application accepts both modalities, while handling audio through a separate approach.
よくある質問
What inputs does OpenAI Moderation support?
The documented `omni-moderation-latest` model accepts text and image inputs. It does not classify audio. Image files can be up to 20 MB.
Can moderation run without generating a response?
Yes. The moderation endpoint can classify standalone text or images without generating a model response.
Can I moderate generated content in the same request?
Yes. For requests made through the Responses API or Chat Completions API, add a top-level moderation object with a moderation model. The API returns moderation results for the model input and generated output while the model still generates normally.
What should an application do with moderation results?
The results can inform actions such as filtering content, routing a request for review, or intervening with accounts that submit flagged content. OpenAI advises reviewing results before displaying generated output or taking downstream actions.
Is the Moderation API intended for CSAM detection?
No. The documented guidance says not to send known or suspected child sexual abuse material to the Moderation API. It is not designed for CSAM detection or handling and is not a substitute for dedicated child-safety safeguards.
クイック情報
- Category
- Developer Tool
- Platform
- OpenAI API
- Primary inputs
- Text and images
- Documented model
- omni-moderation-latest
- Workflow options
- Standalone classification or moderation alongside generated responses
- Endpoint pricing
- Free to use
OpenAI Moderationの代替品
Sightengine
sightengine.com
画像・動画・テキスト・音声をモデレーション・分析するAPIプラットフォーム
Arena
arena.im
Arenaは、ウェブサイトやアプリにコメント、ライブブログ、グループチャット、投票、AI支援モデレーションを追加できるコミュニティエンゲージメントプラットフォームです。パブリッシャー、メディア、ブランド、ECチームが自社プロパティに交流を集約できます。
CommunityOne
communityone.io
CommunityOneは、AIサポート、モデレーション、エンゲージメント、発見、分析に対応するDiscordコミュニティプラットフォームです。運営者は日常業務を自動化し、Discord、ウェブ、YouTubeコメントでのメンバーの交流を把握できます。
Sendbird
sendbird.com
チャット、顧客サポート、モデレーション、AI支援に対応する企業向けAI顧客体験プラットフォーム
Hive
thehive.ai
Hive provides APIs for understanding, searching, moderating, and generating text, image, video, and audio content. Developers can use its pre-trained and open-source models for content workflows such as moderation, classification, brand protection, and media generation.
Canopy
canopy.us
Canopyは、家族や大人向けのコンテンツフィルタリング・ペアレンタルコントロールアプリです。ウェブサイト、SNS、AIチャットボット上の露骨な画像・動画・テキストを、インターネット全体を遮断せずにフィルタリングします。