Support content moderation requests and moderating models (for Ollama, OpenAI, and Bedrock) #395

Ololoshechkin · 2025-07-04T23:32:09Z

Fixes: #396

Type of the change

New feature
Bug fix
Documentation fix

Checklist for all pull requests

The pull request has a description of the proposed change
I read the Contributing Guidelines before opening the pull request
The pull request uses develop as the base branch
Tests for the changes have been added
All new and existing tests passed

Additional steps for pull requests adding a new feature

An issue describing the proposed change exists
The pull request includes a link to the issue
The change was discussed and approved in the issue
Docs have been added / updated

...rc/commonTest/kotlin/ai/koog/agents/core/dsl/extension/AIAgentNodesHistoryCompressionTest.kt

...sts/src/jvmTest/kotlin/ai/koog/integration/tests/MultipleLLMPromptExecutorIntegrationTest.kt

integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/OllamaClientIntegrationTest.kt

...all/src/jvmTest/kotlin/ai/koog/prompt/executor/llms/all/MultipleLLMPromptExecutorMockTest.kt

sdubov

In general, look good for me. I would update test data and rebased onto a new develop branch to fix compilation.

prompt/prompt-model/src/commonMain/kotlin/ai/koog/prompt/dsl/ModerationAPI.kt

.../prompt-executor-model/src/commonMain/kotlin/ai/koog/prompt/executor/model/PromptExecutor.kt

...xecutor-llms/src/commonTest/kotlin/ai/koog/prompt/executor/llms/LLMPromptExecutorMockTest.kt

sdubov · 2025-07-09T16:10:42Z

...t-executor-llms/src/commonMain/kotlin/ai/koog/prompt/executor/llms/MultiLLMPromptExecutor.kt

+        logger.debug { "Moderating multi-modal content with model: ${model.id}" }
+
+        val provider = model.provider
+        val client = llmClients[provider] ?: throw IllegalArgumentException("No client found for provider: $provider")


What do you think about extracting this check into a separate function and generalize the error? Similar code is used in other methods above.

...client/src/commonMain/kotlin/ai/koog/prompt/executor/clients/anthropic/AnthropicLLMClient.kt

...r-openai-client/src/commonMain/kotlin/ai/koog/prompt/executor/clients/openai/OpenAIModels.kt

integration-tests/src/jvmTest/kotlin/ai/koog/integration/tests/OllamaClientIntegrationTest.kt

.../prompt-executor-model/src/commonMain/kotlin/ai/koog/prompt/executor/model/PromptExecutor.kt

...ry/src/jvmMain/kotlin/ai/koog/agents/features/opentelemetry/event/ModerationResponseEvent.kt

sdubov · 2025-07-10T13:37:06Z

@Ololoshechkin , btw, it looks like Mistral has Moderation API already as well - https://docs.mistral.ai/capabilities/guardrailing/

prompt/prompt-model/src/commonMain/kotlin/ai/koog/prompt/dsl/ModerationAPI.kt

EugeneTheDev

I've added a few mosly style related comments

prompt/prompt-model/src/commonMain/kotlin/ai/koog/prompt/message/Message.kt

.../prompt-executor-model/src/commonMain/kotlin/ai/koog/prompt/executor/model/PromptExecutor.kt

prompt/prompt-model/src/commonMain/kotlin/ai/koog/prompt/dsl/ModerationAPI.kt

...client/src/commonMain/kotlin/ai/koog/prompt/executor/clients/openai/OpenAIModerationInput.kt

Ololoshechkin · 2025-07-11T10:35:54Z

@sdubov thanks, will apply your suggestions.
As for the Mistral -- we don't support Mistral yet :) So adding Mistral will be a separate task anyway (and we have it in the roadmap)

…ressionTest

github-actions · 2025-07-12T22:17:52Z

Qodana for JVM

446 new problems were found

Inspection name	Severity	Problems
`Check Kotlin and Java source code coverage`	🔶 Warning	379
`Missing KDoc for public API declaration`	🔶 Warning	47
`Unused import directive`	🔶 Warning	16
`Vulnerable imported dependency`	🔶 Warning	4

@@ Code coverage @@
+ 63% total lines covered
7835 lines analyzed, 5003 lines covered
# Calculated according to the filters of your coverage tool

☁️ View the detailed Qodana report

Contact Qodana team

Contact us at [email protected]

Or via our issue tracker: https://jb.gg/qodana-issue
Or share your feedback: https://jb.gg/qodana-discussions

Ololoshechkin requested review from Rizzen, EugeneTheDev and aozherelyeva July 4, 2025 23:32

Ololoshechkin force-pushed the vbr/moderating-models branch from 719eb00 to 954588e Compare July 5, 2025 00:13

Ololoshechkin requested a review from sdubov July 5, 2025 00:13

Ololoshechkin changed the title ~~Support content moderation requests and moderating models (for Ollama and OpenAI)~~ #396: Support content moderation requests and moderating models (for Ollama and OpenAI) Jul 5, 2025

Ololoshechkin changed the title ~~#396: Support content moderation requests and moderating models (for Ollama and OpenAI)~~ Support content moderation requests and moderating models (for Ollama and OpenAI) Jul 5, 2025

ptitjes reviewed Jul 5, 2025

View reviewed changes

...rc/commonTest/kotlin/ai/koog/agents/core/dsl/extension/AIAgentNodesHistoryCompressionTest.kt Outdated Show resolved Hide resolved

Ololoshechkin requested a review from ptitjes July 5, 2025 18:54

aozherelyeva approved these changes Jul 7, 2025

View reviewed changes

tiginamaria requested review from tiginamaria and removed request for ptitjes July 8, 2025 13:19

sdubov requested changes Jul 10, 2025

View reviewed changes

ptitjes reviewed Jul 11, 2025

View reviewed changes

prompt/prompt-model/src/commonMain/kotlin/ai/koog/prompt/dsl/ModerationAPI.kt Show resolved Hide resolved

EugeneTheDev requested changes Jul 11, 2025

View reviewed changes

EugeneTheDev approved these changes Jul 11, 2025

View reviewed changes

Ololoshechkin force-pushed the vbr/moderating-models branch from 9e7bdb5 to fce56cd Compare July 12, 2025 14:36

Ololoshechkin added 8 commits July 12, 2025 16:37

Implement Moderation API

f83d8c5

fixup! Fix MockLLMBuilder

9781dd6

Add simple integration tests for text moderation

3ec3f82

fixup! Cosmetic change

a4628dd

Add OpenTelemetry support for moderation responses

32da33b

fixup! Revert experimental non-graph API from AIAgentNodesHistoryComp…

5372881

…ressionTest

Fix review comments

fe609c2

fixup! unused imports

4f249e1

Ololoshechkin force-pushed the vbr/moderating-models branch from fce56cd to 4f249e1 Compare July 12, 2025 14:41

Ololoshechkin requested a review from sdubov July 12, 2025 14:41

Provide moderation nodes and write an example

3fafc34

Fix compilation

0f71d9b

Ololoshechkin changed the title ~~Support content moderation requests and moderating models (for Ollama and OpenAI)~~ Support content moderation requests and moderating models (for Ollama, OpenAI, and Bedrock) Jul 12, 2025

Ololoshechkin added 4 commits July 12, 2025 17:38

fixup! test fix

447d1f8

fixup! fix 2 more mock executors

fb24c15

fixup! fix Bedrock test

a667e64

fixup! fix event handler test

fd88932

Ololoshechkin merged commit f1c1e32 into develop Jul 12, 2025
4 checks passed

Ololoshechkin deleted the vbr/moderating-models branch July 12, 2025 22:17

EugeneTheDev mentioned this pull request Jul 13, 2025

Fix custom moderating model selection in graph and single-message moderation #435

Merged

Support content moderation requests and moderating models (for Ollama, OpenAI, and Bedrock) #395

Support content moderation requests and moderating models (for Ollama, OpenAI, and Bedrock) #395

Uh oh!

Conversation

Ololoshechkin commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Type of the change

Checklist for all pull requests

Additional steps for pull requests adding a new feature

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdubov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdubov Jul 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdubov commented Jul 10, 2025

Uh oh!

Uh oh!

EugeneTheDev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Ololoshechkin commented Jul 11, 2025

Uh oh!

Uh oh!

github-actions bot commented Jul 12, 2025

Qodana for JVM

Uh oh!

Uh oh!

Ololoshechkin commented Jul 4, 2025 •

edited

Loading