Skip to content

Conversation

Kashif-E
Copy link
Contributor

This pull request introduces significant enhancements to the OllamaClient and its associated components, enabling dynamic model management and improving the flexibility of interacting with the Ollama API. The most notable changes include adding support for dynamic model resolution, introducing a new OllamaModelFactory for creating models, and implementing a OllamaModelManager for managing model discovery and capabilities. These changes collectively enhance the modularity and extensibility of the Ollama client.

Enhancements to OllamaClient:

  • Added support for dynamic model resolution via the enableDynamicModels property, allowing models to be resolved dynamically using the OllamaModelResolver. This impacts methods like execute, stream, and embed, which now use dynamically resolved model names.
  • Introduced new methods such as getAvailableModels, resolveModel, createDynamicModel, pullModel, and suggestSimilarModels, providing extended functionality for model management.
  • Fixed tool call ID generation to use deterministic IDs instead of hardcoded "id" values.

Introduction of OllamaModelFactory:

  • Added a new OllamaModelFactory class to create and manage Ollama models dynamically. This includes methods for creating models from names, listing available models, grouping models by family, and finding models by patterns.

Introduction of OllamaModelManager:

  • Implemented the OllamaModelManager class to handle model discovery, caching, and capability detection. It includes features like retrieving available models, resolving models, pulling models from the registry, and invalidating the model cache.

Code Examples

Basic Agent with Dynamic Model Selection

fun main() = runBlocking {
    val client = OllamaClient()
    val model = client.modelFromName("llama3.2:3b") // Works with any available model
    
    val agent = simpleSingleRunAgent(
        executor = simpleOllamaAIExecutor(),
        llmModel = model,
        systemPrompt = "You are a helpful assistant."
    )
    
    val result = agent.runAndGetResult("What is Kotlin?")
    println(result)
}

Model Discovery and Management

suspend fun exploreAvailableModels() {
    val client = OllamaClient()
    
    // List all available models
    val models = client.getAvailableModels()
    println("Available models: ${models.map { it.name }}")
    
    // Group models by family
    val groupedModels = client.listModelsGrouped()
    groupedModels.forEach { (family, modelList) ->
        println("$family: ${modelList.joinToString(", ")}")
    }
    
    // Get best model variant for system constraints
    val bestModel = client.getBestModelVariant("llama3.2", availableMemoryGB = 8)
    println("Best model for 8GB RAM: ${bestModel?.id}")
}

Automatic Model Creation with Fallbacks

suspend fun createModelWithFallback(preferredModel: String): LLModel {
    val client = OllamaClient()
    
    return try {
        client.modelFromName(preferredModel)
    } catch (e: Exception) {
        val suggestions = client.suggestSimilarModels(preferredModel, limit = 3)
        println("Model '$preferredModel' not available. Similar: ${suggestions.joinToString(", ")}")
        client.modelFromName(suggestions.firstOrNull() ?: "llama3.2")
    }
}

Temperature Control with Dynamic Models

suspend fun executeWithTemperature() {
    val client = OllamaClient()
    val model = client.modelFromName("qwen2.5-coder:7b")
    
    val agent = simpleSingleRunAgent(
        executor = simpleOllamaAIExecutor(),
        llmModel = model,
        systemPrompt = "You are a coding assistant."
    )
    
    val prompt = prompt {
        user("Write a Kotlin function to reverse a string")
        temperature = 0.3 // Low temperature for precise code generation
    }
    
    val result = agent.runAndGetResult(prompt)
    println(result)
}

These changes lay the groundwork for a more robust and flexible integration with the Ollama API, enabling dynamic and efficient model management without maintaining hardcoded model lists.

fixes #140


Type of the change

  • New feature
  • Bug fix
  • [] Documentation fix

Checklist for all pull requests

Additional steps for pull requests adding a new feature
  • An issue describing the proposed change exists
  • The pull request includes a link to the issue
  • [] The change was discussed and approved in the issue
  • [] Docs have been added/updated

@aozherelyeva aozherelyeva requested a review from Rizzen May 23, 2025 10:10
@ptitjes ptitjes mentioned this pull request May 23, 2025
12 tasks
Copy link
Member

@Rizzen Rizzen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @Kashif-E! Thank you for your contribution!

@Rizzen Rizzen merged commit 0b998db into JetBrains:develop May 23, 2025
3 checks passed
Copy link
Contributor

@ptitjes ptitjes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's far better than what I did in my PR.

In my opinion, we have to use the real model name advertised by Ollama, without any prefix added.

@ptitjes
Copy link
Contributor

ptitjes commented May 23, 2025

@Rizzen wow, you merged really fast. Can I do a PR with the changes that I propose above?

@Rizzen
Copy link
Member

Rizzen commented May 23, 2025

@ptitjes Yeah, I merged it pretty fast because we're working on actual ollama integration tests right now, and these changes are very useful. I suggest to wait for PR with these tests and then make PR with your proposals. What do you think about it?

@Kashif-E
Copy link
Contributor Author

Thanks @Rizzen for the merge, koog is definitely kool 😉
@ptitjes really excited about your upcoming changes, they are going to simplify things a lot

@Kashif-E
Copy link
Contributor Author

resoling conversations for now

@ptitjes
Copy link
Contributor

ptitjes commented May 23, 2025

@Rizzen sorry, I made the PR anyway. It doesn't change the API introduced by @Kashif-E, it just simplify things.
#149

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Trouble running with ollama
3 participants