Incorporate model capabilities into GenAICfEnvProcessor #273
Conversation
* Valid as of GenAI for Tanzu Platform v0.6.0
* Split the GenAICfEnvProcessor into two - one specific to chat and
the other specific to embedding
* This allows a developer to either make one binding to a model that
supports both chat and embeddings, or to make two distinct bindings,
one to a chat model and the other to an embeddings model
* When binding to two different models, each model will have a different
api key, which will be set accordingly on the corresponding
spring.ai.openai.{chat,embedding}.* properties
* Both processors additionally set the top-level property
spring.ai.openai.api-key="redundant" in order to circumvent an error
that is raised when configuring more specific api-keys on the chat and
embedding properties
* If desired, additional GenAI CfEnvProcessors may be written to cater to
image and audio model options at a later time
Signed-off-by: Ed King <[email protected]>
teddyking
force-pushed
the
genai-model-capabilities
branch
from
4ea1af1 to
4fa77fd
Compare
|
cc @svrc |
| ArrayList<String> modelCapabilities = (ArrayList<String>) service.getCredentials().getMap().get("model_capabilities"); | ||
| return modelCapabilities.contains("chat"); | ||
| } else { | ||
| return false; |
There was a problem hiding this comment.
nitpick: I would just return false without the else block
There was a problem hiding this comment.
nitpick: I would use the interface type List instead of the concrete type ArrayList. My java is also rusty though, is there any specific reason that made you use ArrayList? I think at least you should be able to use List in the left hand side
There was a problem hiding this comment.
Thanks @pivotal-marcela-campo, I have removed the redundant "else" blocks. Re: usage of ArrayList, I simply copied from some existing code in the CassandraCfEnvProcessor.
java-cfenv-boot/src/main/java/io/pivotal/cfenv/spring/boot/GenAIEmbeddingCfEnvProcessor.java
Show resolved
Hide resolved
java-cfenv-boot/src/main/java/io/pivotal/cfenv/spring/boot/GenAIEmbeddingCfEnvProcessor.java
Outdated
Show resolved
Hide resolved
|
hello! |
Signed-off-by: Ed King <[email protected]>
|
v0.6.0 of GenAI on Tanzu Platform has been released, so marking this as ready for review. |
|
LGTM! |
About
This PR updates the GenAICfEnvProcessor to incorporate the notion of model capabilities, which will be included in binding credentials from version v0.6.0 of GenAI on Tanzu Platform onwards.
I will leave this PR in draft until v0.6.0 has been officially released.Update: it has now been released.Details
GenAICfEnvProcessorinto two - one specific to chat and the other specific to embeddingspring.ai.openai.{chat,embedding}.*propertiesspring.ai.openai.api-key="redundant"in order to circumvent an error that is raised when configuring more specific api-keys on the chat and embedding properties