Skip to content

[AI Evaluation] Remove CacheOptions from DiskBasedResponseCache #6387

@shyamnamboodiripad

Description

@shyamnamboodiripad

DiskBasedResponseCacheProvider supports an options file that can be used to tweak the default time-to-live for cache entries and a cache mode setting which can be used to completely turn off caching, or to completely block calls to the LLM when there is a cache miss.

AzureStorageResponseCacheProvider does not support similar options file or cache mode functionality - instead it supports tweaking the default time-to-live in code via a constructor parameter.

The cache modes functionality as well as the options file above is pretty specialized and should not be required in normal usage scenarios. We ideally don't want to support this functionality out of the box in the eval library. For advanced usage scenarios that require the special mode above to block LLM calls on cache misses, it should be possible (and relatively easy) to implement a custom IResponseCacheProviders with that behavior.

Logging this issue to remove the options file and modes functionality, and to align the constructors for DiskBasedResponseCacheProvider and AzureStorageResponseCacheProvider such that both support tweaking the default time-to-live for cache entries via code.

Metadata

Metadata

Labels

area-ai-evalMicrosoft.Extensions.AI.Evaluation and related

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions