Skip to content

[AI Evaluation] Add versioning to reporting and caching #6070

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Mar 13, 2025
Merged

[AI Evaluation] Add versioning to reporting and caching #6070

merged 10 commits into from
Mar 13, 2025

Conversation

peterwald
Copy link
Member

@peterwald peterwald commented Mar 10, 2025 •
edited by dotnet-policy-service bot
Loading

These are basically independent escape mechanisms to handle future changes in formats.

ReportFormatVersion - Will record which version of the format was used to persist the scenario run, as well as exposing which version the current library will write. This should allow us to convert/adapt to reporting format changes in the future if needed.

CacheVersion - Will change all the caching keys causing the existing cache entries to be no longer valid. @stephentoub @SteveSandersonMS

Hopefully neither of these will ever be needed.

Microsoft Reviewers: Open in CodeFlow

@peterwald peterwald added area-ai Microsoft.Extensions.AI libraries area-ai-eval Microsoft.Extensions.AI.Evaluation and related labels Mar 10, 2025
@peterwald peterwald self-assigned this Mar 10, 2025
@peterwald peterwald requested review from a team as code owners March 10, 2025 20:34
@RussKie RussKie removed the area-ai Microsoft.Extensions.AI libraries label Mar 10, 2025
@peterwald
Copy link
Member Author

@dotnet/dotnet-extensions-ai This change is ready to merge if you approve.

peterwald and others added 2 commits March 11, 2025 18:29
@peterwald
Copy link
Member Author

Updated based on your feedback @shyamnamboodiripad @stephentoub

@peterwald peterwald merged commit e393f33 into main Mar 13, 2025
6 checks passed
@peterwald peterwald deleted the pewaldsc/versioning branch March 13, 2025 18:29
@github-actions github-actions bot locked and limited conversation to collaborators Apr 13, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@eiriktsarpalis eiriktsarpalis eiriktsarpalis left review comments

@shyamnamboodiripad shyamnamboodiripad shyamnamboodiripad approved these changes

@stephentoub stephentoub stephentoub approved these changes

+1 more reviewer

@RussKie RussKie RussKie left review comments

Reviewers whose approvals may not affect merge requirements
Assignees

@peterwald peterwald

Labels
area-ai-eval Microsoft.Extensions.AI.Evaluation and related
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@peterwald @stephentoub @eiriktsarpalis @RussKie @shyamnamboodiripad