feat(multi-lingual-support) Added extra sanitasation on translated tags #113
+82
−45
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reviewer's GuideExtend dataset parsing to include extra sanitation for multilingual tags by sanitizing per-language tag lists, selecting default-language tags for the main ‘tags’ field, and ensuring all tags conform to CKAN length rules. File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey there - I've reviewed your changes and they look great!
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location> `ckanext/fairdatapoint/profiles.py:90-97` </location>
<code_context>
+ cleaned = validate_tags(tag_dicts)
+ sanitized[lang] = [tag['name'] for tag in cleaned]
+
+ if len(values) != len(sanitized[lang]):
+ log.warning(
+ 'Removed invalid tags for language %s during multilingual sanitation',
</code_context>
<issue_to_address>
**suggestion:** Log message may be too generic for debugging.
Consider updating the log to include details about the removed tags or original values for improved traceability.
```suggestion
cleaned = validate_tags(tag_dicts)
sanitized[lang] = [tag['name'] for tag in cleaned]
if len(values) != len(sanitized[lang]):
removed_tags = [v for v in values if v not in sanitized[lang]]
log.warning(
'Removed invalid tags for language %s during multilingual sanitation. Original: %r, Removed: %r',
lang,
values,
removed_tags
)
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+90
to
+97
| cleaned = validate_tags(tag_dicts) | ||
| sanitized[lang] = [tag['name'] for tag in cleaned] | ||
|
|
||
| if len(values) != len(sanitized[lang]): | ||
| log.warning( | ||
| 'Removed invalid tags for language %s during multilingual sanitation', | ||
| lang | ||
| ) |
There was a problem hiding this comment.
suggestion: Log message may be too generic for debugging.
Consider updating the log to include details about the removed tags or original values for improved traceability.
Suggested change
| cleaned = validate_tags(tag_dicts) | |
| sanitized[lang] = [tag['name'] for tag in cleaned] | |
| if len(values) != len(sanitized[lang]): | |
| log.warning( | |
| 'Removed invalid tags for language %s during multilingual sanitation', | |
| lang | |
| ) | |
| cleaned = validate_tags(tag_dicts) | |
| sanitized[lang] = [tag['name'] for tag in cleaned] | |
| if len(values) != len(sanitized[lang]): | |
| removed_tags = [v for v in values if v not in sanitized[lang]] | |
| log.warning( | |
| 'Removed invalid tags for language %s during multilingual sanitation. Original: %r, Removed: %r', | |
| lang, | |
| values, | |
| removed_tags | |
| ) |
added 2 commits
September 26, 2025 11:05
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🚀 Pull Request Checklist
Title:
[ ]A brief, descriptive title for the changes.Description:
[ ]Provide a clear and concise description of your pull request, including the purpose of the changes and the approach you've taken.Context:
[ ]Why are these changes necessary? What problem do they solve? Link any related issues.Changes:
[ ]List the major changes you've made, ideally organized by commit or feature.Testing:
[ ]Describe how the changes have been tested. Include any relevant details about the testing environment and the test cases.Screenshots (if applicable):
[ ]If your changes are visual, include screenshots to help explain your changes.Additional Information:
[ ]Add any other information that might be useful for reviewers, such as considerations, discussions, or dependencies.Checklist:
[ ]I have checked that my code adheres to the project's style guidelines and that my code is well-commented.[ ]I have performed self-review of my own code and corrected any misspellings.[ ]I have made corresponding changes to the documentation (if applicable).[ ]My changes generate no new warnings or errors.[ ]I have added tests that prove my fix is effective or that my feature works.[ ]New and existing unit tests pass locally with my changes.Summary by Sourcery
Improve multilingual tag handling by sanitizing translated tags and deriving primary tags from default language translations with fallback support
New Features:
Enhancements: