Skip to content

Show sample for each Level type on ROBOT report #134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Show sample for each Level type on ROBOT report #134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6342501
get sample of each level type on robot report
anitacaron Oct 10, 2024
32c9dbb
removing the `truncate` step to download the same sample created
anitacaron Oct 10, 2024
d48181b
Merge branch 'master' into anitacaron/issue133
anitacaron Jun 7, 2025
634ede8
Enhance error handling and logging in create_report_html.py
anitacaron Jun 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ clean:
rm -rf build dashboard dependencies

# Truncate potentially huge robot reports
truncate_reports_for_github:
$(eval REPORTS := $(wildcard dashboard/*/robot_report.tsv))
for REP in $(REPORTS); do \
touch $$REP; \
cat $$REP | head -$(REPORT_LENGTH_LIMIT) > $$REP.tmp; \
mv $$REP.tmp $$REP; \
done
# truncate_reports_for_github:
# $(eval REPORTS := $(wildcard dashboard/*/robot_report.tsv))
# for REP in $(REPORTS); do \
# touch $$REP; \
# cat $$REP | head -$(REPORT_LENGTH_LIMIT) > $$REP.tmp; \
# mv $$REP.tmp $$REP; \
# done

# ------------------- #
### DIRECTORY SETUP ###
Expand Down
49 changes: 45 additions & 4 deletions util/create_report_html.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,23 @@

import argparse
import json
import logging
import os
import re
import sys

import pandas as pd
from jinja2 import Template

logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')


def main(args):
"""
"""
parser = argparse.ArgumentParser(description='Create a report HTML page')
parser.add_argument('report',
type=argparse.FileType('r'),
type=argparse.FileType('r+'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Consider using a separate output file instead of overwriting the input.

Opening the report file in 'r+' mode and overwriting it at line 82 is risky. If an error occurs during processing, the original data could be lost.

Consider adding a separate output parameter for the filtered report:

 parser.add_argument('report',
-                    type=argparse.FileType('r+'),
+                    type=argparse.FileType('r'),
                     help='TSV report to convert to HTML')
+parser.add_argument('--output-report',
+                    type=argparse.FileType('w'),
+                    help='Output filtered report TSV file (optional)')

Then modify line 82:

-if len(report_filtered) > args.limitlines:
-    report_filtered.to_csv(args.report, sep="\t", index=False)
+if args.output_report and len(report) > args.limitlines:
+    report_filtered.to_csv(args.output_report, sep="\t", index=False)

Also applies to: 82-82

🤖 Prompt for AI Agents 
In util/create_report_html.py at line 21, the argument parser currently opens
the input report file in 'r+' mode, which risks overwriting the original file if
an error occurs. To fix this, add a new command-line argument for a separate
output file to write the filtered report. Then, at line 82, change the code to
write to this new output file instead of overwriting the input file. This
ensures the original input remains intact in case of processing errors.

help='TSV report to convert to HTML')
parser.add_argument('context',
type=argparse.FileType('r'),
Expand All @@ -38,20 +41,58 @@ def main(args):

error_count_rule = {}
error_count_level = {}
report_filtered = pd.DataFrame()

try:
report = pd.read_csv(args.report, sep="\t")

# Get sample of each level only for ROBOT report
if "Level" in report.columns and "Rule Name" in report.columns:
error_count_level = report["Level"].value_counts()
error_count_rule = report["Rule Name"].value_counts()
except Exception:
print("No report")

error_count_error = error_count_level.get("ERROR", 0)
if error_count_error < args.limitlines:
rest = args.limitlines - error_count_level["ERROR"]

# Calculate the sample number for each level based on group size
def calculate_sample_size(group, rest):
if group["Level"].iloc[0] == "ERROR":
return group.shape[0]

return min(group.shape[0], rest)

Comment on lines +56 to +64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix the sampling logic to properly distribute remaining lines.

The current logic incorrectly uses the full rest value for each non-ERROR group, which could exceed the limit when multiple groups exist.

The sampling should distribute the remaining lines proportionally among non-ERROR levels:

-error_count_error = error_count_level.get("ERROR", 0)
-if error_count_error < args.limitlines:
-    rest = args.limitlines - error_count_level["ERROR"]
-
-    # Calculate the sample number for each level based on group size
-    def calculate_sample_size(group, rest):
-        if group["Level"].iloc[0] == "ERROR":
-            return group.shape[0]
-
-        return min(group.shape[0], rest)
+error_count_error = error_count_level.get("ERROR", 0)
+if error_count_error < args.limitlines:
+    rest = args.limitlines - error_count_error
+    
+    # Calculate proportional sample sizes for non-ERROR levels
+    non_error_counts = {level: count for level, count in error_count_level.items() if level != "ERROR"}
+    total_non_error = sum(non_error_counts.values())
+    
+    level_sample_sizes = {"ERROR": error_count_error}
+    if total_non_error > 0:
+        for level, count in non_error_counts.items():
+            # Proportionally distribute remaining lines
+            level_sample_sizes[level] = int(rest * count / total_non_error)
+    
+    def calculate_sample_size(group):
+        level = group["Level"].iloc[0]
+        group_key = (level, group["Rule Name"].iloc[0], group["Subject"].iloc[0])
+        # Distribute level sample size among groups of that level
+        level_sample = level_sample_sizes.get(level, 0)
+        # This is a simplified approach - you may want to further distribute among groups
+        return min(group.shape[0], max(1, level_sample))

Also, line 56 still uses direct dictionary access instead of .get():

-rest = args.limitlines - error_count_level["ERROR"]
+rest = args.limitlines - error_count_error

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In util/create_report_html.py around lines 56 to 64, fix the sampling logic by
replacing the direct dictionary access on line 56 with .get() to avoid KeyError,
and update the calculate_sample_size function to distribute the remaining lines
proportionally among non-ERROR groups instead of using the full rest value for
each group. Compute the total size of non-ERROR groups, then allocate sample
sizes based on each group's proportion of that total, ensuring the sum does not
exceed the limit.

required_columns = ["Level", "Rule Name", "Subject", "Property", "Value"]
missing_columns = [col for col in required_columns if col not in report.columns]
if missing_columns:
raise KeyError(f"Missing columns in report: {missing_columns}")

# Get a sample of each Level type
report_filtered = report.groupby(by=["Level","Rule Name","Subject"])[
["Level", "Rule Name", "Subject", "Property", "Value"]
].apply(
lambda x: x.sample(calculate_sample_size(x, rest))
).reset_index(drop=True)
Comment on lines +71 to +75
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Make the sampling more robust against edge cases.

The current implementation could fail if groups are empty or if sampling parameters are invalid.

Consider adding error handling:

 # Get a sample of each Level type
-report_filtered = report.groupby(by=["Level","Rule Name","Subject"])[
-    ["Level", "Rule Name", "Subject", "Property", "Value"]
-].apply(
-    lambda x: x.sample(calculate_sample_size(x, rest))
-).reset_index(drop=True)
+def safe_sample(group, sample_size_func):
+    try:
+        n = sample_size_func(group)
+        if n > 0 and len(group) > 0:
+            return group.sample(min(n, len(group)))
+        return group
+    except Exception as e:
+        logging.warning(f"Sampling failed for group: {e}")
+        return group
+
+report_filtered = report.groupby(by=["Level","Rule Name","Subject"])[
+    ["Level", "Rule Name", "Subject", "Property", "Value"]
+].apply(
+    lambda x: safe_sample(x, calculate_sample_size)
+).reset_index(drop=True)

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In util/create_report_html.py around lines 71 to 75, the current groupby
sampling can fail if groups are empty or sampling parameters are invalid. To fix
this, add error handling inside the lambda function to check if the group is
empty or if the sample size is valid before calling sample. If invalid, return
the group as is or an empty DataFrame to avoid exceptions during sampling.

else:
report_filtered = report.head(args.limitlines)
else:
report_filtered = report.head(args.limitlines)

if len(report_filtered) > args.limitlines:
report_filtered.to_csv(args.report, sep="\t", index=False)
Comment on lines +81 to +82
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Remove or fix the illogical condition.

The condition len(report_filtered) > args.limitlines doesn't make sense after filtering. The filtered report should already be within the limit.

Either remove this block or change the logic to save the filtered report when the original report exceeds the limit:

-if len(report_filtered) > args.limitlines:
-    report_filtered.to_csv(args.report, sep="\t", index=False)
+# Save filtered report if original exceeded limit
+if len(report) > args.limitlines and args.output_report:
+    report_filtered.to_csv(args.output_report, sep="\t", index=False)

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In util/create_report_html.py around lines 81 to 82, the condition checking if
the filtered report length exceeds the limit is illogical since filtering should
already enforce the limit. Fix this by changing the condition to check if the
original report length exceeds args.limitlines before saving the filtered
report, or remove the condition entirely to always save the filtered report.


except pd.errors.EmptyDataError as e:
logging.error("Empty data error: %s", e)
except FileNotFoundError as e:
logging.error("File not found: %s", e)
except Exception as e:
logging.error("An unexpected error occurred: %s", e)

# Load Jinja2 template
template = Template(args.template.read())

# Generate the HTML output
res = template.render(contents=report.head(args.limitlines),
res = template.render(contents=report_filtered.reset_index(drop=True),
maybe_get_link=maybe_get_link,
context=context,
title=args.title,
Expand Down
4 changes: 2 additions & 2 deletions util/dashboard_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,8 @@ def rundashboard(configfile, clean):
prepare_ontologies(ontologies['ontologies'], ontology_dir, dashboard_dir, make_parameters, config)
logging.info("Building the dashboard")
runcmd(f"make dashboard {make_parameters} -B", config.get_dashboard_report_timeout_seconds())
logging.info("Postprocess files for github")
runcmd(f"make truncate_reports_for_github {make_parameters} -B", config.get_dashboard_report_timeout_seconds())
# logging.info("Postprocess files for github")
# runcmd(f"make truncate_reports_for_github {make_parameters} -B", config.get_dashboard_report_timeout_seconds())

info_usage_namespace = 'Info: Usage of namespaces in axioms'

Expand Down