-
-
Notifications
You must be signed in to change notification settings - Fork 56
Description
Is your feature request related to a problem? Please describe.
I am using @wdio/visual-service for visual regression testing.
While trying to integrate visual diff results into Allure reports, I couldn't find a straightforward way to attach the baseline, actual, and diff images to the test results.
There doesn’t seem to be a public API or built-in support today to easily hook into the visual diff results and attach them to the Allure reporter.
Describe the solution you'd like
I would like to see it be seem less similar to how test and screenshot per tests work
Describe alternatives you've considered
Right now, I’m manually parsing the output.json file generated by the visual service.
From there, I extract the paths to the baseline, actual, and diff images, and manually attach them to Allure like this:
// Read the output.json file
const outputFilePath = path.join(process.cwd(), 'tmp', 'actual', 'output.json');
const outputData = JSON.parse(fs.readFileSync(outputFilePath, 'utf-8'));
// Find the relevant test data
const testData = outputData.find(entry => entry.data.some(t => t.test === test.title));
if (testData) {
const testDetails = testData.data.find(t => t.test === test.title);
// Loop through all tags for the test
testDetails.data.forEach(tagData => {
const { actualFilePath, baselineFilePath, diffFilePath } = tagData.fileData;
if (actualFilePath && baselineFilePath) {
const expected = fs.readFileSync(baselineFilePath, 'base64');
const actual = fs.readFileSync(actualFilePath, 'base64');
const diff = diffFilePath ? fs.readFileSync(diffFilePath, 'base64') : null;
// Wrap in a JSON, encode as string
const content = JSON.stringify({
expected: `data:image/png;base64,${expected}`,
actual: `data:image/png;base64,${actual}`,
diff: diff ? `data:image/png;base64,${diff}` : null,
});
// Attach to the test report
allureReporter.addAttachment('Visual regression result', content, 'application/vnd.allure.image.diff');
} else {
console.warn(`Missing file paths for tag: ${tagData.tag}`);
}
});
}
}
Question
Is there an easier or more official way today to attach visual regression results from @wdio/visual-service to the Allure test reporter?
If not, would this be something better solved on the WebdriverIO side (visual service API) or on the Allure reporter side (supporting attachments for visual diffs automatically)?
Additional context
The goal is to automatically capture and attach visual diff artifacts (baseline, actual, diff) to each test result without requiring manual work inside each test file.
This is what I was able to achieve with the my workaround
https://github.com/user-attachments/assets/2a1ea296-4bc4-4c25-9185-6c11b77b3204