chore(ci): e2e report improve#2355
Open
universal-itengineer wants to merge 10 commits into
Open
Conversation
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Capture Ginkgo output as a fallback report source so BeforeSuite failures are surfaced even when the JSON report is missing. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
nevermarine
requested changes
May 15, 2026
- Merge Ginkgo JSON and output parsers behind a single parseGinkgoFile helper with declarative source descriptors. - Build a single suiteNodePattern and extract small predicates for the Ginkgo stdout reason block. - Show the suite node name (e.g. SynchronizedAfterSuite) as the failed group in the messenger thread instead of "Unknown". Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The previous suite node regex was too permissive: it matched a plain "[SynchronizedBeforeSuite]" section header that appears in Ginkgo stdout before the actual "[FAILED]" line. As a result, the log-only fallback could not extract suite setup failures from real CI output. Tighten the pattern to require either "[X] [FAILED]" (body) or "[FAIL] [X]" (summary) and add a regression test based on a real BeforeSuite timeout from CI. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
- Describe the test-results table as a single columns array with header, alignment, and value functions so the "Errors" column is just one entry that is appended conditionally, instead of duplicating the row template for the with- and without-errors variants. - Move getFailedTestEntries inside summarizeFailedTestGroups so the caller passes a single report argument instead of pre-extracting entries on every call site. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Document the new helpers introduced by recent refactors: - cluster-report.js: findGinkgoOutput, emptyParsedReport, parseGinkgoFile plus a GinkgoSourceDescriptor typedef used by ginkgoJsonSource and ginkgoOutputSource. - shared/ginkgo-report-utils.js: findFailedSuiteNode, extractFailureReasonFromOutput, parseGinkgoOutput. - messenger/markdown.js: buildTestResultsColumns, buildMarkdownRow, summarizeFailedTestGroups plus a TestResultsColumn typedef. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Improve the E2E messenger report generated by CI:
Reasoncolumn to the failed-tests thread table;Test grouptoTests;Errorscolumn in the main test-results table when there are no Ginkgo errors;Success Ratefrom executed specs only:passed / (passed + failed + errors);Failure.Messageas the failed-test reason;%wso Ginkgo does not dump*exec.ExitErrorinternals into the report;SynchronizedBeforeSuitewhen the JSON report is missing or contains only setup failure data.Why do we need it, and what problem does it solve?
The current E2E notification thread only lists failed test groups and does not show why a test failed. This forces engineers to open CI artifacts/logs even for simple failures.
The report also displayed skipped specs as part of the success-rate denominator, which made Ginkgo success rate look worse than the actual executed-spec result. In addition, SSH command failures could include noisy Go internals in
Failure.Messagebecause*exec.ExitErrorwas wrapped with%w.When E2E fails in
SynchronizedBeforeSuite, Ginkgo may skip all specs and the JSON report may be absent or contain no regular test failures. Previously the messenger report could only show a generic storage failure, for examplenfswithE2E TEST FAILED, without the actual setup failure reason.This change makes E2E notifications more actionable directly in the messenger thread and keeps the main summary easier to read.
What is the expected result?
Errorscolumn is hidden when all error counts are zero;Success Rateis calculated without skipped specs;TestsandReasoncolumns;Reasoncontains the GinkgoFailure.Messagetext.*exec.ExitErrorinternals.SynchronizedBeforeSuiteand no JSON report is created, the report is still generated from the captured Ginkgo output and the failed-tests thread shows the suite-level failure reason. Suite-level failures do not change spec metrics such as passed, failed, errors, or total.Checklist
Changelog entries