Skip to main content
Unlisted page
This page is unlisted. Search engines will not index it, and only users having a direct link can access it.

Investigate E2E test failures from GitHub Actions run 25572054507

GitHub Actions run: 25572054507

Run details​

  • Workflow: E2E Tests
  • Trigger: push on May 8, 2026 at 18:19 UTC
  • Branch: main
  • Commit: e5c2da1
  • Status: failure
  • Duration: 23m 51s
  • Artifacts: 11 Allure HTML artifacts were produced, including browser/grid/mobile/API/database reports.

Public findings​

Public GitHub Actions metadata shows these failing jobs:

JobResult from public annotation
Ubuntu_Firefox_Grid4 failed, 4 broken out of 1029 tests
Ubuntu_Chrome_Grid4 failed, 4 broken out of 1030 tests
Ubuntu_MicrosoftEdge_Grid4 failed, 4 broken out of 1032 tests
Android_Native_BrowserStack2 failed, 0 broken out of 61 tests
MacOSX_Safari_BrowserStack1 failed, 0 broken out of 36 tests

The failing jobs failed in the Post-Test Report and Check step after Allure summary enforcement detected failed or broken tests. Maven test execution was allowed to finish so coverage/reporting artifacts could still be generated.

Access limitation​

The exact failing test method names and stack traces are hidden behind authenticated GitHub Actions logs/artifacts. Public metadata exposes only aggregate Allure counts. An unauthenticated artifact ZIP request returned 401 Unauthorized, and an unauthenticated job-log request returned 403 Forbidden.

Implementation started​

This branch adds scripts/ci/extract_allure_failures.py, which parses Allure JSON result/report folders, JSON files, or ZIP archives and emits a Markdown or JSON list of every failed or broken test with the method name, failure reason, top stack-trace frame, and source JSON file.

The post-test-report action now appends a Failed and broken test methods section to the GitHub job summary whenever Allure summary counts show failed or broken tests and raw allure-results are available. This makes future runs self-triaging without requiring a manual Allure HTML download just to identify failing methods.

Plan​

  1. Download the five failing Allure artifacts from run 25572054507 with an authenticated GitHub session.
  2. Run scripts/ci/extract_allure_failures.py against the downloaded artifacts or raw allure-results folders.
  3. Group repeated grid failures across Chrome, Firefox, and Edge.
  4. Triage Android native BrowserStack failures separately.
  5. Triage the isolated macOS Safari BrowserStack failure separately.
  6. Fix shared grid failures first.
  7. Fix mobile/Safari-specific failures next.
  8. Rerun the affected CI jobs.
  9. Confirm Allure reports show failed = 0 and broken = 0.
  10. Confirm JaCoCo coverage is still generated and uploaded.