Skip to content

Fix Dataflow deferrable trigger handling of transient 503#66293

Merged
shahar1 merged 3 commits intoapache:mainfrom
nailo2c:fix-57359-google_dataflow_add_retry_when_503
May 5, 2026
Merged

Fix Dataflow deferrable trigger handling of transient 503#66293
shahar1 merged 3 commits intoapache:mainfrom
nailo2c:fix-57359-google_dataflow_add_retry_when_503

Conversation

@nailo2c
Copy link
Copy Markdown
Contributor

@nailo2c nailo2c commented May 3, 2026

closes: #57359

How to reproduce

mock_57359

Using test_run_continues_polling_after_retryable_service_unavailable

  1. template_job_start_trigger.run() calls get_job_status(), which is mocked to raise ServiceUnavailable for a transient Dataflow API 503.
  2. Before this fix, the trigger yielded an error TriggerEvent immediately instead of sleeping and polling again.

How to fix

Catch ServiceUnavailable inside the Dataflow job-status polling loop, sleep for poll_sleep, and continue polling.

Tests

uv run --project providers/google pytest providers/google/tests/unit/google/cloud/triggers/test_dataflow.py -xvs




Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    Generated-by: Codex 5.5 following the guidelines

@nailo2c nailo2c requested a review from shahar1 as a code owner May 3, 2026 01:50
@boring-cyborg boring-cyborg Bot added area:providers provider:google Google (including GCP) related issues labels May 3, 2026
@eladkal
Copy link
Copy Markdown
Contributor

eladkal commented May 5, 2026

cc @VladaZakharova can you review?

Copy link
Copy Markdown
Contributor

@shahar1 shahar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!

@shahar1 shahar1 merged commit ece8ebf into apache:main May 5, 2026
94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Google Dataflow provider does not retry on service 503 errors

3 participants