Skip to content

tests(appsec): wait for sidecar telemetry ready before TelemetryTests#3816

Draft
Leiyks wants to merge 1 commit intomasterfrom
leiyks/fix-ci-appsec-telemetry-startup
Draft

tests(appsec): wait for sidecar telemetry ready before TelemetryTests#3816
Leiyks wants to merge 1 commit intomasterfrom
leiyks/fix-ci-appsec-telemetry-startup

Conversation

@Leiyks
Copy link
Copy Markdown
Contributor

@Leiyks Leiyks commented Apr 23, 2026

Problem

TelemetryTests share a single Docker container across 9 ordered tests. The @BeforeAll restarts php-fpm, but if the helper-rust sidecar startup is slow, the first test's 30s polling window expires before waf.init telemetry arrives. All 9 tests then fail in sequence (30s apart, perfectly timed out).

Fix

Add a startup health-check in @BeforeAll that waits for waf.init telemetry before releasing the first test, and increase the first test's timeout to 60s (it bears the full startup cost). This removes the race between sidecar startup and the first test's poll window.

…tryTests

Add a startup health-check in @BeforeAll that uses waitForMetrics() to wait
for waf.init telemetry (up to 60s) before releasing any test. This eliminates
the race between a slow helper/sidecar startup and the first test's polling
window, which was causing all 9 ordered tests to time out in sequence.

Also increase the @order(1) test's waitForMetrics timeout from 30s to 60s,
since @BeforeAll may consume the first waf.init batch (10s metric interval)
and the test needs headroom to catch the next emission cycle.
@datadog-datadog-prod-us1
Copy link
Copy Markdown

datadog-datadog-prod-us1 Bot commented Apr 23, 2026

Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 1 Test failed

❄️ Known flaky: telemetry data is received() from com.datadog.appsec.php.integration.TelemetryTests   View in Datadog   (Fix with Cursor)
Assertion failed: 

assert initSpan.metrics.'_dd.appsec.event_rules.loaded' > 0
       |        |       |                               |
       |        |       null                            false
       |        ['process_id':150.0, 'php.memory.peak_usage_bytes':695776.0, 'php.compilation.total_time_ms':107.995, '_dd.agent_psr':1.0, 'php.memory.peak_real_usage_bytes':2097152.0, '_sampling_priority_v1':1.0]
       com.datadog.appsec.php.model.Span@73ae82da

Assertion failed: 

...

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 60.69% (+0.00%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4c558bd | Docs | Datadog PR Page | Give us feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant