feat: add evaluate_flags() API for single-call flag evaluation

[dmarticus]

Problem

Phase 1 + Phase 2 of the Server SDK Feature Flag Evaluations RFC for posthog-python. Companion to the Node SDK PR (PostHog/posthog-js#3476).

Today every flag check fires its own /flags request, and capture(send_feature_flags=True) silently fires yet another on every captured event. The flag values on a captured event can diverge from the ones the code actually branched on when person/group properties differ between calls. send_feature_flags also attaches every evaluated flag to every event, which bloats properties on high-volume events.

Changes

New API (Phase 1)

posthog.evaluate_flags(distinct_id, ...) returns a FeatureFlagEvaluations snapshot:

flags = posthog.evaluate_flags(distinct_id, person_properties={"plan": "enterprise"})
if flags.is_enabled("new-dashboard"):
    render_new_dashboard()
posthog.capture("page_viewed", distinct_id=distinct_id, flags=flags)

A single /flags request powers both branching and event enrichment. is_enabled() and get_flag() fire $feature_flag_called events (deduped through the existing cache) with the full metadata — $feature_flag_id, $feature_flag_version, $feature_flag_reason, $feature_flag_request_id — so experiment exposure tracking keeps working.

Two layers of scoping

• Network-level (flag_keys option): scopes the underlying /flags request itself.

  flags = posthog.evaluate_flags(distinct_id, flag_keys=["new-dashboard", "checkout-flow"])

• Event-level (filter helpers): narrow which flags get attached to a captured event without re-fetching.

  posthog.capture(..., flags=flags.only_accessed())           # only flags the developer checked
  posthog.capture(..., flags=flags.only(["new-dashboard"]))   # specific keys

Deprecation warnings (Phase 2)

The legacy single-flag surface keeps working but now emits DeprecationWarnings pointing at evaluate_flags():

• feature_enabled()
• get_feature_flag()
• get_feature_flag_payload()
• capture(send_feature_flags=...) (only when truthy)

feature_enabled and get_feature_flag are restructured to call _get_feature_flag_result directly instead of routing through each other, so a single user-level call emits exactly one warning instead of cascading.

Phase 3 (removal in next major) ships separately.

Local evaluation

Transparent. When the poller resolves a flag, the snapshot carries locally_evaluated=True and reason "Evaluated locally", matching what get_feature_flag() emits today.

Backwards compatibility

No breaking changes. All existing call paths return the same values they did before — the only behavior change is the new DeprecationWarning emissions, which can be silenced via Python's standard warnings filter.

Internals

_capture_feature_flag_called was refactored: the dedup + capture portion is extracted into _capture_feature_flag_called_if_needed, which is shared between the single-flag path and the new FeatureFlagEvaluations object. Both paths now dedupe identically.

Response-level errors (errors_while_computing_flags, quota_limited) are propagated into $feature_flag_called events from the snapshot, matching the granularity of the single-flag path.

Tests

posthog/test/test_evaluate_flags.py — 27 tests covering remote evaluation, local evaluation, filtering helpers, capture integration, flag_keys round-trip, empty-distinct_id safety, error-granularity propagation, and deprecation warning emission (with no-cascade verification).

Full suite: 489 passed. ruff format and ruff check clean.

---

Created with PostHog Code

[github-actions]

posthog-python Compliance Report

Date: 2026-05-01 19:53:32 UTC
Duration: 160018ms

⚠️ Some Tests Failed

29/30 tests passed, 1 failed

---

Capture Tests

✅ 29/29 tests passed

View Details

Test	Status	Duration
Format Validation.Event Has Required Fields	✅	517ms
Format Validation.Event Has Uuid	✅	1507ms
Format Validation.Event Has Lib Properties	✅	1507ms
Format Validation.Distinct Id Is String	✅	1506ms
Format Validation.Token Is Present	✅	1507ms
Format Validation.Custom Properties Preserved	✅	1507ms
Format Validation.Event Has Timestamp	✅	1506ms
Retry Behavior.Retries On 503	✅	9519ms
Retry Behavior.Does Not Retry On 400	✅	3505ms
Retry Behavior.Does Not Retry On 401	✅	3507ms
Retry Behavior.Respects Retry After Header	✅	9513ms
Retry Behavior.Implements Backoff	✅	23530ms
Retry Behavior.Retries On 500	✅	7505ms
Retry Behavior.Retries On 502	✅	7507ms
Retry Behavior.Retries On 504	✅	7515ms
Retry Behavior.Max Retries Respected	✅	23525ms
Deduplication.Generates Unique Uuids	✅	1501ms
Deduplication.Preserves Uuid On Retry	✅	7510ms
Deduplication.Preserves Uuid And Timestamp On Retry	✅	14525ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7508ms
Deduplication.No Duplicate Events In Batch	✅	1503ms
Deduplication.Different Events Have Different Uuids	✅	1507ms
Compression.Sends Gzip When Enabled	✅	1507ms
Batch Format.Uses Proper Batch Structure	✅	1507ms
Batch Format.Flush With No Events Sends Nothing	✅	1004ms
Batch Format.Multiple Events Batched Together	✅	1506ms
Error Handling.Does Not Retry On 403	✅	3508ms
Error Handling.Does Not Retry On 413	✅	3507ms
Error Handling.Retries On 408	✅	7514ms

Feature_Flags Tests

⚠️ 0/1 tests passed, 1 failed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	❌	514ms

Failures

request_payload.request_with_person_properties_device_id

Field 'token' not found in /flags request body at path 'token'. Available keys: ['distinct_id', 'groups', 'person_properties', 'group_properties', 'geoip_disable', 'device_id', 'flag_keys_to_evaluate', 'sentAt', 'api_key']

[greptile-apps]

Prompt To Fix All With AI

This is a comment left during a code review.
Path: posthog/test/test_evaluate_flags.py
Line: 42-44

Comment:
**Missing `tearDown` leaks Client background threads**

`TestEvaluateFlagsRemote` creates a `Client` in `setUp` but never calls `self.client.shutdown()`. The other two test classes in this file (`TestEvaluateFlagsFiltering`, `TestCaptureWithFlagsSnapshot`) both have correct `tearDown` implementations. Without shutdown, the client's background consumer thread is never joined, which can cause test-suite noise, delayed process exit, or flaky behaviour when mocks from one test bleed into the next.

```suggestion
class TestEvaluateFlagsRemote(unittest.TestCase):
    def setUp(self):
        self.client = Client(FAKE_TEST_API_KEY)

    def tearDown(self):
        self.client.shutdown()
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: posthog/feature_flag_evaluations.py
Line: 69-80

Comment:
**`flag_definitions_loaded_at` is accepted but never supplied**

`_record_access` (line 242) emits `$feature_flag_definitions_loaded_at` only when `self._flag_definitions_loaded_at is not None`, but `Client.evaluate_flags()` never passes this argument to the `FeatureFlagEvaluations` constructor — so the property is always `None` and the event property is never emitted for locally-evaluated flags. Either wire up the value from the poller or remove the dead parameter and the guarded block in `_record_access` until it's ready.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: posthog/test/test_evaluate_flags.py
Line: 42-321

Comment:
**Prefer parameterised tests**

The test suite exercises multiple flag types (boolean, variant, disabled, missing) through per-assertion branches inside single test methods — e.g. `test_is_enabled_returns_correct_values_and_fires_events` and `test_get_flag_returns_variant_or_bool_with_full_metadata`. The project's review standard is to prefer parameterised tests. Using `subTest` (or a library like `ddt`) would give each flag-type scenario its own named, independently-failing case and reduce the assertion density per method.

**Context Used:** Do not attempt to comment on incorrect alphabetica... ([source](https://app.greptile.com/review/custom-context?memory=instruction-0))

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (1): Last reviewed commit: "revert: drop deprecation warnings (Phase..." | Re-trigger Greptile}

[dustinbyrne]

Looks good! I have a few questions/considerations

[dustinbyrne]

If this is a legitimate case of nothing being accessed, there's no way to avoid getting all flags. This could be more confusing than getting nothing?

E.g., I'm imaging a case where flags are first retrieved and consumed as needed. Should an early call to capture(flags=flags.only_accessed()) include all flags? I think it's contradictory with the name only_accessed

[dmarticus]

Good catch — fixed in 95eb1e9. only_accessed() now honors its name and returns an empty snapshot when nothing has been accessed (no fallback, no warning). The previous behavior was a misguided safety net that ended up being more surprising than helpful for exactly the scenario you describe (early capture() before any branching). The Node-side equivalent has the same change queued.

Updated test: test_only_accessed_returns_empty_when_no_flags_accessed.

[dustinbyrne]

Should we deprecate the other methods?

[dmarticus]

Reversed course on this — shipped Phase 2 in eda573d alongside Phase 1 in this PR (rather than splitting it into a follow-up).

feature_enabled, get_feature_flag, get_feature_flag_payload, and capture(send_feature_flags=...) now emit DeprecationWarnings pointing at evaluate_flags(). The methods continue to return the same values; users who pin warnings to errors will get a heads-up on first use, and the rest see them surface via pytest / python -W / IDE inspectors.

feature_enabled and get_feature_flag were restructured to call _get_feature_flag_result directly so a single user-level call emits exactly one warning instead of cascading three.

Phase 3 (removal in next major) is separate.

[dustinbyrne]

We should pass device_id here, as it may be present in the context (via tracing headers).

I don't think it's necessary to add it as a parameter of evaluate_flags as well, but I'll leave it up to you.

[dmarticus]

Good catch — wired up in 95eb1e9. evaluate_flags() now resolves device_id from context (get_context_device_id()) at the top of the method, then forwards it to the get_flags_decision(...) call so it lands in the /flags request body. I went with your suggestion and didn't add it as a method parameter — context-only is cleaner and matches the existing distinct_id resolution pattern.

[dustinbyrne]

Just noting that we're losing granularity of $feature_flag_error values here

[dmarticus]

You're right — fixed in 95eb1e9. The snapshot now tracks response-level errors (errors_while_computing_flags, quota_limited) at construction and _record_access builds a comma-joined $feature_flag_error matching the single-flag path's granularity. So a missing flag during a quota-limited response now reports quota_limited,flag_missing instead of just flag_missing.

New test: test_errors_while_computing_flags_propagates_to_event covers both the standalone (errors_while_computing_flags) and combined (errors_while_computing_flags,flag_missing) cases.

Not covered yet: TIMEOUT/CONNECTION_ERROR/api_error_NNN. Those manifest as exceptions on the request and currently just log + leave the snapshot empty (so flags would surface as flag_missing). Could fold those into the snapshot in a follow-up if useful, but the most common cases (errors_while_computing and quota_limited) are now covered.

[marandaneto]

how does auto captured events and errors set flags to those events if you need the instance from evaluate_flags

[marandaneto]

No breaking changes. Existing feature_enabled, get_feature_flag, get_feature_flag_payload, and capture(send_feature_flags=...) paths all continue to work unchanged. Phase 2 (runtime DeprecationWarnings pointing at evaluate_flags) and Phase 3 (removal in next major) ship in follow-up PRs — this minor is API-additive only so users have a quiet release to migrate before warnings start.

i'd say we should deprecate and add a warning now otherwise theres multiple ways of doing the same thing and users and agents will be very confused
theres an RFC about that already

[dmarticus]

how does auto captured events and errors set flags to those events if you need the instance from evaluate_flags

Good question — the answer has two layers.

Manual error captures: capture_exception(exception, ..., flags=flags) now forwards the snapshot through to the inner capture() call so the $exception event carries the same flag context as your other events. This was a small gap I missed — fixed in db6d3c0:

flags = posthog.evaluate_flags(distinct_id)
try:
    risky_thing()
except Exception as e:
    posthog.capture_exception(e, distinct_id=distinct_id, flags=flags)

The same applies to anywhere the developer is the one calling capture() — they can thread flags= through.

Auto-captured exceptions (the case you're really asking about — sys.excepthook in ExceptionCapture, plus the new_context() block-level exception handler): you're right that there's no surface today for the developer to attach a flag snapshot, because they're not the one calling capture. Two paths I see for a follow-up PR:

1. Stash flags on the context. new_context() already carries distinct_id, session_id, properties — adding flags would mean any exception captured within the block (block-level handler, or even sys.excepthook if the context is active) would automatically attach the snapshot. Most ergonomic for the request-scoped pattern.

2. A "default flags provider" callback on the client that auto-evaluates at capture time. More magic, more risk of latency surprises. Probably not the right answer.

Happy to do (1) as a separate PR — felt out of scope for this one since it touches the context module and needs its own design pass. Worth tracking as a follow-up issue?

[dmarticus]

No breaking changes. Existing feature_enabled, get_feature_flag, get_feature_flag_payload, and capture(send_feature_flags=...) paths all continue to work unchanged. Phase 2 (runtime DeprecationWarnings pointing at evaluate_flags) and Phase 3 (removal in next major) ship in follow-up PRs — this minor is API-additive only so users have a quiet release to migrate before warnings start.

i'd say we should deprecate and add a warning now otherwise theres multiple ways of doing the same thing and users and agents will be very confused theres an RFC about that already

You're right — reversed course on this and shipped Phase 2 in eda573d alongside Phase 1 in this PR. The PR description has been updated.

feature_enabled, get_feature_flag, get_feature_flag_payload, and capture(send_feature_flags=...) now emit DeprecationWarnings pointing at evaluate_flags(). They keep working unchanged in this minor — Phase 3 (removal) lands in the next major.

feature_enabled and get_feature_flag were restructured to call _get_feature_flag_result directly so a single user-level call emits exactly one warning instead of cascading three.

[dustinbyrne]

Let's deprecate get_feature_flag_result as well. It lived a short life, but better to clean it up now.

[dmarticus]

done!

[marandaneto]

@dustinbyrne you'll need to coordinate this with the @PostHog/team-docs-wizard as well otherwise the wizard will be lost or instrumenting new apps with deprecated APIs
we should get the docs here https://posthog.com/docs/references/posthog-python?filter=feature-flags and here https://posthog.com/docs/libraries/python updated and most likely all the other code snippets spread around the codebase
can we also write a skill that we can add to https://github.com/PostHog/skills/tree/main/skills and point out in our docs so users can just run that skill in that codebase for the migration?
this is not a blocker for this PR but ideally somehow synced otherwise we'll just get support tickets I guess

[marandaneto]

I think 'Stash flags on the context' makes sense
if we do this on a follow up PR, does that mean that new exceptions wont have flags context as they did before until we address this issue? or they currently dont have anyway so its not an issue?

[dustinbyrne]

@dustinbyrne you'll need to coordinate this with the @PostHog/team-docs-wizard as well otherwise the wizard will be lost or instrumenting new apps with deprecated APIs

Edwin gave me a short into on how the wizard sources context for these types of things. As I understand it, as long as it's documented on the website, the wizard will automatically start doing the right thing.

See https://github.com/PostHog/context-mill/blob/main/transformation-config/skills/feature-flags/config.yaml

For Python context it sources:

• https://posthog.com/docs/feature-flags/adding-feature-flag-code.md
• https://posthog.com/docs/feature-flags/best-practices.md
• https://posthog.com/docs/feature-flags/installation/python.md

So these are the documents that we will need to update. Same for every other language.

[gewenyu99]

Updating the docs should do it. There isn't a feature flag specific thing that we run during setup, so it should be safe to get this in. There is a best practices skill, and I think this might change some of the feature flag best practices?

[dustinbyrne]

I can spend some time tomorrow updating documentation for this. If you update the other SDK branches to match this spec I'll review those as well! 👍

[dustinbyrne]

surfaced when documenting: we should include a device_id override for parity with the deprecated methods

this might be relevant for other sdks as well?

[dmarticus]

good shout! So most of the other SDKs didn't actually support this yet – it was only this one and JS that supported that operation. I do think we should support device_id in this method going forward for other SDKs, but given that only Python and JS have it now, it's only a concern for those SDKs. Definitely will update JS, though.