fix(managed): carry X-Database-Id on result reads; keep API error bodies#35
Merged
Merged
Conversation
fetch_table submitted the query WITH the database scope (x_database_id)
but then polled the stored result, polled the query run, and fetched the
Arrow body WITHOUT it. Results of database-scoped queries are themselves
database-scoped, so the results endpoints 400 ("X-Database-Id header is
required: this endpoint is scoped to a database") on every read against
an existing synced table — merge/append loads and dlt state restore
failed on any second run into the same database (dlthubworker#70), while
first runs passed because the unsynced-table guard short-circuits before
querying.
- _wait_result_ready / _await_query_run take the database_id and pass it
via the generated methods' _headers.
- The Arrow helper has no per-call headers, so _fetch_result_arrow sets
a scoped default header for the duration of the call and removes it
after (this client is not shared across threads).
- classify_sdk_error / api_error_message now append the response body
(flattened, 500-char cap): the body is where the API explains itself,
and dropping it made this bug a bare "400: Bad Request" for a week.
Verified live against api.hotdata.dev: the previously-failing state-table
append path (fetch -> concat -> load) now completes.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Verified the database-scope header is now carried on the result poll, the query-run poll, and the Arrow fetch, and that _fetch_result_arrow cleans up the client-level default header via finally (safe across retries). Error-body inclusion is bounded and guarded. Tests cover the regression and the truncation/no-body paths. LGTM.
eddietejeda
added a commit
that referenced
this pull request
Jul 8, 2026
…36) The 0.6.0 SDK (bumped in #34) added x_database_id to get_result, get_query_run, and get_result_arrow — and made it REQUIRED on get_result_arrow, so the header-based scope from #35 would raise TypeError at the Arrow fetch under the new SDK (the mocked tests could not see a generated-signature change). - _wait_result_ready / _await_query_run / _fetch_result_arrow pass x_database_id natively; the scoped default-header workaround is gone. - hotdata pin raised to >=0.6.0 (older framework code cannot run on 0.6.0 anyway, per the required param). - Test fakes mirror the 0.6.0 signatures, with x_database_id required on the Arrow fake so dropping the scope fails the suite again. Verified live against api.hotdata.dev: the state-table append path (fetch -> concat -> load) completes on SDK 0.6.0. Co-authored-by: Eddie A Tejeda <669988+eddietejeda@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fetch_table submitted the query WITH the database scope (x_database_id)
but then polled the stored result, polled the query run, and fetched the
Arrow body WITHOUT it. Results of database-scoped queries are themselves
database-scoped, so the results endpoints 400 ("X-Database-Id header is
required: this endpoint is scoped to a database") on every read against
an existing synced table — merge/append loads and dlt state restore
failed on any second run into the same database, while
first runs passed because the unsynced-table guard short-circuits before
querying.
via the generated methods' _headers.
a scoped default header for the duration of the call and removes it
after (this client is not shared across threads).
(flattened, 500-char cap): the body is where the API explains itself,
and dropping it made this bug a bare "400: Bad Request" for a week.
Verified live against api.hotdata.dev: the previously-failing state-table
append path (fetch -> concat -> load) now completes.