Skip to content

Commit b869edb

Browse files
feat(tables): support loading from query results (#136)
Co-authored-by: hotdata-automation[bot] <267177015+hotdata-automation[bot]@users.noreply.github.com>
1 parent 6ebc2f9 commit b869edb

17 files changed

Lines changed: 79 additions & 42 deletions

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Changed
11+
12+
- feat(tables): support loading from query results
13+
1014
## [0.6.0] - 2026-07-07
1115

1216
### Changed

docs/ConnectionsApi.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ Method | HTTP request | Description
1313
[**get_connection**](ConnectionsApi.md#get_connection) | **GET** /v1/connections/{connection_id} | Get connection
1414
[**get_table_profile**](ConnectionsApi.md#get_table_profile) | **GET** /v1/connections/{connection_id}/tables/{schema}/{table}/profile | Get table profile
1515
[**list_connections**](ConnectionsApi.md#list_connections) | **GET** /v1/connections | List connections
16-
[**load_managed_table**](ConnectionsApi.md#load_managed_table) | **POST** /v1/connections/{connection_id}/schemas/{schema}/tables/{table}/loads | Load managed table from upload
16+
[**load_managed_table**](ConnectionsApi.md#load_managed_table) | **POST** /v1/connections/{connection_id}/schemas/{schema}/tables/{table}/loads | Load managed table from upload or query result
1717
[**purge_connection_cache**](ConnectionsApi.md#purge_connection_cache) | **DELETE** /v1/connections/{connection_id}/cache | Purge connection cache
1818
[**purge_table_cache**](ConnectionsApi.md#purge_table_cache) | **DELETE** /v1/connections/{connection_id}/tables/{schema}/{table}/cache | Purge table cache
1919

@@ -808,9 +808,9 @@ This endpoint does not need any parameter.
808808
# **load_managed_table**
809809
> LoadManagedTableResponse load_managed_table(connection_id, var_schema, table, load_managed_table_request)
810810
811-
Load managed table from upload
811+
Load managed table from upload or query result
812812

813-
Publish a previously-uploaded file as the new contents of a managed table. CSV, JSON, and Parquet uploads are supported; the format is auto-detected from the upload's `Content-Type` and file contents, or set explicitly via the `format` field. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the upload is applied: `replace` overwrites the table's contents, `append` inserts the uploaded rows on top of the existing data. Concurrent loads against the same upload return 409. Set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID.
813+
Publish data as the new contents of a managed table from one of two sources — provide exactly one. With `upload_id`, a previously-uploaded file is published: CSV, JSON, and Parquet are supported; the format is auto-detected from the upload's `Content-Type` and file contents, or set explicitly via the `format` field. With `result_id`, a persisted query result is copied into the table, so the table keeps its data even after the result expires; a result can be loaded into any number of tables. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the data is applied: `replace` overwrites the table's contents, `append` inserts the new rows on top of the existing data. Concurrent loads against the same upload return 409. For an upload, set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID. A `result_id` load runs synchronously.
814814

815815
### Example
816816

@@ -856,7 +856,7 @@ with hotdata.ApiClient(configuration) as api_client:
856856
load_managed_table_request = hotdata.LoadManagedTableRequest() # LoadManagedTableRequest |
857857

858858
try:
859-
# Load managed table from upload
859+
# Load managed table from upload or query result
860860
api_response = api_instance.load_managed_table(connection_id, var_schema, table, load_managed_table_request)
861861
print("The response of ConnectionsApi->load_managed_table:\n")
862862
pprint(api_response)
@@ -894,10 +894,10 @@ Name | Type | Description | Notes
894894
| Status code | Description | Response headers |
895895
|-------------|-------------|------------------|
896896
**200** | Managed table loaded | - |
897-
**202** | Load accepted and running in the background; poll the returned job for status and result | - |
898-
**400** | Invalid request (bad mode, non-managed connection, invalid identifier, bad parquet) | - |
899-
**404** | Connection or upload not found, or the table was deleted | - |
900-
**409** | Upload already consumed or in flight, or the uploaded data changes a column&#39;s type incompatibly (only widening to a larger compatible type can be applied automatically); the existing data is unchanged and remains queryable | - |
897+
**202** | Upload load accepted and running in the background; poll the returned job for status and result | - |
898+
**400** | Invalid request (bad mode, both or neither of &#x60;upload_id&#x60;/&#x60;result_id&#x60;, &#x60;format&#x60; combined with &#x60;result_id&#x60;, non-managed connection, invalid identifier, bad parquet, or the result failed to compute) | - |
899+
**404** | Connection, upload, or result not found, or the table was deleted | - |
900+
**409** | Upload already consumed or in flight, the result is still being computed, or the incoming data changes a column&#39;s type incompatibly (only widening to a larger compatible type can be applied automatically); the existing data is unchanged and remains queryable | - |
901901

902902
[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
903903

docs/DatabaseDetailResponse.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ Response body for GET /databases/{database_id}
77
Name | Type | Description | Notes
88
------------ | ------------- | ------------- | -------------
99
**attachments** | [**List[DatabaseAttachmentInfo]**](DatabaseAttachmentInfo.md) | |
10+
**created_at** | **datetime** | When the database was created. | [optional]
1011
**default_catalog** | **str** | Name the database&#39;s default catalog answers to inside its query scope (&#x60;default&#x60; unless overridden at create time). |
1112
**default_connection_id** | **str** | |
1213
**expires_at** | **datetime** | When this database expires. | [optional]

docs/DatabaseSummary.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ Summary item in GET /databases
66

77
Name | Type | Description | Notes
88
------------ | ------------- | ------------- | -------------
9+
**created_at** | **datetime** | When the database was created. | [optional]
910
**default_catalog** | **str** | Name the database&#39;s default catalog answers to inside its query scope. |
1011
**expires_at** | **datetime** | | [optional]
1112
**id** | **str** | |

docs/DatabasesApi.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ Method | HTTP request | Description
1212
[**detach_database_catalog**](DatabasesApi.md#detach_database_catalog) | **DELETE** /v1/databases/{database_id}/catalogs/{connection_id} | Detach catalog from database
1313
[**get_database**](DatabasesApi.md#get_database) | **GET** /v1/databases/{database_id} | Get database
1414
[**list_databases**](DatabasesApi.md#list_databases) | **GET** /v1/databases | List databases
15-
[**load_database_table**](DatabasesApi.md#load_database_table) | **POST** /v1/databases/{database_id}/schemas/{schema}/tables/{table}/loads | Load database table from upload
15+
[**load_database_table**](DatabasesApi.md#load_database_table) | **POST** /v1/databases/{database_id}/schemas/{schema}/tables/{table}/loads | Load database table from upload or query result
1616

1717

1818
# **add_database_schema**
@@ -710,9 +710,9 @@ This endpoint does not need any parameter.
710710
# **load_database_table**
711711
> LoadManagedTableResponse load_database_table(database_id, var_schema, table, load_managed_table_request)
712712
713-
Load database table from upload
713+
Load database table from upload or query result
714714

715-
Publish a previously-uploaded file as the new contents of a table on the database's default catalog. The database-scoped equivalent of the connection-scoped managed-table load — addressed by `database_id`, so no `default_connection_id` is needed. CSV, JSON, and Parquet uploads are supported; the format is auto-detected or set via `format`. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the upload is applied: `replace` overwrites the table's contents, `append` inserts the uploaded rows on top of the existing data. Concurrent loads against the same upload return 409. Set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID.
715+
Publish data as the new contents of a table on the database's default catalog, from one of two sources — provide exactly one. The database-scoped equivalent of the connection-scoped managed-table load — addressed by `database_id`, so no `default_connection_id` is needed. With `upload_id`, a previously-uploaded file is published: CSV, JSON, and Parquet are supported; the format is auto-detected or set via `format`. With `result_id`, a persisted query result is copied into the table, so the table keeps its data even after the result expires. If the target table (or its schema) has not been declared yet, it is created automatically as part of the load — declaring tables up front is optional. `mode` selects how the data is applied: `replace` overwrites the table's contents, `append` inserts the new rows on top of the existing data. Concurrent loads against the same upload return 409. For an upload, set `async` to run the load in the background and get back a job ID to poll; add `async_after_ms` to wait briefly for it to finish before falling back to a job ID. A `result_id` load runs synchronously.
716716

717717
### Example
718718

@@ -758,7 +758,7 @@ with hotdata.ApiClient(configuration) as api_client:
758758
load_managed_table_request = hotdata.LoadManagedTableRequest() # LoadManagedTableRequest |
759759

760760
try:
761-
# Load database table from upload
761+
# Load database table from upload or query result
762762
api_response = api_instance.load_database_table(database_id, var_schema, table, load_managed_table_request)
763763
print("The response of DatabasesApi->load_database_table:\n")
764764
pprint(api_response)
@@ -796,10 +796,10 @@ Name | Type | Description | Notes
796796
| Status code | Description | Response headers |
797797
|-------------|-------------|------------------|
798798
**200** | Table loaded | - |
799-
**202** | Load accepted and running in the background; poll the returned job for status and result | - |
800-
**400** | Invalid request (bad mode, invalid identifier, bad parquet) | - |
801-
**404** | Database or upload not found, or the table was deleted | - |
802-
**409** | Upload already consumed or in flight, or the uploaded data changes a column&#39;s type incompatibly (only widening to a larger compatible type can be applied automatically); the existing data is unchanged and remains queryable | - |
799+
**202** | Upload load accepted and running in the background; poll the returned job for status and result | - |
800+
**400** | Invalid request (bad mode, both or neither of &#x60;upload_id&#x60;/&#x60;result_id&#x60;, &#x60;format&#x60; combined with &#x60;result_id&#x60;, invalid identifier, bad parquet, or the result failed to compute) | - |
801+
**404** | Database, upload, or result not found, or the table was deleted | - |
802+
**409** | Upload already consumed or in flight, the result is still being computed, or the incoming data changes a column&#39;s type incompatibly (only widening to a larger compatible type can be applied automatically); the existing data is unchanged and remains queryable | - |
803803

804804
[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)
805805

docs/LoadManagedTableRequest.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,17 @@
11
# LoadManagedTableRequest
22

3-
Request body for the managed-table load endpoints — the connection-scoped `POST /v1/connections/{connection_id}/schemas/{schema}/tables/{table}/loads` and the database-scoped equivalent. Publishes a previously-uploaded file to the named table. CSV and JSON uploads are converted to columnar storage on load; Parquet uploads are published directly. `mode` selects whether the upload replaces the table's contents or is appended on top of them.
3+
Request body for the managed-table load endpoints — the connection-scoped `POST /v1/connections/{connection_id}/schemas/{schema}/tables/{table}/loads` and the database-scoped equivalent. Publishes data to the named table from one of two sources: a previously uploaded file (`upload_id`) or a persisted query result (`result_id`). Provide exactly one. CSV and JSON uploads are converted to columnar storage on load; Parquet uploads and query results are published directly. `mode` selects whether the data replaces the table's contents or is appended on top of them.
44

55
## Properties
66

77
Name | Type | Description | Notes
88
------------ | ------------- | ------------- | -------------
99
**var_async** | **bool** | When true, run the load as a background job and return a job ID to poll instead of blocking until it finishes. Recommended for large uploads, which can take longer than an HTTP request should stay open. | [optional]
1010
**async_after_ms** | **int** | If set (requires &#x60;async&#x60; &#x3D; true), wait up to this many milliseconds for the load to finish: if it completes in time the full result is returned (200), otherwise a 202 with a job ID to poll. Must be between 1000 and the server maximum; a value out of that range, or set without &#x60;async&#x60; &#x3D; true, is rejected with 400. | [optional]
11-
**format** | **str** | File format of the upload: &#x60;\&quot;csv\&quot;&#x60;, &#x60;\&quot;json\&quot;&#x60;, or &#x60;\&quot;parquet\&quot;&#x60;. Optional — when omitted, the format is auto-detected from the upload&#39;s &#x60;Content-Type&#x60; and, failing that, from the file contents. Provide it explicitly to override detection or when the contents are ambiguous. &#x60;\&quot;json\&quot;&#x60; expects newline-delimited JSON (one object per line), not a JSON array. | [optional]
12-
**mode** | **str** | How the upload is applied: &#x60;\&quot;replace\&quot;&#x60; overwrites the table&#39;s contents, &#x60;\&quot;append\&quot;&#x60; inserts the uploaded rows on top of the existing data. |
13-
**upload_id** | **str** | ID of a previously-staged upload (see &#x60;POST /v1/files&#x60;). The upload is claimed atomically; concurrent loads against the same &#x60;upload_id&#x60; return 409. |
11+
**format** | **str** | File format of the upload: &#x60;\&quot;csv\&quot;&#x60;, &#x60;\&quot;json\&quot;&#x60;, or &#x60;\&quot;parquet\&quot;&#x60;. Optional — when omitted, the format is auto-detected from the upload&#39;s &#x60;Content-Type&#x60; and, failing that, from the file contents. Provide it explicitly to override detection or when the contents are ambiguous. &#x60;\&quot;json\&quot;&#x60; expects newline-delimited JSON (one object per line), not a JSON array. Only applies to &#x60;upload_id&#x60;; query results are always parquet. | [optional]
12+
**mode** | **str** | How the data is applied: &#x60;\&quot;replace\&quot;&#x60; overwrites the table&#39;s contents, &#x60;\&quot;append\&quot;&#x60; inserts the new rows on top of the existing data. |
13+
**result_id** | **str** | ID of a persisted query result (see &#x60;GET /v1/results/{result_id}&#x60;) to publish as the table&#39;s contents. The result is copied into the table, so the table keeps its data even after the result expires. A result can be loaded into any number of tables. Provide either this or &#x60;upload_id&#x60;, not both. | [optional]
14+
**upload_id** | **str** | ID of a previously-staged upload (see &#x60;POST /v1/files&#x60;). The upload is claimed atomically; concurrent loads against the same &#x60;upload_id&#x60; return 409. Provide either this or &#x60;result_id&#x60;, not both. | [optional]
1415

1516
## Example
1617

0 commit comments

Comments
 (0)