feat(uploads): harden multipart uploads to sdk-rust parity (#133)

* feat(uploads): harden multipart upload resilience

* test(auth): cover token-exchange body-read retry

* feat(uploads): rust parity gaps and upload ergonomics

Author: zfarrell

README.md

@@ -121,18 +121,40 @@ with ApiClient(Configuration(api_key="...", workspace_id="...")) as client:
 (`size` is inferred for all three; a file object is read from its current
 position to the end). The SDK picks single vs. multipart from the size,
 auto-scales the part size, and bounds part concurrency to a peak-memory budget
-(override with `part_size` / `max_concurrency` / `part_retry`). Storage `PUT`s go
-through a dedicated, header-isolated connection pool, so the SDK's auth and
-workspace headers never reach object storage (which would otherwise reject the
-upload). Finalize is sent with retries disabled so the exactly-once call is never
-accidentally replayed.
-
-Failures surface as a typed hierarchy under `hotdata.uploads.UploadError`:
-`StorageError` (storage returned a non-2xx), `StorageTransportError` (the PUT
-failed before any response), `MissingETagError`, `MalformedSessionError`, and
-`SizeLimitError`. Opening the session or finalizing raises the usual
-`hotdata.exceptions.ApiException` — for example a `501` `PRESIGN_UNSUPPORTED`,
-meaning the backend cannot issue upload URLs.
+(override with `part_size` / `max_concurrency` / `part_retry`). A file larger than
+8 MiB uploads via a **streaming** session — the SDK mints each part's URL just
+before its `PUT`, so a presigned URL can't expire mid-transfer on a slow upload.
+Storage `PUT`s go through a dedicated, header-isolated connection pool (auth and
+workspace headers never reach object storage, which would otherwise reject the
+upload) with a 30s connect timeout so a dead endpoint fails fast. Finalize is sent
+with retries disabled so the exactly-once call is never accidentally replayed.
+
+Every failure is a subclass of `hotdata.uploads.UploadError` (also importable as
+`from hotdata import UploadError`), so a single `except UploadError` catches the
+whole flow: `SessionCreateError` (opening the session — check `.status` for a
+`501` `PRESIGN_UNSUPPORTED`), `StorageError` (storage returned a non-2xx; `.exhausted`
+is `True` if it outlived every retry round), `StorageTransportError` (the PUT
+failed before any response), `MissingETagError`, `MintPartError` (minting a part
+URL), `FinalizeError`, `MalformedSessionError`, `SizeLimitError`, and
+`UploadCancelledError`. The phase errors that wrap a control-plane call chain the
+underlying `hotdata.exceptions.ApiException` as `__cause__` (and expose it as
+`.api_exception` / `.status`). A local file read error surfaces as `OSError`.
+
+The `progress` callback receives a **cumulative** `(bytes_done, total)` — for a
+tqdm bar (whose `update(n)` wants a delta, and which isn't thread-safe under
+multipart) use the ready-made adapter:
+
+```python
+from hotdata import tqdm_progress
+from tqdm import tqdm
+
+with tqdm(total=size, unit="B", unit_scale=True) as bar:
+    uploads.upload_file("data.parquet", progress=tqdm_progress(bar))
+```
+
+Pass a `threading.Event` as `cancel_event` to abort an in-flight upload; tune the
+control-plane calls with `request_timeout` (storage-PUT timeouts are automatic and
+size-scaled).
 
 For that fallback (or to upload from a non-seekable stream), use `upload_stream`,
 which sends the bytes to the legacy `POST /v1/files` endpoint in one request,

hotdata/__init__.py

@@ -334,3 +334,38 @@
 from hotdata.query import QueryApi as QueryApi  # noqa: E402,F811
 from hotdata.arrow import ResultsApi as ResultsApi  # noqa: E402,F811
 from hotdata.uploads import UploadsApi as UploadsApi  # noqa: E402,F811
+# The upload error hierarchy + option/helper types, so `from hotdata import
+# UploadError` / `StorageError` / ... works (they otherwise live only under
+# hotdata.uploads and are invisible to autocomplete on `hotdata.`).
+from hotdata.uploads import (  # noqa: E402,F401
+    UploadError as UploadError,
+    SessionCreateError as SessionCreateError,
+    MalformedSessionError as MalformedSessionError,
+    SizeLimitError as SizeLimitError,
+    StorageError as StorageError,
+    StorageTransportError as StorageTransportError,
+    MissingETagError as MissingETagError,
+    MintPartError as MintPartError,
+    FinalizeError as FinalizeError,
+    UploadCancelledError as UploadCancelledError,
+    UploadOptions as UploadOptions,
+    UploadProgress as UploadProgress,
+    UploadSource as UploadSource,
+    tqdm_progress as tqdm_progress,
+)
+__all__ += [  # noqa: F405
+    "UploadError",
+    "SessionCreateError",
+    "MalformedSessionError",
+    "SizeLimitError",
+    "StorageError",
+    "StorageTransportError",
+    "MissingETagError",
+    "MintPartError",
+    "FinalizeError",
+    "UploadCancelledError",
+    "UploadOptions",
+    "UploadProgress",
+    "UploadSource",
+    "tqdm_progress",
+]