Skip to content

feat: add ApifyApiError subclasses grouped by HTTP status#737

Open
vdusek wants to merge 4 commits intomasterfrom
feat/api-error-subclasses
Open

feat: add ApifyApiError subclasses grouped by HTTP status#737
vdusek wants to merge 4 commits intomasterfrom
feat/api-error-subclasses

Conversation

@vdusek
Copy link
Copy Markdown
Contributor

@vdusek vdusek commented Apr 17, 2026

Summary

Addresses #423. Adds a small set of ApifyApiError subclasses so callers can narrow except clauses to specific failure modes. Dispatch is driven by HTTP status — a stable contract — rather than the per-endpoint error.type string. type, message, and data remain exposed as metadata.

All subclasses inherit from ApifyApiError, so existing except ApifyApiError handlers keep matching.

Class Status
InvalidRequestError 400
UnauthorizedError 401
ForbiddenError 403
NotFoundError 404
ConflictError 409
RateLimitError 429
ServerError 5xx

Unmapped statuses and unparsable bodies fall back to the base ApifyApiError.

Migration note

Constructing ApifyApiError(response, ...) directly now returns an instance of the matching subclass. except ApifyApiError is unaffected; tests asserting type(exc) is ApifyApiError on a mapped status would need updates. catch_not_found_or_throw now uses isinstance(exc, NotFoundError) instead of a hardcoded error.type allowlist.

@vdusek vdusek added adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. labels Apr 17, 2026
@vdusek vdusek self-assigned this Apr 17, 2026
@github-actions github-actions bot added this to the 138th sprint - Tooling team milestone Apr 17, 2026
@github-actions github-actions bot added the tested Temporary label used only programatically for some analytics. label Apr 17, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 17, 2026

Codecov Report

❌ Patch coverage is 93.47826% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.49%. Comparing base (6c3b84d) to head (7bc0d51).
⚠️ Report is 6 commits behind head on master.

Files with missing lines Patch % Lines
src/apify_client/errors.py 93.18% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #737      +/-   ##
==========================================
+ Coverage   95.36%   95.49%   +0.12%     
==========================================
  Files          45       45              
  Lines        5118     5152      +34     
==========================================
+ Hits         4881     4920      +39     
+ Misses        237      232       -5     
Flag Coverage Δ
integration 95.49% <93.47%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vdusek vdusek requested review from Pijukatel and janbuchar April 17, 2026 11:16
@vdusek
Copy link
Copy Markdown
Contributor Author

vdusek commented Apr 17, 2026

Let me guys know whether you think this is worth it. Currently, I'm slightly more inclined not to do it and keep the status quo. Since the API type errors are just general strings, they can often change.

@janbuchar
Copy link
Copy Markdown
Contributor

Arguments in favor

  1. Better DX - easier to catch individual types of exception, while you can still just catch ApifyApiError if you don't care much
  2. Better DX - the exception name has a non-generic name, and that's the first thing you look at when you see an exception traceback in logs

Arguments against

  1. It' a lot of generated code
  2. We need to maintain the generator
  3. If an error type disappears from a response object, it might get deleted from the code base, which is a breaking change (but it's also a breaking change in the API, so I'm not sure about the validity of this concern)

@Pijukatel
Copy link
Copy Markdown
Contributor

I would add one convenience argument against this change, which would only be used as a tie breaker:

Adding this feature is non-breaking. Removing it is breaking.

So I would wait until we are more certain about it, as we can easily add this in any minor release.

vdusek and others added 3 commits April 21, 2026 09:32
Addresses #423. Each `ErrorType` enum member from the OpenAPI spec now has a
matching `ApifyApiError` subclass (e.g. `RecordNotFoundError`) generated into
`_generated_errors.py`. `ApifyApiError.__new__` dispatches to the right
subclass based on the response's `error.type`, so `except ApifyApiError` keeps
working while `except RecordNotFoundError` becomes possible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…d __init__

`__new__` used to parse `response.json()` for dispatch and `__init__` parsed
it again for attribute assignment. Now `__new__` stashes the parsed `error`
dict on the instance and `__init__` reads it, so each raised error parses
the body once. Also trims the stale star-import comment.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vdusek vdusek force-pushed the feat/api-error-subclasses branch from 189e0aa to cb57e87 Compare April 21, 2026 08:00
@vdusek vdusek changed the title feat: generate Exception subclass per API error type feat: add ApifyApiError subclasses grouped by HTTP status Apr 21, 2026
@vdusek vdusek requested review from Copilot and removed request for Pijukatel and janbuchar April 21, 2026 11:17
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces HTTP-status-based ApifyApiError subclasses so callers can catch specific API failure modes (e.g., NotFoundError for 404) without relying on endpoint-specific error.type strings.

Changes:

  • Added ApifyApiError.__new__ dispatch to map selected HTTP statuses to dedicated exception subclasses (and 5xx to ServerError).
  • Refactored API error-body parsing into _extract_error_payload() and kept type/message/data as metadata.
  • Updated catch_not_found_or_throw() and unit tests to use NotFoundError / status-based behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
src/apify_client/errors.py Adds status→exception dispatch and new ApifyApiError subclasses; refactors payload extraction.
src/apify_client/_utils.py Updates not-found suppression to use isinstance(..., NotFoundError).
tests/unit/test_utils.py Adjusts not-found suppression expectations and mocks JSON payload parsing.
tests/unit/test_client_errors.py Adds tests asserting correct subclass dispatch for mapped statuses, streamed responses, 5xx, and fallback cases.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/apify_client/errors.py
Comment thread src/apify_client/errors.py Outdated
@vdusek vdusek requested review from Pijukatel and janbuchar April 21, 2026 11:58
@vdusek vdusek marked this pull request as ready for review April 21, 2026 12:05
is_not_found_status = exc.status_code == HTTPStatus.NOT_FOUND
is_not_found_type = exc.type in ['record-not-found', 'record-or-token-not-found']
if not (is_not_found_status and is_not_found_type):
if not isinstance(exc, NotFoundError):
Copy link
Copy Markdown
Contributor

@Pijukatel Pijukatel Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about this (but the previous variant had the same issues.)

The problem of swallowing 404s is that you lose the information about what was not found. For example

dataset_1 = client.run(run_id="RUN DOES NOT EXIST").dataset().get() # Run does not exist
dataset_2 = client.run(run_id="RUN EXISTS").dataset().get() # Dataset does not exist

In both cases it returns None and user has no idea which of the cases happened.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably just swallow 404 related to the last resource in the chain and throw all the others?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adhoc Ad-hoc unplanned task added during the sprint. t-tooling Issues with this label are in the ownership of the tooling team. tested Temporary label used only programatically for some analytics.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants