Skip to content

feat!: kwargs API — replace *Request objects on public methods#88

Merged
VinciGit00 merged 6 commits intomainfrom
feat/ergonomic-api
Apr 21, 2026
Merged

feat!: kwargs API — replace *Request objects on public methods#88
VinciGit00 merged 6 commits intomainfrom
feat/ergonomic-api

Conversation

@FrancescoSaverioZuppichini
Copy link
Copy Markdown
Member

@FrancescoSaverioZuppichini FrancescoSaverioZuppichini commented Apr 21, 2026

Summary

  • Public client methods now take inline kwargs instead of Pydantic *Request objects — matches the JS SDK and standard Python SDK ergonomics (OpenAI, Anthropic, Stripe).
  • *Request classes remain exported for users who want to build payloads manually, but client methods no longer accept them.
  • Structured types (FetchConfig, format configs, schema dicts) are still passed as-is.

Before / After

# Before
sgai.scrape(ScrapeRequest(url="https://example.com", formats=[MarkdownFormatConfig()]))

# After
sgai.scrape("https://example.com", formats=[MarkdownFormatConfig()])

Scope

  • ScrapeGraphAI / AsyncScrapeGraphAI: scrape, extract, search
  • crawl.start
  • monitor.create, monitor.update, monitor.activity
  • history.list
  • Unchanged: credits, health, crawl.get/stop/resume/delete, monitor.get/list/delete/pause/resume, history.get

Internal detail

A small _compact helper strips None kwargs so Pydantic fields with non-None defaults (e.g. formats, max_depth, page) fall back to their defaults instead of raising a ValidationError on None.

Breaking Change

Callers must switch from sgai.method(SomeRequest(...))sgai.method(...) per the new signatures. Suggest minor version bump (2.1.0) — fresh 2.x line, low blast radius.

Test plan

  • All 28 unit tests pass (uv run pytest tests/)
  • Live smoke test against default host — credits, scrape, search, crawl.start + poll, extract (with and without schema) all succeed
  • uv run ruff format src tests clean
  • uv run ruff check src tests clean
  • uv build succeeds
  • Integration tests (live API) — updated to new shape, ran a subset manually

Files

  • src/scrapegraph_py/{client,async_client}.py — public method signatures
  • tests/test_client.py, tests/test_integration.py — updated call sites
  • examples/**/*.py — all 30 examples updated
  • README.md — every snippet updated

🤖 Generated with Claude Code

BREAKING CHANGE: Public client methods now accept keyword arguments
directly instead of Pydantic *Request objects. This matches the JS SDK
and conventional Python SDK ergonomics (OpenAI, Anthropic, Stripe).

Before:
    sgai.scrape(ScrapeRequest(url="https://example.com", formats=[...]))

After:
    sgai.scrape("https://example.com", formats=[...])

The *Request classes remain exported for users who want to build
payloads manually, but are no longer accepted by the client methods.
Structured types (FetchConfig, format configs, schema dicts) still
passed as-is.

- Refactored ScrapeGraphAI + AsyncScrapeGraphAI sync/async methods
- Added _compact helper to strip None kwargs so Pydantic defaults apply
- Updated all 30 examples and README
- Updated unit + integration tests

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 21, 2026

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

Bump minor for new kwargs-based public API (see #88).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The union type was awkwardly named. FormatConfig matches the naming of
its members (MarkdownFormatConfig, HtmlFormatConfig, ...) and reads
naturally in signatures: formats: list[FormatConfig].

Also exported from the top-level package so users can type-annotate
their own code.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@gitguardian
Copy link
Copy Markdown

gitguardian Bot commented Apr 21, 2026

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Renames:
- ApiService -> Service
- ApiHtmlMode -> HtmlMode
- ApiFetchMode -> FetchMode
- ApiTimeRange -> TimeRange
- ApiCrawlStatus -> CrawlStatus
- ApiCrawlPageStatus -> CrawlPageStatus
- ApiHistoryStatus -> HistoryStatus
- ApiFetchContentType -> FetchContentType
- ApiMonitorTickStatus -> MonitorTickStatus

Removed (unused):
- ApiStatus
- ApiScrapeFormat
- ApiHistoryService (duplicate of ApiService)

Kept: ApiResult. The prefix earns its keep there since it's
semantically distinct from the *Response types and "Result[T]"
alone would be too generic.

All aliases now exported from scrapegraph_py for user type
annotations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@VinciGit00 VinciGit00 merged commit f996715 into main Apr 21, 2026
6 checks passed
@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version 3.0.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants