Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 29 additions & 7 deletions .agent/system/mcp_protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -564,15 +564,37 @@ MCP supports streaming results for long-running operations. This could be useful

### Authentication

**Current Approach:**
- API key passed via environment variable or config parameter
- Single API key for entire server instance
- No per-tool authentication
The server resolves a per-request credential via `get_credentials(ctx)` and builds
the downstream HTTP client with `make_client(ctx)`. Two modes are supported, OAuth
first with API key as fallback:

**1. OAuth 2.1 (remote / HTTP — preferred):**
- Enabled when `MCP_OAUTH_AUTH_SERVER` is set. The server then runs as an MCP OAuth
*resource server* via FastMCP's `RemoteAuthProvider`.
- The ScrapeGraphAI web app at `https://scrapegraphai.com` (better-auth `mcp()`
plugin, serving `/api/auth/*`) is the *authorization server*. Clients sign in
there; no API key is needed afterwards.
- Incoming Bearer tokens are validated by `BetterAuthTokenVerifier`, which calls the
AS session endpoint (`/api/auth/mcp/get-session`, override via `MCP_OAUTH_VERIFY_URL`).
better-auth issues *opaque* tokens (not JWTs), so validation is a server-to-server
lookup that returns `null` (HTTP 200) for invalid/expired tokens.
- The verified raw token is forwarded to the ScrapeGraphAI API as
`Authorization: Bearer <token>`; the API resolves the user from it.
- Protected-resource metadata is exposed at `/.well-known/oauth-protected-resource`
using `MCP_PUBLIC_URL` as this server's public base URL.

**2. API key (stdio / Smithery / legacy remote — fallback):**
- Used when OAuth is disabled or no OAuth context is present on the request.
- Key sourced from the `X-API-Key` header (remote), `SGAI_API_KEY` env, or `--config`.
- Forwarded to the ScrapeGraphAI API as the `SGAI-APIKEY` header (scrapegraph-py v2
wire format).

**Env vars:** `MCP_OAUTH_AUTH_SERVER` (AS root URL), `MCP_PUBLIC_URL` (this server's
public URL, default `http://localhost:8000`), `MCP_OAUTH_VERIFY_URL` (optional override).

**Future Consideration:**
- Support multiple API keys (user-specific)
- OAuth integration
- JWT tokens
- Token refresh handling for long-running MCP connections
- Workspace/subscription attribution for OAuth tokens

### Rate Limiting

Expand Down
17 changes: 13 additions & 4 deletions .agent/system/project_architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -558,10 +558,19 @@ scrapegraph-mcp

### Configuration

**API Key Sources (in order of precedence):**
1. `--config` parameter (Smithery): `"{\"scrapegraphApiKey\":\"key\"}"`
2. Environment variable: `SGAI_API_KEY`
3. Default: `None` (server fails to initialize)
**Credential Sources (in order of precedence):**
1. **OAuth 2.1 Bearer token** (remote only) — when `MCP_OAUTH_AUTH_SERVER` is set, a
verified access token from the request is forwarded as `Authorization: Bearer`.
See [MCP Protocol — Authentication](./mcp_protocol.md#authentication).
2. `--config` parameter (Smithery): `"{\"scrapegraphApiKey\":\"key\"}"`
3. `X-API-Key` header (remote/mcp-remote)
4. Environment variable: `SGAI_API_KEY`
5. Default: `None` (server fails to initialize)

**OAuth env vars (remote resource-server mode):**
- `MCP_OAUTH_AUTH_SERVER` — authorization server root URL: the better-auth web app that serves `/api/auth/*` (e.g. `https://scrapegraphai.com`); unset = OAuth disabled
- `MCP_PUBLIC_URL` — this server's public base URL (default `http://localhost:8000`)
- `MCP_OAUTH_VERIFY_URL` — optional override of the token verification endpoint (default `{AS}/api/auth/mcp/get-session`)

**Server Transport:**
- **stdio** - Standard input/output (default for MCP)
Expand Down
25 changes: 23 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,30 @@ HOST=0.0.0.0
PORT=8000

# ===========================================
# FOR USERS CONNECTING TO REMOTE SERVER
# OAUTH 2.1 (MCP AUTHORIZATION) — REMOTE ONLY
# ===========================================
# Pass your ScrapeGraph API key via the X-API-Key header using mcp-remote.
# Set MCP_OAUTH_AUTH_SERVER to enable OAuth. The server then acts as an OAuth
# resource server: MCP clients sign in through the ScrapeGraphAI web app
# (the authorization server) and no API key is needed afterwards. The verified
# access token is forwarded to the ScrapeGraphAI API, which resolves the user.
#
# Public root URL of the authorization server (the better-auth web app — the host
# that serves /api/auth/*, e.g. /api/auth/ok and the sign-up/login pages).
MCP_OAUTH_AUTH_SERVER=https://scrapegraphai.com
# This server's own public base URL (for OAuth protected-resource metadata).
MCP_PUBLIC_URL=https://mcp.scrapegraphai.com
# Optional: override the token verification endpoint.
# Default: ${MCP_OAUTH_AUTH_SERVER}/api/auth/mcp/get-session
# MCP_OAUTH_VERIFY_URL=https://scrapegraphai.com/api/auth/mcp/get-session
#
# Local testing example:
# MCP_OAUTH_AUTH_SERVER=http://localhost:3000
# MCP_PUBLIC_URL=http://localhost:8000

# ===========================================
# FOR USERS CONNECTING TO REMOTE SERVER (LEGACY API-KEY MODE)
# ===========================================
# Without OAuth, pass your ScrapeGraph API key via the X-API-Key header using mcp-remote.
#
# Example Claude Desktop config:
# {
Expand Down
19 changes: 17 additions & 2 deletions render.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,5 +23,20 @@ services:
- key: MCP_TRANSPORT
value: http

# Note: SGAI_API_KEY is NOT set here.
# Each user passes their own API key via mcp-remote from their local environment.
# ── OAuth 2.1 (MCP authorization) ──────────────────────────────────────
# When MCP_OAUTH_AUTH_SERVER is set, this server runs as an OAuth resource
# server: clients sign in via the better-auth web app (no API key needed) and
# the access token is forwarded to the ScrapeGraphAI API. Unset it to fall back
# to API-key-only auth.
- key: MCP_OAUTH_AUTH_SERVER
value: https://scrapegraphai.com
# This server's own public URL (used for OAuth protected-resource metadata).
- key: MCP_PUBLIC_URL
value: https://mcp.scrapegraphai.com
# Optional override of the token verification endpoint. Defaults to
# {MCP_OAUTH_AUTH_SERVER}/api/auth/mcp/get-session
# - key: MCP_OAUTH_VERIFY_URL
# value: https://scrapegraphai.com/api/auth/mcp/get-session

# Note: SGAI_API_KEY is NOT set here. Legacy clients can still pass their own
# API key via the X-API-Key header (mcp-remote); OAuth is additive.
Loading
Loading