feat(auth): Phase 2 — AWS Sigv4, GCP IAP, Azure AD providers (v0.11.0)

[naveen-kurra]

Summary

Phase 2 of the pluggable auth provider work — three cloud-native providers
on top of the Phase 1 foundation (#77 / 7998f12). Customers authenticate
to Forge using identities they already have in their cloud; no parallel
IdP required.

Provider	What it does	Client token format
aws_sigv4	Caller mints a pre-signed STS GetCallerIdentity URL with their AWS SDK; Forge invokes it. STS returns the caller's canonical ARN. Same pattern as aws-iam-authenticator (EKS).	Authorization: Bearer forge-aws-v1.<base64-of-presigned-sts-url>
gcp_iap	Verify the JWT GCP IAP forwards as X-Goog-Iap-Jwt-Assertion when Forge sits behind a GCP HTTPS LB + IAP.	X-Goog-Iap-Jwt-Assertion: <jwt>
azure_ad	Verify Microsoft Entra ID Bearer tokens with tenant lock-in and optional Microsoft Graph group enrichment. Composes the Phase 1 oidc provider.	Authorization: Bearer <aad-jwt>

Forge never holds any IdP secrets — all three providers verify a caller-
minted credential against a third party (STS / GCP JWKS / AAD JWKS).

Why this matters

Today, putting Forge behind any of the three big cloud IdPs requires
standing up a parallel OIDC issuer (Cognito for AWS, Workspace SAML, etc.).
This PR removes that friction:

• AWS: any Lambda / EC2 / EKS / IAM user calls Forge using their
  existing IAM credentials. Zero secrets stored on Forge, no token endpoint
  to host.
• GCP: Forge sitting behind GCP LB+IAP consumes IAP's signed user
  assertion directly.
• Azure: Entra tokens (humans, CI, Azure workloads) work natively with
  AAD-specific quirks (tenant gate, groups overage) handled correctly.

Design pivot during PR review — aws_sigv4 switched to pre-signed URL pattern

The PR went through an in-flight design correction caught by real-AWS
smoke testing. The TL;DR:

What was wrong (original design — commits 9a1ebae through 382294e)

The first design had clients sign their POST to Forge using AWS Sigv4
("header reflection"). Forge would forward the signed headers to STS,
expecting STS to validate them.

This is broken in a deterministic way: Sigv4 binds its signature to
the destination host as part of the canonicalized signing input.
Headers signed for Forge's hostname can't be replayed against STS —
STS computes host: sts.<region>.amazonaws.com, recomputes the
signature, gets a different hash, and rejects with
SignatureDoesNotMatch.

Standard tools (awscurl, boto3.client('sts'), all AWS SDKs) always
sign for the URL they're calling, so there was no working client path.

What's correct (current design — commit 8568535)

Switched to the pre-signed URL pattern that aws-iam-authenticator
uses for EKS:

1. Caller's AWS SDK mints a pre-signed GetCallerIdentity URL — signature
   embedded in query params, signed for STS's host.
2. Caller base64-encodes the URL, prepends forge-aws-v1., sends as a
   standard Authorization: Bearer … header.
3. Forge decodes, validates the URL host matches sts.<region>.amazonaws.com
   (SSRF guard), GETs the URL, parses the XML response, stamps Identity.

Why this is the right design (not just a fix)

Three design properties land cleanly:

1. Client UX matches OIDC/JWT/Azure AD. All three Phase 2 providers
   now have the same caller experience: mint a Bearer token, attach it,
   send. Three lines of Python (or any AWS-SDK-bearing language). No
   custom signing logic, no header manipulation, no per-call procedure.
2. No re-engineering of standard tools. aws-iam-authenticator-style
   token format works with any AWS SDK's SigV4QueryAuth API.
3. Server is simpler. Forge doesn't reflect headers; it just GETs a
   URL. ~80 LOC of STS client code instead of header-forwarding plumbing.

Locked design decisions (unchanged through the pivot)

• §9.1 — No aws-sdk-go-v2 dependency. STS client is hand-rolled HTTP + XML.
• §9.2 — azure_ad composes Phase 1 oidc.Provider; no JWT/JWKS code in azure_ad/.
• §9.3 — allowed_principals uses path.Match shell globs (no regex).
• §9.4 — IAP issuer + JWKS URL hardcoded; no operator-overridable knob.

What clients write (the actual 3 lines)

import boto3, base64
from botocore.auth import SigV4QueryAuth
from botocore.awsrequest import AWSRequest

creds = boto3.Session().get_credentials().get_frozen_credentials()
req = AWSRequest(method='GET',
                 url='https://sts.us-east-1.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15')
SigV4QueryAuth(creds, 'sts', 'us-east-1', expires=900).add_auth(req)
token = 'forge-aws-v1.' + base64.urlsafe_b64encode(req.url.encode()).rstrip(b'=').decode()

requests.post(forge_url, headers={'Authorization': f'Bearer {token}'}, data=msg)

A reference client lives at scripts/forge-aws-sign.py (~80 LOC; CLI flags
for --region, --profile, --token-only, etc.).

Note on boto3: the high-level boto3.client('sts').generate_presigned_url('get_caller_identity', ...) returns a URL STS rejects with SignatureDoesNotMatch even on direct curl — known boto3 limitation (it signs as if the request were a POST). The reference client uses the lower-level SigV4QueryAuth API directly, same as aws-iam-authenticator does internally.

✅ Real-AWS validation (live AWS account)

End-to-end validated against real AWS STS using SSO assumed-role
credentials in account 412664885516. Forge instance built from
commit 8568535, running locally.

#	Scenario	Expected	Actual	Validates
1	curl with no auth headers	401, missing_token	✅ 401 + valid bearer token required	Phase 1 middleware enforces auth; review #4 reason-code preserved
2	Pre-signed URL token, empty allowlist	200 / handler-error, auth_verify fires	✅ HTTP 400 (body shape; auth path complete), auth_verify emitted	STS reflection works end-to-end against real STS
3	Same token, matching allowed_principals	200 / handler-error (auth pass)	✅ HTTP 400, auth_verify fires	ArnMatcher accepts matching ARNs
4	Same token, wrong-account allowed_principals	401, rejected	✅ 401 + token rejected by auth provider	ArnMatcher correctly denies non-matching ARNs

Audit log emitted (Test #2 success path)

{
  "event":"auth_verify",
  "fields":{
    "method":"POST",
    "path":"/tasks/send",
    "provider":"aws_sigv4",
    "user_id":"arn:aws:sts::412664885516:assumed-role/AWSReservedSSO_PowerUserAccess_c794d5f2c2fe4370/Naveen",
    "org_id":"412664885516",
    "token_kind":"sigv4",
    "groups_count":0,
    "remote_addr":"[::1]:62448"
  }
}

Every field correct: provider matches the configured name, user_id is the STS-returned canonical ARN (assumed-role form, including session name), org_id is the AWS account number, token_kind is the new forge-aws-v1.-prefix detection.

Two bugs caught during live testing (both fixed in 8568535)

1. URL re-encoding through net/url. Round-tripping the presigned URL through Go's *url.URL.String() re-encoded query params in ways that differed from how the AWS SDK emitted them (e.g., / in X-Amz-Credential, + inside X-Amz-Security-Token). STS recomputed the canonical request from those re-encoded bytes and rejected. Fix: PresignedToken keeps a RawURL string field with the byte-for-byte original; the parsed *url.URL is used only for SSRF host validation and query-param inspection.
2. boto3's generate_presigned_url quirk. The reference client originally used boto3.client('sts').generate_presigned_url('get_caller_identity', …), which produces a URL STS rejects with SignatureDoesNotMatch (known boto3 quirk — signs as if POST). Fix: the reference client now uses the lower-level SigV4QueryAuth.add_auth() directly, same pattern aws-iam-authenticator uses.

Other layers also exercised live

• ✅ Egress allowlist auto-extended to include sts.us-east-1.amazonaws.com — Forge's outbound STS call was permitted.
• ✅ Loopback static_token still works for the dashboard at http://localhost:9999/ (auto-prepended via PrependChain — Phase 1 review Add per-agent secrets, build signing, and forge framework #10 invariant intact under Phase 2).
• ✅ Config hot-reload picks up forge.yaml content changes for most fields. Caveat: auth-chain providers are constructed once at startup; modifying allowed_principals requires a hard restart (Ctrl-C + forge run again) for the new allowlist to take effect. Documented as a follow-up — affects all providers, not just aws_sigv4.
• ✅ Auth-before-Egress wizard ordering (639bfa9) ensures wizard-scaffolded configs include the STS host in egress_hosts.

What was NOT live-tested (and why)

• gcp_iap provider — requires a GCP project with HTTPS LB + IAP enabled. Covered by unit tests with a fake JWKS signer.
• azure_ad provider — requires an Entra tenant + app registration. Covered by unit tests with a fake AAD.

Per PHASE2_TEST_STRATEGY.md §8.2, those live tests run at release-tag time, not on every PR.

What landed (12 commits)

#	Commit	Theme
PR 1	55942d8	Header contract extension (foundation)
PR 2	9a1ebae	aws_sigv4 provider (initial — header-reflection design)
PR 3	5b71071	gcp_iap provider
PR 4	1e23140	azure_ad provider
PR 5	47c4748	Wizard + non-interactive flags + Web UI integration
PR 6	98578f9	Operator docs + CHANGELOG
fix	639bfa9	Auth step runs before Egress; auth hosts auto-added to allowlist
audit	b5f303b	Audit-pass refinements (iap_jwt token_kind, URL parse caching, cleanup)
docs	aaf8375	Gitignore docs/auth (per reviewer feedback)
client	382294e	Reference client + "client signing contract" docstring (original pattern)
pivot	b3444c2	Switch aws_sigv4 to pre-signed URL pattern
live	8568535	Live-test fixes: preserve raw URL; use SigV4QueryAuth in client

Total: ~42 files, +5,500 / -700 lines net (mostly tests + the design-pivot rewrite).

Phase 1 compatibility

• Zero forge.yaml changes required for callers using Phase 1 providers (static_token, oidc, http_verifier). Phase 1 test suite passes unmodified.
• The auth Headers map gained one new key — X-Goog-Iap-Jwt-Assertion for gcp_iap. Existing keys unchanged.
• The oidc package gained an internal SkipIssuerCheck field with yaml:"-" — unreachable from forge.yaml, only set by azure_ad multi-tenant. Operators see no change.

Security model highlights

• No IdP secret on Forge. STS / GCP / AAD do the cryptographic work; Forge reflects or verifies, never holds keys.
• SSRF guard on aws_sigv4 — the pre-signed URL host MUST match sts.<configured-region>.amazonaws.com. A token whose URL points elsewhere is rejected at parse time, before any outbound request.
• Algorithm whitelist before key lookup on the OIDC / IAP / AAD providers — algorithm-confusion attacks rejected before reaching JWKS.
• Tenant gate before Graph on azure_ad — multi-tenant requires explicit opt-in; Graph calls only fire after tid validation.
• Caller's Bearer reflected to Graph — Forge holds no Graph credentials.

Known deferred work

• TUI sub-step input flows for the three new providers — non-interactive flag path is the production-critical surface; TUI users pick "Custom" and edit forge.yaml directly until the TUI follow-up lands.
• Hot-reload of auth chain — config edits to auth.providers (incl. allowed_principals) require a hard forge run restart. Affects all providers, not just Phase 2.

Test plan

• Unit tests: happy path, malformed input, scope/tenant rejection, allowlist miss, cache hit/miss, stale-grace, soft-fail, alg confusion, body caps, redirect attacks — all three new providers
• Pre-signed URL parser fuzz test (corpus-seeded, runs in CI)
• Phase 1 regression suite passes unmodified
• go test -race -count=1 ./... — 42 packages green
• golangci-lint v2.10.1 — 0 issues
• gofmt -l forge-core forge-cli forge-plugins — clean
• Anti-pattern grep gates (no aws-sdk-go import, IAP constants confined, no JWT in azure_ad, skip_issuer_check never in YAML) — all pass
• Live AWS validation — happy + deny paths against real STS (see "Real-AWS validation" above)
• Manual smoke for gcp_iap and azure_ad — runs at release-tag time per PHASE2_TEST_STRATEGY.md §8.2

Design artifacts (offline)

Full design package in ~/Desktop/forge_designs_and_PRD/phase2_implementation/:

• PHASE2_CLOUD_NATIVE_PROVIDERS.md — top-level design + §9 locked decisions
• PHASE2_PROGRESS_MAP.md — diagram-to-PR tracker
• PHASE2_TEST_STRATEGY.md — pyramid, harnesses, security catalog, CI gates, manual smoke
• 7 PlantUML diagrams (component / class / startup / request flow / per-provider deep-dives)
• PR1_HEADER_CONTRACT.md through PR6_DOCS.md — per-PR checklists with code sketches and acceptance criteria