Skip to content

feat(core): add QuotaManager with exponential backoff and persistence#34

Open
iceteaSA wants to merge 1 commit into
cortexkit:mainfrom
iceteaSA:feat/quota-manager
Open

feat(core): add QuotaManager with exponential backoff and persistence#34
iceteaSA wants to merge 1 commit into
cortexkit:mainfrom
iceteaSA:feat/quota-manager

Conversation

@iceteaSA
Copy link
Copy Markdown
Contributor

@iceteaSA iceteaSA commented May 21, 2026

Unified quota cache and API gateway for main + fallback quota state. All consumers share one QuotaManager instance for consistent caching.

Features:

  • Inflight deduplication prevents concurrent API calls
  • Exponential backoff (60s–15min) for 429/5xx errors with escalating retry counts
  • Persists main quota and backoff state to disk via callbacks
  • Cross-process file lock guard for quota API dedup
  • Seeds from persisted storage on construction (avoids cold-start API call)
  • Integrates with FallbackAccountManager for shared staleness checks
  • Captures storage path at init to prevent test config corruption
  • Request-count-based refresh trigger (refreshEveryNRequests)

Files:

  • packages/core/src/quota-manager.ts — new (391 lines)
  • packages/core/src/accounts.ts — adds backoff helpers, FallbackAccountManager integration
  • packages/opencode/src/index.ts — QuotaManager construction, persistence callbacks, routing
  • packages/opencode/src/tests/quota-manager.test.ts — new (294 lines)

Summary by cubic

Adds a unified QuotaManager for main and fallback quotas with shared in-memory cache, a serial API gate, exponential backoff, and on-disk persistence. Integrates it into packages/opencode for consistent routing, shared staleness with fallbacks, and safer behavior when rate-limited.

  • New Features

    • Single QuotaManager with inflight dedup, a serialized API gate, and a cross‑process file lock.
    • Exponential backoff for 429/5xx (60s up to 15m), with persisted backoff state and callbacks (onMainQuotaFetched, onApiError).
    • Seeds main quota and backoff from disk on startup; adds request-count refresh via quota.refreshEveryNRequests.
    • Integrates with FallbackAccountManager: shared staleness checks, seeding from persisted fallback quotas, and gated refreshAllFallbacks.
    • Optional fail-closed when the quota API is backed off and no cached quota (failClosedOnUnknownQuota returns 429 with retry-after).
  • Migration

    • @cortexkit/anthropic-auth-core now exports QuotaManager.
    • If consuming FallbackAccountManager, pass a shared QuotaManager to unify cache and staleness across processes.
    • To persist main quota/backoff, wire QuotaManager callbacks to save quota.mainQuota, quota.mainQuotaCheckedAt, and quota.mainLastQuotaApiError (already implemented in packages/opencode).

Written for commit 731431a. Summary will update on new commits. Review in cubic

Unified quota cache and API gateway for main + fallback quota state.
All consumers share one QuotaManager instance for consistent caching.

Features:
- Inflight deduplication prevents concurrent API calls
- Exponential backoff (60s-15min) for 429/5xx errors
- Persists main quota and backoff state to disk via callbacks
- Cross-process file lock guard for quota API dedup
- Seeds from persisted storage on construction
- Integrates with FallbackAccountManager for shared staleness
- Captures storage path at init to prevent test config corruption
- Request-count-based refresh trigger (refreshEveryNRequests)
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 6 files

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread packages/opencode/src/index.ts
Comment thread packages/core/src/accounts.ts
@iceteaSA iceteaSA force-pushed the feat/quota-manager branch from 3767882 to 731431a Compare May 21, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant