Skip to content

Improve interruption responsiveness#2296

Draft
GeorgeNgMsft wants to merge 1 commit intomainfrom
dev/georgeng/cli_interrupt
Draft

Improve interruption responsiveness#2296
GeorgeNgMsft wants to merge 1 commit intomainfrom
dev/georgeng/cli_interrupt

Conversation

@GeorgeNgMsft
Copy link
Copy Markdown
Contributor

Before these changes, pressing Escape mid-query felt sluggish or unresponsive:

  • The UI would show "cancelled" eventually, but only after the LLM finished responding
  • During a slow/long generation (or a 429 retry pause), you'd be stuck waiting even though you
    already cancelled
  • Streaming chunks could keep rendering in the UI for a moment after cancel

After these changes, cancel feels immediate:

  • Press Escape → the HTTP connection to the LLM drops instantly
  • No more waiting for the full response to come back before "⚠ Cancelled" appears
  • Streaming stops mid-token rather than finishing the current response
  • If you cancel during a rate-limit retry, you're freed immediately rather than sleeping the full
    backoff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant