Skip to content

Add Config.SoftStopTimeout for a cleaner way to shut a client down#1239

Open
brandur wants to merge 1 commit intomasterfrom
brandur-soft-stop-timeout
Open

Add Config.SoftStopTimeout for a cleaner way to shut a client down#1239
brandur wants to merge 1 commit intomasterfrom
brandur-soft-stop-timeout

Conversation

@brandur
Copy link
Copy Markdown
Contributor

@brandur brandur commented May 3, 2026

A part of River code that's always bothered me is the graceful shutdown
example [1]. It's really, really complicated, and I think that
complicated orchestration is very likely a sign that our code isn't
quite right.

I initially tried putting the graceful shutdown code into a helper like
river.GracefulShutdown, but I found that also problematic because it's
not super clean to call, and even minor customizations would require a
user to copy out the entire function.

Playing with an LLM a bit, I came up with this alternative: build an
understanding of soft/hard stop into the client's default Stop
function by adding Config.SoftStopTimeout. By default, leaving this
value unconfigured keeps the existing behavior of River today where a
call to Stop waits unbounded time for jobs to finish working. Adding a
SoftStopTimeout value brings in functionality similar to the graceful
shutdown example: stopping proceeds normally waiting for jobs to finish,
but in case they don't inside of SoftStopTimeout, they're cancelled as
if StopAndCancel had been called.

The soft timeout is also respected for a cancelled Start context,
which makes the basic use of River for soft/hard stop very nice (you
don't even need to call Stop anymore):

// Use signal.NotifyContext to cancel the start context on SIGINT/SIGTERM.
// When the signal fires, the client initiates a soft stop. If running jobs
// don't finish within SoftStopTimeout, their contexts are automatically
// cancelled.
ctx, stop := signal.NotifyContext(ctx, syscall.SIGINT, syscall.SIGTERM)
defer stop()

if err := riverClient.Start(ctx); err != nil {
    panic(err)
}

[1] https://github.com/riverqueue/river/blob/master/example_graceful_shutdown_test.go

@brandur brandur marked this pull request as draft May 3, 2026 20:14
A part of River code that's always bothered me is the graceful shutdown
example [1]. It's really, really complicated, and I think that
complicated orchestration is very likely a sign that our code isn't
quite right.

I initially tried putting the graceful shutdown code into a helper like
`river.GracefulShutdown`, but I found that also problematic because it's
not super clean to call, and even minor customizations would require a
user to copy out the entire function.

Playing with an LLM a bit, I came up with this alternative: build an
understanding of soft/hard stop into the client's default `Stop`
function by adding `Config.SoftStopTimeout`. By default, leaving this
value unconfigured keeps the existing behavior of River today where a
call to `Stop` waits unbounded time for jobs to finish working. Adding a
`SoftStopTimeout` value brings in functionality similar to the graceful
shutdown example: stopping proceeds normally waiting for jobs to finish,
but in case they don't inside of `SoftStopTimeout`, they're cancelled as
if `StopAndCancel` had been called.

The soft timeout is also respected for a cancelled `Start` context,
which makes the basic use of River for soft/hard stop very nice (you
don't even need to call `Stop` anymore):

    // Use signal.NotifyContext to cancel the start context on SIGINT/SIGTERM.
    // When the signal fires, the client initiates a soft stop. If running jobs
    // don't finish within SoftStopTimeout, their contexts are automatically
    // cancelled.
    ctx, stop := signal.NotifyContext(ctx, syscall.SIGINT, syscall.SIGTERM)
    defer stop()

    if err := riverClient.Start(ctx); err != nil {
        panic(err)
    }

[1] https://github.com/riverqueue/river/blob/master/example_graceful_shutdown_test.go
@brandur brandur force-pushed the brandur-soft-stop-timeout branch from c52de36 to b7f599c Compare May 3, 2026 20:28
@brandur brandur marked this pull request as ready for review May 3, 2026 20:38
@brandur brandur requested a review from bgentry May 3, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant