fix: auto-start workers in context manager; warn on RUNNING+failed tasks#391
Open
nthmost-orkes wants to merge 4 commits intomainfrom
Open
fix: auto-start workers in context manager; warn on RUNNING+failed tasks#391nthmost-orkes wants to merge 4 commits intomainfrom
nthmost-orkes wants to merge 4 commits intomainfrom
Conversation
… tasks - TaskHandler.__enter__ now calls start_processes() so `with TaskHandler(...) as h:` works out of the box without a separate h.start_processes() call. start_processes() is idempotent via _processes_started guard, so existing code that calls it explicitly inside the with-block is unaffected. Fixes conductor-oss/getting-started#42. - WorkflowExecutor.execute() and execute_workflow() now log a WARNING when the workflow returns RUNNING after the wait timeout but a task is already in FAILED/FAILED_WITH_TERMINAL_ERROR state, surfacing the failure reason that would otherwise be invisible to the caller. Fixes conductor-oss/getting-started#41. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The conductor_server.sh one-liner downloaded a 606 MB JAR to the current directory with no progress indicator and ran in the foreground. Replace it with the `conductor server start` CLI approach, which handles download location, caching, and UX correctly. Fixes conductor-oss/getting-started#47 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
v1r3n
reviewed
Apr 16, 2026
| logger.info("TaskHandler initialized") | ||
|
|
||
| def __enter__(self): | ||
| self.start_processes() |
Contributor
There was a problem hiding this comment.
what are the implications of auto starting the process?
Contributor
Author
There was a problem hiding this comment.
Good question -- mostly pros in my reckoning but i'll take a second look.
- Fixes the context manager contract. with-as-no-op is a Python anti-pattern. File handles, locks, threads, subprocess all acquire on enter. TaskHandler was the odd one.
- Fixes a real user bug — Python SDK: TaskHandler context manager silently starts 0 workers if start_processes() is not called explicitly getting-started#42. Users wrote with TaskHandler(...) as h: expecting it to run and got silence.
- Idempotency guard preserves every existing caller. Anyone doing with h: h.start_processes() is unaffected — second call is a no-op.
- stop_processes() resets the flag, so stop → restart works. (Onboard repo to PyPI (Python Package Index) #3 from last turn is handled.)
I'll go look at some things that come to mind... will update
If start_processes() raised partway through spawning workers, the flag was already True, causing a subsequent retry call to be a silent no-op. Moving the assignment to the end means a failed start remains retryable.
Python skips __exit__ when __enter__ raises, so an exception mid-startup would leak any workers already spawned. Wrap the call so stop_processes() runs before the exception propagates.
Contributor
Author
|
Changes prompted by this review: Thinking through the implications surfaced two edge cases worth hardening, both now pushed as follow-up commits:
For the main behavior change itself:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TaskHandlercontext manager auto-starts workers (__enter__now callsstart_processes()), sowith TaskHandler(...) as h:works correctly without a separateh.start_processes()call.start_processes()gains an idempotency guard (_processes_startedflag) so existing code that calls it explicitly inside thewithblock is unaffected. Fixes Python SDK: TaskHandler context manager silently starts 0 workers if start_processes() is not called explicitly getting-started#42.WorkflowExecutor.execute()warns when workflow is RUNNING but a task has already failed — whenexecute()returns aRUNNINGrun after the wait timeout expires, it now does a follow-upget_workflow()call, finds anyFAILED/FAILED_WITH_TERMINAL_ERRORtasks, and logs aWARNINGwith the task reference name andreason_for_incompletion. This surfaces failure details that would otherwise be invisible (workflow stays RUNNING while the failed task awaits retries). Fixes Python SDK: worker exception causes silent stuck RUNNING workflow — execute() returns RUNNING with no error getting-started#41.Test plan
TestTaskHandlerContextManager::test_context_manager_enter— verifieswith handler as h: h is handlerTestTaskHandlerContextManager::test_context_manager_exit— verifiesstop_processes()is called on exit (updated to patchProcesssince__enter__now starts workers)python3 -m pytest tests/unit/— 440 passed🤖 Generated with Claude Code