From ec28ac25b0a195772373811e8bce50795cf8e7af Mon Sep 17 00:00:00 2001 From: Rohan Devasthale Date: Wed, 29 Apr 2026 14:29:44 -0400 Subject: [PATCH] docs(proposal): add iterative bootstrap proposal Propose converting the recursive bootstrapper to an iterative approach using an explicit work stack to eliminate recursion depth limits. Co-Authored-By: Claude Opus 4.6 (1M context) Signed-off-by: Rohan Devasthale --- docs/proposals/index.rst | 1 + docs/proposals/iterative-bootstrap.rst | 60 ++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) create mode 100644 docs/proposals/iterative-bootstrap.rst diff --git a/docs/proposals/index.rst b/docs/proposals/index.rst index 75c368b0..f6786064 100644 --- a/docs/proposals/index.rst +++ b/docs/proposals/index.rst @@ -4,6 +4,7 @@ Fromager Enhancement Proposals .. toctree:: :maxdepth: 1 + iterative-bootstrap new-patcher-config new-resolver-config release-cooldown diff --git a/docs/proposals/iterative-bootstrap.rst b/docs/proposals/iterative-bootstrap.rst new file mode 100644 index 00000000..4567db1b --- /dev/null +++ b/docs/proposals/iterative-bootstrap.rst @@ -0,0 +1,60 @@ +Proposal: Convert Bootstrapper from Recursive to Iterative +========================================================== + +Problem +------- + +The ``Bootstrapper`` class processes dependency trees using recursive calls. +When bootstrapping packages with deep or wide dependency graphs — especially +with ``--multiple-versions`` enabled — this hits Python's recursion depth limit +and causes stack overflow errors. The recursion occurs at two points: processing +install dependencies after building a package, and processing build +dependencies before building. + +Proposed Solution +----------------- + +Replace the recursive depth-first traversal with an iterative approach using an +explicit work stack. Each package version to bootstrap is represented as a +``BootstrapWorkItem`` dataclass that flows through five linear phases: + +1. **START** — Add to dependency graph and check if already seen (early exit if so) +2. **EXTRACT_BUILD_DEPS** — Collect build dependencies and push them onto the stack +3. **BUILD_PACKAGE** — Build sdist and/or wheel, record in build order +4. **EXTRACT_INSTALL_DEPS** — Extract install dependencies and push onto the stack +5. **COMPLETE** — Clean up build directories + +The LIFO stack ensures depth-first traversal order is preserved. Build +dependencies are pushed in reverse order so they pop in the correct sequence +(BUILD_SYSTEM before BUILD_BACKEND before BUILD_SDIST). Each dependency +completes all five phases before its parent continues to the next phase. + +This pattern already exists in the codebase: ``dependency_graph.py`` and +``commands/graph.py`` both use stack-based DFS traversals. + +Scope +----- + +This is a pure refactoring — no behavior changes. The external interface +(``bootstrap()`` method signature, CLI flags, output files) remains identical. +All existing error handling modes (normal fail-fast, ``--test-mode``, +``--multiple-versions``) are preserved. Progress bar semantics are unchanged. + +The change is limited to ``src/fromager/bootstrapper.py``. No other files +require modification. All existing tests must pass without changes. + +Benefits +-------- + +- Eliminates recursion depth limits entirely (work stack lives on the heap) +- Handles arbitrarily deep and wide dependency graphs +- Improves debuggability with explicit state in work items +- Follows proven patterns already established in the codebase + +Verification +------------ + +- All existing unit and e2e tests must pass unchanged +- Bootstrap output for the same inputs must be identical +- Performance should remain within 10% of the recursive version +- Rollback via ``git revert`` if issues are discovered post-merge