Skip to content

gh-149564: Propagate call-site hotness to callees via dynamic exits#149575

Open
anujbharambe wants to merge 3 commits intopython:mainfrom
anujbharambe:gh-149564-jit-dynamic-exit-boost
Open

gh-149564: Propagate call-site hotness to callees via dynamic exits#149575
anujbharambe wants to merge 3 commits intopython:mainfrom
anujbharambe:gh-149564-jit-dynamic-exit-boost

Conversation

@anujbharambe
Copy link
Copy Markdown
Contributor

@anujbharambe anujbharambe commented May 8, 2026

Summary

  • When a hot loop calls many distinct exec()-generated functions, the loop trace fires _DYNAMIC_EXIT on each call because the function version guard fails for each different callee. Each callee's RESUME_CHECK_JIT counter starts high and only decrements by 1 per call, so with ~60 calls per callee, the counter never reaches zero and the callee never gets its own executor.
  • This PR fixes the issue by boosting the callee's RESUME counter to trigger immediately when _DYNAMIC_EXIT or _COLD_DYNAMIC_EXIT lands on a RESUME_CHECK_JIT instruction.
    Being called from a hot trace via dynamic exit is strong evidence the callee deserves compilation.

Changes

  • Python/bytecodes.c: Added counter boost logic to _DYNAMIC_EXIT and _COLD_DYNAMIC_EXIT
  • Python/executor_cases.c.h: Regenerated
  • Lib/test/test_capi/test_opt.py: Added test_dynamic_exit_boosts_resume

Risks

  • Memory: Compiling many small callees increases memory usage. Acceptable since the backoff mechanism prevents runaway recompilation on failure.
  • Overhead: The if (target->op.code == RESUME_CHECK_JIT) check is a single byte comparison on the dynamic exit path (already cold/rare). Negligible cost.

AI Disclosure: Some parts of the code and this PR were written with the help of Claude(AI).

Copy link
Copy Markdown
Member

@picnixz picnixz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not very familiary with this part of the code, but I don't like having an LLM touch the bytecode properly. In addition, I don't think dynamic exits are really common apart from specific use cases, and we need to see how it affects regular code (we are still adding checks). So I would suggest that we first discuss whether the feature is acceptable before creating a PR, especially with LLM. This is valid for any issues: don't create PRs unless we reached a consensus.

Comment thread Python/bytecodes.c
// If we're landing on a callee's RESUME, boost its counter so it
// gets traced sooner (it was called from a hot trace).
if (target->op.code == RESUME_CHECK_JIT) {
target[1].counter = trigger_backoff_counter();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can it "boost" its counter? This sets its counter to 0 here. So it resets it. Not boost it.

@bedevere-app
Copy link
Copy Markdown

bedevere-app Bot commented May 9, 2026

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants