-
-
Notifications
You must be signed in to change notification settings - Fork 34.1k
gh-142349: Fix refcount corruption in lazy import specialization #144733
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
3007570 to
239c717
Compare
Yhg1s
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this is good enough for a free-threaded build. The error case happens when _PyImport_LoadLazyImportTstate is called with a borrowed reference. The GIL won't be protecting that borrowed reference in a free-threaded build, so the INCREF already happens too late. We need the caller to acquire an owned reference instead, in a safe way. But... where does this happen? As far as I can tell the callers all own the reference they pass.
If that were true then there would not be crashes no? |
Unless the problem is actually something different, yes :) I would like to know which it is. |
ca6f0f9 to
2fe3763
Compare
_PyDict_LookupIndexAndValue() returns a borrowed reference via _Py_dict_lookup(), but specialize_load_global_lock_held() called Py_DECREF(value) on it when bailing out for lazy imports. Each time the adaptive counter fired while a lazy import was still in globals, this stole one reference from the dict's object. With 8+ threads racing through LOAD_GLOBAL during concurrent lazy import resolution, enough triggers accumulated to drive the refcount to zero while the dict and other threads still referenced the object, causing use-after-free.
2fe3763 to
c7defe2
Compare
Narrator: and the problem was indeed something different. Kind of |
_PyDict_LookupIndexAndValue() returns a borrowed reference via _Py_dict_lookup(), but specialize_load_global_lock_held() called Py_DECREF(value) on it when bailing out for lazy imports. Each time the adaptive counter fired while a lazy import was still in globals, this stole one reference from the dict's object. With 8+ threads racing through LOAD_GLOBAL during concurrent lazy import resolution, enough triggers accumulated to drive the refcount to zero while the dict and other threads still referenced the object, causing use-after-free.