gh-151763: Fix OOM-0013 crash when the parser or compiler fails to allocate#151968
Open
tonghuaroot wants to merge 2 commits into
Open
gh-151763: Fix OOM-0013 crash when the parser or compiler fails to allocate#151968tonghuaroot wants to merge 2 commits into
tonghuaroot wants to merge 2 commits into
Conversation
… to allocate compile(), exec(), eval() and ast.parse() could return NULL without an exception set when an allocation failed, breaking the result/error contract and crashing on the assertion that checks it. Set the error indicator at the points that returned NULL silently: new_compiler(), the _PyPegen_run_parser() result/exception check, and the tokenizer-init helpers.
tonghuaroot
requested review from
iritkatriel,
lysnikolaou,
markshannon and
pablogsal
as code owners
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This fixes OOM-0013 from the gh-151763 umbrella.
compile(),exec(),eval()andast.parse()can return NULL withoutsetting an exception when an allocation fails, breaking the result/error
contract every consumer relies on and crashing on the assertion that checks it
(for example the
(res != NULL) ^ (PyErr_Occurred())check reached from theevaluation loop, and
assert(!PyErr_Occurred())at the top of_PyAST_Validate).Three points returned NULL silently under a failed allocation:
new_compiler()returned NULL without callingPyErr_NoMemory()when itsinitial
PyMem_Callocfailed -- the dominantcompile()/exec()/eval()path.
_PyPegen_run_parser()could return a valid result while leaving a staleexception pending.
Setting the error indicator at each point makes a failed allocation surface as
MemoryErrorinstead of a NULL-without-exception crash.Reproduction (debug build,
_testcapi.set_nomemory)ast.parse(...)underset_nomemoryaborted with "a function returned NULLwithout setting an exception"; a clean
MemoryErroris raised after the fix.compile()/exec()/eval()underset_nomemoryaborted at theresult/error contract assertion; clean after the fix.
No test is added, following the convention for these OOM crash fixes.