marshal.loads() crashes on a back-referenced dict when an allocation fails (3.15 regression)

[tonghuaroot]

Crash report

marshal.loads() crashes (SIGSEGV on a release build, an assertion abort on a debug build) when it loads a back-referenced dictionary — one marshalled with FLAG_REF, e.g. a dict that appears more than once or under a referenced parent — and the interpreter is under memory pressure. A single allocation failure inside marshal's reference-tracking turns into a NULL dereference instead of a clean, catchable MemoryError, so a program that correctly handles MemoryError can still be crashed (or left with a corrupted interpreter) by a marshal stream loaded under low memory.

Reproduction (debug build)

import _testcapi, marshal

# TYPE_DICT|FLAG_REF payload that decodes to {1: 2}; FLAG_REF makes
# marshal reference-track the dict, which is what triggers the bug.
payload = b'\xfbi\x01\x00\x00\x00i\x02\x00\x00\x000'
assert marshal.loads(payload) == {1: 2}

_testcapi.set_nomemory(0, 1)          # fail the next allocation
try:
    marshal.loads(payload)            # -> SIGSEGV / assertion abort
finally:
    _testcapi.remove_mem_hooks()

$ ./python crash.py
Segmentation fault

faulthandler / lldb pin the crash to PyDict_SetItem() being called with a NULL dict, from r_object() in Python/marshal.c. The same fault fires under a genuine MemoryError (no test API) when marshal's reference-tracking allocation is the one that fails. Loading a non-back-referenced dict, or a back-referenced tuple/list/set, under the same single-allocation failure raises a clean MemoryError — so the crash is specific to the dict path.

Root cause

In Python/marshal.c, r_object(), the TYPE_DICT branch registers the new dict for back-references with R_REF(v):

v = PyDict_New();
if (v == NULL) {
    break;
}
if (type == TYPE_DICT) {
    R_REF(v);                 // can set v = NULL (r_ref() -> PyList_Append() fails)
}
...
for (;;) {
    ...
    if (PyDict_SetItem(v, key, val) < 0) {   // v may be NULL here -> crash

R_REF(v) expands to if (flag) v = r_ref(v, flag, p);, and r_ref() returns NULL (after decref'ing its argument) when its internal PyList_Append() fails. Unlike the sibling container branches — TYPE_TUPLE, TYPE_LIST, TYPE_FROZENDICT, and the set/frozenset branches all follow R_REF(v) with an if (v == NULL) break; (or if (idx < 0)) guard — the TYPE_DICT branch has none, so it falls through to PyDict_SetItem(NULL, ...). It is the only unguarded R_REF(v) in r_object().

Suggested fix

Add the same if (v == NULL) break; guard after R_REF(v) in the TYPE_DICT branch, mirroring the sibling branches. r_ref() has already decref'd the dict on failure, so the early break propagates the MemoryError cleanly.

Affected versions

main (3.16) and 3.15 only — this is a regression. The if (v == NULL) break; guard was present in the TYPE_DICT branch until 2e37d83 (gh-148653, "Fix some marshal errors related to recursive immutable objects"), which refactored the TYPE_DICT/TYPE_FROZENDICT branches to share code and dropped the guard from the dict path. It first shipped in v3.15.0b1, so no released version is affected; 3.13 and 3.14 still have the guard. The fix should be backported to 3.15.

Note

marshal is documented as not intended for use with untrusted data, so this is a robustness / crash bug rather than a security vulnerability.

Linked PR

A PR follows.

Linked PRs

• gh-151895: Fix marshal.loads() crash on dict reference-tracking failure #151896