fix: add cycle detection to TOML renderer (manifestTomlEx)#963
Open
He-Pin wants to merge 1 commit into
Open
Conversation
Motivation:
std.manifestTomlEx has its own recursive traversal in renderTableInternal
that bypassed the Materializer's IdentityHashMap-based cycle detection.
Cyclic objects like `{a: $}` caused OOM instead of a clean error.
Modification:
- Add depth counter and IdentityHashMap parameters to renderTableInternal
- Check depth against maxMaterializeDepth and detect revisited objects
- Remove from visited set in finally block (backtracking)
- Wrap evalRhs in try-catch for StackOverflowError/OutOfMemoryError
- Add cyclic reference tests for manifestJsonEx, manifestTomlEx, and
manifestYamlDoc using `{a: $}` pattern
Result:
Cyclic objects now produce "Stackoverflow while materializing" error
instead of OOM, consistent with JSON and YAML renderers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
std.manifestTomlExhas its own recursive traversal inrenderTableInternalthat bypassed theMaterializer'sIdentityHashMap-based cycle detection. Cyclic objects like{a: $}caused OutOfMemoryError instead of the clean "Stackoverflow while materializing" error that JSON and YAML renderers produce.Modification
depth: Intandvisited: IdentityHashMap[Val.Obj, Boolean]parameters torenderTableInternal, matching the pattern inMaterializer.MaterializeContextmaxMaterializeDepthand detect revisited objects at method entry; remove from visited set infinallyblock (backtracking)evalRhsin try-catch forStackOverflowError/OutOfMemoryErrormanifestJsonEx,manifestTomlEx,manifestYamlDoc— all using{a: $}patternResult
Cyclic objects now produce "Stackoverflow while materializing" error instead of OOM, consistent with JSON and YAML renderers. All existing TOML tests (array, null, standard) continue to pass.
Cross-implementation comparison
Test case:
std.manifestTomlEx({a: $}, " ")(cyclic self-reference)RUNTIME ERROR: max manifest depth exceeded, possible infinite recursionstack overflow, try to reduce recursion, or set --max-stack to bigger valuejava.lang.OutOfMemoryError: Java heap spacesjsonnet.Error: Stackoverflow while materializing, possibly due to recursive valueKey observations
Test plan
std.manifestTomlEx({a: $}, " ")produces clean error instead of OOMstd.manifestJsonEx({a: $}, ...)continues to work (already had detection)std.manifestYamlDoc({a: $})continues to work (already had detection)