fix(llm): apply structured response defaults#6444
Open
u9g wants to merge 5 commits into
Open
Conversation
theomonnom
reviewed
Jul 15, 2026
Comment on lines
-121
to
-132
| # Treat any parameter with a default value as optional. If the parameter’s type doesn't | ||
| # support None, the default will be used instead. | ||
| t = json_schema.get("type") | ||
| if isinstance(t, str): | ||
| json_schema["type"] = [t, "null"] | ||
|
|
||
| elif isinstance(t, list): | ||
| types = t.copy() | ||
| if "null" not in types: | ||
| types.append("null") | ||
|
|
||
| json_schema["type"] = types |
Member
There was a problem hiding this comment.
Can you verify it doesn't break existing behaviour? I think the reasoning behind that is that users using default values on parameters don't want to require the LLMs to generate a value for it
Contributor
Author
There was a problem hiding this comment.
Updated in 80a6538. I kept the existing default-as-null schema behavior, so the LLM is still not required to generate a value it cannot know. The new validate_response_format helper recursively replaces null with the Pydantic default before validation, while preserving null for fields whose declared type is actually nullable. The focused response-format and tool suites pass (101 tests).
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
u9g
force-pushed
the
fix/response-format-default-types
branch
from
b960f6c to
2ba5b76
Compare
u9g
changed the title
…ema_defaults The helper is not specific to response formats; prepare it for reuse by tool argument preparation.
Tool argument preparation only replaced null with the parameter default for top-level parameters, so a defaulted non-nullable field inside a nested model argument failed validation. Reuse the recursive _inject_schema_defaults pass (shared with structured responses) instead of walking the signature. A null for a required parameter with no default is now rejected by pydantic validation instead of a hand-written ValueError; both surface as ToolError.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Context
Strict JSON schemas require every property and do not support Python defaults. LiveKit therefore represents a defaulted field as nullable: null means use the Python default. This lets the LLM avoid generating a value it cannot know.
Tool execution already resolves that sentinel before validation. Structured responses did not, so direct Pydantic validation could reject a null that was valid according to the generated schema. This change keeps the existing schema behavior and adds validate_response_format as the matching middleware step: it recursively replaces null with the declared default only when the original field type is non-nullable, then performs Pydantic validation.
Testing