Skip to content

add support for missing marker for bytes field in TypedDict#25

Merged
giograno merged 1 commit intomainfrom
bytes-marker
Mar 27, 2026
Merged

add support for missing marker for bytes field in TypedDict#25
giograno merged 1 commit intomainfrom
bytes-marker

Conversation

@bentsku
Copy link
Copy Markdown

@bentsku bentsku commented Mar 26, 2026

While working on SQS, I realized we had an issue with typed dict with bytes field and the missing marker, because Avro could not determine unions of bytes and string, it does not have a way to choose between the 2 types.

And we'd get this error:

>   ???
E   UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte

This PR implements behavior to properly generate schema and a new Missing marker type in bytes. Serialization and deserialization logic will need to be updated to take this into account.

I've already re-generated the schemas and validated that everything is working properly with this version

@bentsku bentsku self-assigned this Mar 26, 2026
@bentsku bentsku requested a review from giograno March 26, 2026 20:25
@bentsku bentsku marked this pull request as ready for review March 26, 2026 20:28
Copy link
Copy Markdown
Member

@giograno giograno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this and guiding me through the issue 💯
Luckily, this did not cause a big disruption for our schemas!

@giograno giograno merged commit b76e14c into main Mar 27, 2026
6 checks passed
@giograno giograno deleted the bytes-marker branch March 27, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants