[feat][client] Support null value messages#563
Open
Bhargavkonidena wants to merge 5 commits intoapache:mainfrom
Open
[feat][client] Support null value messages#563Bhargavkonidena wants to merge 5 commits intoapache:mainfrom
Bhargavkonidena wants to merge 5 commits intoapache:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds first-class “null value” (tombstone) support to the Pulsar C++ client so producers can emit tombstones on compacted topics and consumers can distinguish null vs empty payloads, including correct handling for batched messages and C bindings.
Changes:
- Introduces
MessageBuilder::setNullValue()andMessage::hasNullValue()(plus C API equivalents) to represent/inspect tombstone messages. - Preserves
null_valuethrough batch message serialization/deserialization. - Adds unit + integration tests covering null value behavior across Message, batch parsing, TableView, and Reader scenarios.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/TableViewTest.cc | Adds integration tests validating TableView tombstone behavior and empty-vs-null scenarios. |
| tests/ReaderTest.cc | Adds integration tests for reading compacted topics with tombstones and tombstone property preservation. |
| tests/MessageTest.cc | Adds unit tests for basic null value vs empty message semantics. |
| tests/BatchMessageTest.cc | Adds unit test ensuring null_value survives batch serialize/deserialize. |
| lib/c/c_Message.cc | Exposes C API wrappers to set/check null value. |
| lib/MessageBuilder.cc | Implements MessageBuilder::setNullValue(). |
| lib/Message.cc | Implements Message::hasNullValue() and copies null_value from SingleMessageMetadata for batch entries. |
| lib/Commands.cc | Serializes null_value into SingleMessageMetadata for batch payloads. |
| include/pulsar/c/message.h | Declares/document C API for null value. |
| include/pulsar/MessageBuilder.h | Declares/documents setNullValue(). |
| include/pulsar/Message.h | Declares/documents hasNullValue(). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
BewareMyPower
left a comment
There was a problem hiding this comment.
Overall LGTM, please fix the code format and address comments from copilot
d23945b to
59a61ab
Compare
BewareMyPower
previously approved these changes
Apr 1, 2026
Made-with: Cursor
The test was triggering compaction but reading immediately without waiting for it to complete. Since compaction is asynchronous, the reader might read uncompacted data. Added polling of the compaction status endpoint to wait until compaction completes before reading. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #489
Motivation
Currently the producer cannot send a null value message, which is used as a tombstone for a specific key on a compacted topic.
In addition, the consumer cannot differentiate empty values from null values, because all these values are presented as empty std::strings.
Modifications
Verifying this change
Unit Tests (no broker required):
MessageTest.testNullValueMessage - Basic null value API functionality
MessageTest.testEmptyMessage - Verifies empty message is NOT a null value
MessageTest.testEmptyStringNotNullValue - Verifies empty string "" is NOT the same as null value
BatchMessageTest.testParseMessageBatchEntryWithNullValue - Verifies null value survives batch serialization/deserialization
Integration Tests (require Pulsar broker):
TableViewTest.testNullValueTombstone - Send tombstone, verify key removed from TableView
TableViewTest.testNullValueVsEmptyString - Compare empty string vs null value behavior
ReaderTest.testReadCompactedWithNullValue - Reader detects tombstones with hasNullValue()
ReaderTest.testNullValueMessageProperties - Null value message preserves properties
Documentation
doc-required(Your PR needs to update docs and you will update later)
doc-not-needed(Please explain why)
doc(Your PR contains doc changes)
doc-complete(Docs have been already added)