Skip to content

Speed up TLV serialization length accounting#4642

Open
joostjager wants to merge 3 commits into
lightningdevkit:mainfrom
joostjager:ser-tlv-length-fastpath
Open

Speed up TLV serialization length accounting#4642
joostjager wants to merge 3 commits into
lightningdevkit:mainfrom
joostjager:ser-tlv-length-fastpath

Conversation

@joostjager
Copy link
Copy Markdown
Contributor

Recent splice fuzzing coverage made it clear that some chanmon consistency inputs spend a surprising amount of time encoding snapshots. This PR tightens the serialization length path used by those snapshots.

Slow fuzz byte string used while checking this:

a1a0a3ffa2a1ffffa2a1ff00a2a1ffa2a1a0a3ffa2a1ff19ffffffffffffffac

For that byte string, median real time over 5 runs with target output suppressed:

Profile Pre changes Post changes
Debug 17.17s 3.61s
Release 0.60s 0.24s

Add an environment-variable switch that lets stdin fuzz targets use the
dev-null test logger. This keeps direct invocations verbose by default,
while external runners can opt into quieter passing-case replays.
@ldk-reviews-bot
Copy link
Copy Markdown

ldk-reviews-bot commented May 28, 2026

👋 Thanks for assigning @wpaulino as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@joostjager joostjager requested a review from wpaulino May 28, 2026 14:28
@joostjager joostjager marked this pull request as ready for review May 28, 2026 14:28
@ldk-claude-review-bot
Copy link
Copy Markdown
Collaborator

ldk-claude-review-bot commented May 28, 2026

No issues found.

I performed a thorough line-by-line review of all substantive changes:

Serialization macros (lightning/src/util/ser_macros.rs):

  • _bigsize_serialized_length! — verified ranges match BigSize::write() at ser.rs:512-524 (1/3/5/9 bytes for 0xFC/0xFFFF/0xFFFFFFFF/larger boundaries)
  • _tlv_record_serialized_length! — correctly sums type BigSize + value-length BigSize + value data
  • _get_varint_length_prefixed_tlv_length! — all 15 field-type arms verified against their _encode_tlv! counterparts, including delegation chains (default_valuerequired, default_value_vecrequired_vec, upgradable_requiredrequired, etc.)
  • _tlv_fields_serialized_length! — correctly computes bigsize_prefix_len + tlv_data_len, matching _encode_varint_length_prefixed_tlv!'s write structure
  • Reference consistency (& prefix) verified across impl_writeable_tlv_based! and impl_writeable_only_tlv_fields! — both write and length paths produce &self.$field
  • extra_tlvs handling in _encode_varint_length_prefixed_tlv! is consistent between length and write paths
  • Test tlv_length_counting_matches_written_bigsize_boundaries covers 42 combinations across all BigSize boundaries

DevNull logger (fuzz/src/utils/test_logger.rs):

  • TypeId::of::<Out>() comparison is monomorphized at compile time — correct zero-cost optimization
  • Early return correctly avoids both writeln! formatting and I/O

Fuzz targets (~60 files + template):

  • All stdin targets consistently updated with LDK_FUZZ_SUPPRESS_LOGS env var, no targets missed
  • Remaining Stdout references are all in the else branch (line 77)

Add direct serialized length implementations for common serialization
wrappers and use direct BigSize length accounting when computing TLV
record lengths. This avoids routing length calculations back through
in-memory writers for common nested serialization paths.

Cover the BigSize boundary values used by TLV record length counting
by comparing the direct calculation against actual serialized TLV
records.
@joostjager joostjager force-pushed the ser-tlv-length-fastpath branch from 7afe504 to 084d0ac Compare May 28, 2026 15:21
@joostjager joostjager self-assigned this May 28, 2026
Comment thread lightning/src/util/ser_macros.rs Outdated
} }
}

macro_rules! impl_writeable_tlv_fields {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we already have existing macros that implement both Writeable and Readable with the name impl_writeable_*, we might want to rename this slightly

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed

@ldk-reviews-bot
Copy link
Copy Markdown

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

Add a write-only TLV helper macro that emits both write and
serialized_length from the same field list. Reuse the shared TLV length
helper from impl_writeable_tlv_based so the existing generated writer
path and the new custom-read path stay aligned.

Use the new helper for the hot channel funding and commitment
transaction TLV writers while leaving their custom read implementations
unchanged.
@joostjager joostjager force-pushed the ser-tlv-length-fastpath branch from 084d0ac to b0e52bd Compare May 28, 2026 17:41
@joostjager joostjager requested a review from wpaulino May 28, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

4 participants