feat(archiver): RpcSyncArchiver that syncs from another node#22732
Draft
spalladino wants to merge 7 commits intomerge-train/spartanfrom
Draft
feat(archiver): RpcSyncArchiver that syncs from another node#22732spalladino wants to merge 7 commits intomerge-train/spartanfrom
spalladino wants to merge 7 commits intomerge-train/spartanfrom
Conversation
…ockStream Introduces a read-only archiver variant that follows an upstream node's data source via L2BlockStream instead of polling L1. Useful for read-only RPC gateways, tools that embed a queryable archiver, or tests that want an archiver without an L1 stack. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
spalladino
added
ci-no-fail-fast
…ollower - handleChainPruned cross-checkpoint branch always emits L2PruneUnproven; previously it fell back to L2PruneUncheckpointed when the proven tip equalled the target, which was wrong (that event is only for proposed-only prunes). - blocksBeingRemoved now starts at the first block of the first checkpoint being removed, not targetBlockNumber + 1, so mid-checkpoint rollback reports the full set of removed blocks and the epoch derived from their slot is correct. - In epochs_sync_after_reorg, the RpcSyncArchiver is now created right after the primary node is up and checked at every checkpoint-number assertion instead of once at the end, avoiding the race against the still-advancing primary. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
L2BlockStream already awaits each handleBlockStreamEvent call before emitting the next event, so the extra serialization layer was dead weight. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sync in RpcSyncArchiver - Close the underlying data store on stop(); the factory is the sole opener so the archiver owns it. - Handle chain-pruned events targeting block 0 explicitly to avoid a hot-loop throw from getCheckpointedBlock(0) returning undefined. - After each block stream batch, compare local and source tips and mark initial sync complete; prevents waitForInitialSync() from hanging when start() was called with blockUntilSync=false. - Add unit tests covering the above plus multi-block checkpoints, zero-message checkpoints, chain-finalized advancement, and partial-insert recovery.
Adds getL1ToL2Messages to the AztecNode interface + JSON-RPC schema so any AztecNode (in-process or remote client) can be used as an RpcSyncArchiver source. Retypes RpcSyncArchiverSource as Pick<AztecNode, ...> to enforce the subset relationship at compile time, and updates the e2e test to pass the AztecNode directly to the factory. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collaborator
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. |
…al data The L2BlockStream short-circuits block 0 comparisons internally, but the local-data provider should still return a real hash rather than undefined so callers that bypass that short-circuit see consistent behaviour. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… source Messages are no longer persisted locally. The archiver does not run an L1 synchronizer, so it cannot observe inbox reorgs or maintain the rolling-hash chain safely; every lookup now goes straight to the upstream source, which is already the source of truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The current
Archiveris heavyweight: it polls L1, validates attestations, fetches blobs, and doubles as anL2BlockSinkfor the sequencer. For read-only consumers that just want a local queryable copy of another node's state (RPC gateways, embedded archivers, test harnesses that don't want to replay L1 sync), all that machinery is overkill.Approach
Added a second archiver variant,
RpcSyncArchiver, that reuses the existing archiver store andArchiverDataSourceBaseread APIs but replaces the L1 synchronizer with anL2BlockStreampointed at an upstreamArchiverDataSource. Events from the stream (blocks-added,chain-checkpointed,chain-pruned,chain-proven,chain-finalized) map onto the existingArchiverDataStoreUpdaterprimitives. L1→L2 messages are pulled via the source'sgetL1ToL2MessagesRPC and the archiver reconstructsInboxMessagerecords locally (indices viaInboxLeaf.smallestIndexForCheckpoint, rolling hashes chained on the fly).Changes
RpcSyncArchiverclass andcreateRpcSyncArchiverfactory. No changes to the existingArchiver. README gains a short "Variants" section.rpc_sync_archiver.test.tsdrives a follower from a real upstreamArchiver + FakeL1Statethrough checkpoint sync, message sync, proven advancement, reorg, and idempotent resync.epochs_sync_after_reorg.test.tsnow spins up anRpcSyncArchiveragainst the primary before the node is stopped and asserts tip parity.