sync: implement fast sync / state sync for new nodes

[TheNewAutonomy]

Goal

Make it practical for a brand-new node to join a large network without replaying everything from genesis.

Milestone A plan

This issue is the epic for the reliable-join workstream.

Sub-tickets (Milestone A)

• sync: snapshot advertisement format + RPC discoverability #226 snapshot advertisement format + RPC discoverability
• sync: snapshot download + verify + restore path #227 snapshot download + verify + restore path
• sync: catch-up to head after snapshot restore (bounded time) #228 catch-up to head after snapshot restore (bounded time)
• p2p: basic envelope safety limits (payload caps, TTL/hops, rate limiting) #229 basic envelope safety limits (payload caps, TTL/hops, rate limiting)

Scope

• Define sync modes: full, snapshot-based, state-root/proof-based
• Ensure correctness: verify state roots / proofs
• RPC/CLI UX for selecting sync strategy

Acceptance criteria

• A node can sync from scratch to head within a bounded time on WAN testnet
• Sync correctness is verified via state root checks
• Abuse resistance: sync/join does not allow unbounded resource growth

[TheNewAutonomy]

Added snapshot-based sync scaffolding in 4fc5ef8.

• New CLI commands to support snapshot-style sync workflows:
  • catalyst-cli db-backup --data-dir <dir> --out-dir <backup_dir>
  • catalyst-cli db-restore --data-dir <dir> --from-dir <backup_dir>

Commit: 4fc5ef8

[TheNewAutonomy]

Progress: made snapshot-based fast sync workflow more usable in 7cdc091.

• db-backup now writes catalyst_snapshot.json into the snapshot directory (chain identity + applied head metadata).
• db-restore verifies chain_id and genesis_hash against catalyst_snapshot.json when present.
• RPC: added catalyst_getSyncInfo returning {chain_id, network_id, genesis_hash, head} for fast-sync verification.
• Docs: added docs/sync-guide.md describing the workflow.

Commit: 7cdc091

[TheNewAutonomy]

Implemented the next step of fast-sync as a network-usable workflow.

• Added RPC method catalyst_getSnapshotInfo (operator-published snapshot advertisement: archive URL + sha256 + bytes + chain identity + head).
• Extended CLI db-backup with --archive <path> to package the exported snapshot directory into a tar archive (prints sha256/bytes).
• Added CLI snapshot-publish to store the latest snapshot advertisement in the node DB (served via RPC).
• Added CLI sync-from-snapshot to download the published tar archive, verify sha256, extract, and run db-restore locally.
• Updated docs/sync-guide.md with the new flow.

Commit: 62f6566

[TheNewAutonomy]

Added operator-friendly snapshot lifecycle automation.

• New CLI command snapshot-make-latest:
  • creates a timestamped snapshot export + tar archive in --out-base-dir
  • publishes/overwrites the latest snapshot ad (served by catalyst_getSnapshotInfo)
  • cleans up older snapshot dirs + tar files with --retain N
• Updated docs/sync-guide.md with the recommended flow.

Commit: 0607f2d

[TheNewAutonomy]

Added the final operator-facing pieces for snapshot distribution.

• RpcSnapshotInfo now includes published_at_ms and optional expires_at_ms (backwards-compatible via serde defaults).
• CLI snapshot-publish and snapshot-make-latest accept --ttl-seconds to set expiry.
• New CLI long-running sidecar snapshot-serve --dir ... --bind ...:
  • serves *.tar archives over HTTP
  • supports Range: bytes=... for resumable downloads
  • intended as a lightweight alternative to nginx/caddy
• Updated docs/sync-guide.md with the sidecar flow.

Commit: 5a567ef

[TheNewAutonomy]

Field report from WAN testnet: EU RPC node at head (applied_cycle=29526391) but missing older cycles (getBlockByNumber(29526355)=>null). This creates visible block gaps in explorers and stalls getBlocksByNumberRange pagination at the first missing cycle.

Tracking in #228 (catch-up/backfill) and may require RPC/indexer adjustments.

[TheNewAutonomy]

Milestone A reliable-join epic is complete.

Implemented and shipped:

• snapshot advertisement + RPC discoverability (sync: snapshot advertisement format + RPC discoverability #226)
• snapshot download + verify + atomic restore (sync: snapshot download + verify + restore path #227)
• catch-up/backfill after restore (sync: catch-up to head after snapshot restore (bounded time) #228)
• P2P safety limits + TTL/relay bounds (p2p: basic envelope safety limits (payload caps, TTL/hops, rate limiting) #229)
• prev_state_root continuity gating to prevent gaps (sync: enforce prev_state_root continuity (no apply onto unknown root) #230)

Remaining storage growth policy work is tracked separately (see #203).