sdk: add topology support to Python/TypeScript SDKs for RFC-18 by ben-malbeclabs · Pull Request #3514 · malbeclabs/doublezero

ben-malbeclabs · 2026-04-10T04:44:55Z

RFC-18 flex-algo · PR 4 of 5 · see rfcs/rfc-0018-flex-algo.md
Depends on: #3512 (PR 2), #3513 (PR 3)
Series: #3497 · #3512 · #3513 · #3514 · #3515

Summary of Changes

Python SDK: deserialize TopologyConstraint, TopologyInfo, and FlexAlgoNodeSegment; read flex_algo_node_segments from the RFC-18 InterfaceV3 account format
TypeScript SDK: deserialize FlexAlgoNodeSegment from InterfaceV3 account format; guard the segments loop against pre-RFC-18 mainnet accounts where the segment count reads garbage bytes
SDK fixtures: regenerate device, link, tenant, and user fixtures to include the new InterfaceV3 fields
TypeScript RPC client: remove the AbortController timeout wrapper around fetch (was surfacing as spurious TimeoutError on slow getProgramData responses); bump the compat test's request timeout to 120s
tryReadString in borsh-incremental: check remaining buffer length before reading to avoid out-of-bounds reads
CHANGELOG / e2e compatibility test: document the mandatory CLI upgrade boundary for RFC-18 InterfaceV3 (devices written by the new program cannot be deserialized by pre-RFC-18 CLI versions)

Note: activator changes that were originally planned for this PR have been dropped — the doublezero activator is deprecated.

Diff Breakdown

Category	Files	Lines (+/-)
Python SDK	1	+61 / -1
TS SDK	3	+27 / -3
Fixtures	5	+4 / -1 + bin
Docs/e2e	2	+2 / -2

Testing Verification

Python SDK: uv run pytest passes under sdk/serviceability/python/, including fixture deserialization
TypeScript SDK: bun test passes under sdk/serviceability/typescript/, including compat fixture deserialization with the new InterfaceV3 fields
cargo check --workspace clean

…y processors (#3497) RFC-18 flex-algo · PR 1 of 5 · see `rfcs/rfc-0018-flex-algo.md` Series: #3497 · #3512 · #3513 · #3514 · #3515 ## Summary of Changes - Adds `TopologyInfo` onchain account (RFC-18 flex-algo) with `admin_group_bit`, `flex_algo_number`, and `TopologyConstraint`; creates `AdminGroupBits` resource extension singleton for bit allocation - Adds four new foundation-only instructions (107–110): `CreateTopology`, `DeleteTopology`, `ClearTopology`, `BackfillTopology` - Extends `Link` with `link_topologies` (Vec<Pubkey>) and `link_flags` (LINK_FLAG_UNICAST_DRAINED); extends `Tenant` with `include_topologies`; adds `flex_algo_node_segments` to Interface V2 - `CreateLink` now requires the unicast-default topology account and auto-tags the link into it - `UpdateLink` gains foundation-gated `link_topologies` update and contributor-gated `unicast_drained` flag ## Diff Breakdown | Category | Files | Lines (+/-) | Net | |-------------|-------|---------------|-------| | Core logic | 22 | +1,306 / -38 | +1,268| | Scaffolding | 49 | +133 / -3 | +130 | | Tests | 27 | +4,592 / -74 | +4,518| The scaffolding is mechanical struct field additions (`link_topologies: vec![]`, `include_topologies: vec![]`, `flex_algo_node_segments: vec![]`) across CLI, SDK, activator, and client to keep the workspace compiling. Core logic is concentrated in the new topology processors, state types, and link/activate updates. <details> <summary>Key files (click to expand)</summary> - `smartcontract/programs/doublezero-serviceability/src/state/interface.rs` — updates Interface V2 (discriminant 1) to include `flex_algo_node_segments: Vec<FlexAlgoNodeSegment>`; V1 accounts (pre-CYOA format) are left as-is; pre-RFC-18 V2 accounts on mainnet are upgraded in-place by `MigrateDeviceInterfaces` before this deserialization path is used - `smartcontract/programs/doublezero-serviceability/src/state/topology.rs` — new `TopologyInfo` account: `admin_group_bit` (u8), `flex_algo_number` (128+bit), `TopologyConstraint` (IncludeAny/Exclude) - `smartcontract/programs/doublezero-serviceability/src/processors/topology/` — five new processors: create (allocates AdminGroupBit, validates IdRange 1–127), delete, clear (removes topology from all links), backfill (allocates FlexAlgoNodeSegment on all device interfaces) - `smartcontract/programs/doublezero-serviceability/src/processors/link/update.rs` — adds foundation-gated `link_topologies` update (validates each topology account onchain) and contributor-gated `unicast_drained` flag (LINK_FLAG_UNICAST_DRAINED bit 0) - `smartcontract/programs/doublezero-serviceability/src/processors/link/activate.rs` — requires `unicast_default_topology_account` as a mandatory account; auto-tags new link into the unicast-default topology on activation - `smartcontract/programs/doublezero-serviceability/src/processors/globalconfig/set.rs` — creates `AdminGroupBits` ResourceExtension PDA on global config initialization - `smartcontract/programs/doublezero-serviceability/src/state/link.rs` — adds `link_topologies: Vec<Pubkey>`, `link_flags: u64`, and `LINK_FLAG_UNICAST_DRAINED = 0x01` - `smartcontract/programs/doublezero-serviceability/src/instructions.rs` — registers four new instruction variants (CreateTopology=107, DeleteTopology=108, ClearTopology=109, BackfillTopology=110) </details> ## Testing Verification - `make rust-lint` and `make rust-test` pass clean on this branch - `topology_test.rs` (~2,400 lines) covers all four topology processors: create/delete/clear/backfill, including error paths (duplicate names, out-of-range bits, unauthorized signers, invalid topology accounts), segment allocation, and multi-topology backfill - Existing `link_wan_test.rs`, `tenant_test.rs`, and telemetry tests updated to account for the new mandatory `unicast_default_topology_account` in `ActivateLink` --------- Co-authored-by: Greg Mitchell <greg@malbeclabs.com>

RFC-18 flex-algo · PR 2 of 5 · see `rfcs/rfc-0018-flex-algo.md` Series: #3497 · #3512 · #3513 · #3514 · #3515 ## Summary of Changes - Adds five `doublezero link topology` subcommands: `create`, `delete`, `clear`, `backfill`, `list` - `topology create` allocates the admin-group bit and flex-algo number onchain, then automatically backfills `FlexAlgoNodeSegment` entries on all activated Vpnv4 loopback interfaces - `topology backfill` and `topology clear` batch at 16 accounts per transaction to stay within Solana's 32-account limit; zero accounts → zero transactions sent - `topology list` displays each topology's name, admin-group bit, flex-algo number, BGP color community, constraint, and count of tagged links - Backs each subcommand with a Rust SDK command in `doublezero_sdk::commands::topology` - Simplifies the `BackfillTopology` onchain processor: removes the two-pass SR ID pre-marking loop; keeping the `SegmentRoutingIds` resource in sync with off-chain allocations is the activator's responsibility ## Diff Breakdown | Category | Files | Lines (+/-) | Net | |-------------|-------|----------------|--------| | Core logic | 11 | +1,572 / -32 | +1,540 | | Scaffolding | 8 | +91 / -6 | +85 | | Tests | 1 | +25 / -16 | +9 | Core logic includes inline unit tests in the SDK command files. <details> <summary>Key files (click to expand)</summary> - `smartcontract/sdk/rs/src/commands/topology/create.rs` — creates topology, then enumerates devices and calls `BackfillTopologyCommand` for those with Vpnv4 loopbacks; returns `CreateTopologyResult { signature, topology_pda, backfill_signatures }` - `smartcontract/sdk/rs/src/commands/topology/backfill.rs` — batches devices in chunks of 16; returns `Vec<Signature>`; empty device list sends no transactions - `smartcontract/sdk/rs/src/commands/topology/clear.rs` — batches links in chunks of 16; returns `Vec<Signature>`; empty link list sends no transactions - `smartcontract/sdk/rs/src/commands/topology/list.rs` — fetches all `TopologyInfo` accounts and cross-references link accounts to compute per-topology link counts - `smartcontract/cli/src/topology/create.rs` — CLI wrapper; prints PDA and backfill transaction count - `smartcontract/cli/src/topology/list.rs` — tabular display of name, bit, algo number, BGP color, constraint, link count - `smartcontract/programs/doublezero-serviceability/src/processors/topology/backfill.rs` — removes two-pass SR ID pre-marking; single-pass allocation only </details> ## Testing Verification - `cargo test -p doublezero_sdk -p doublezero_cli` passes - `cargo clippy -- -D warnings` clean - `make rust-fmt` applied

migrate command (#3513) RFC-18 flex-algo · PR 3 of 5 · see `rfcs/rfc-0018-flex-algo.md` Depends on: #3512 (PR 2) Series: #3497 · #3512 · #3513 · #3514 · #3515 ## Summary of Changes - Extends `doublezero link get/list/update` to display topology assignments (`link_topologies`) and drain status; adds `--link-topology` (comma-separated topology names, `default` to clear all) and `--unicast-drained` flags to `link update`; adds `--topology` filter to `link list` - Extends `doublezero tenant get/list/update` to display included topologies (`include_topologies`) and adds `--include-topologies` (comma-separated topology names, `default` to clear) to `tenant update` - Adds `doublezero-admin migrate flex-algo [--dry-run]` command that backfills link topology assignments and VPNv4 loopback flex-algo node segments across all existing devices and links ## Diff Breakdown | Category | Files | Lines (+/-) | Net | |-------------|-------|-------------|------| | Core logic | 11 | +471 / -57 | +414 | | Scaffolding | 7 | +16 / -5 | +11 | Mostly core logic — the migrate command alone is 180 lines of backfill and dry-run orchestration. <details> <summary>Key files (click to expand)</summary> - `controlplane/doublezero-admin/src/cli/migrate.rs` — new: `migrate flex-algo` command; validates UNICAST-DEFAULT PDA exists, backfills link topologies for all links, backfills flex-algo node segments for all VPNv4 loopback interfaces on all devices; supports `--dry-run` - `smartcontract/cli/src/link/list.rs` — adds `--topology <name>` filter and displays `link_topologies`/`unicast_drained` columns, resolving topology pubkeys to human-readable names - `smartcontract/cli/src/link/get.rs` — displays topology assignments and drain status; fetches topology map to resolve pubkeys to names - `smartcontract/cli/src/link/update.rs` — adds `--link-topology` (accepts comma-separated list of topology names; use `default` to clear) and `--unicast-drained` flags - `smartcontract/cli/src/tenant/list.rs` — displays `include_topologies` column - `smartcontract/cli/src/tenant/get.rs` — displays included topology names - `smartcontract/cli/src/tenant/update.rs` — adds `--include-topologies` (accepts comma-separated list; use `default` to clear) </details> ## Testing Verification - `cargo test -p doublezero_sdk -p doublezero_cli -p doublezero-admin` passes - `cargo clippy -- -D warnings` clean - `make rust-fmt` applied before commit --------- Co-authored-by: Greg Mitchell <greg@malbeclabs.com>

…format change

compat test to use 120s request timeout

…counts

SetGlobalConfig writes to 10 accounts and can exceed the 200k CU default. Add an execute_transaction_with_compute_unit_limit method on the DoubleZeroClient trait and use it from SetGlobalConfigCommand to request the max per-transaction budget. Other commands are unaffected.

…troller support RFC-18 flex-algo · PR 5 of 5 · see `rfcs/rfc-0018-flex-algo.md` Depends on: #3497 (PR 1) — does not require PRs 2–4; can merge after PR 1 Series: #3497 · #3512 · #3513 · #3514 · #3515 ## Summary of Changes - Adds `TopologyInfo` account type to the Go SDK (`state.go`, `deserialize.go`, `client.go`), including `TopologyType`, `TopologyConstraint`, `LinkFlagUnicastDrained`, `LinkTopologies`/`LinkFlags` on `Link`, and `IncludeTopologies` on `Tenant` - Auto-loads `/etc/doublezero-controller/features.yaml` at startup if present (silently skips if absent); when `flex_algo.enabled: true`, populates topology data into the state cache, resolves tenant color communities via `resolveTenantColors`, and emits IS-IS flex-algo node segment and link configuration into the Arista EOS template - Template emits correct flex-algo definitions for both `include-any` constraint (`administrative-group include any N exclude 0`) and `exclude` constraint (`administrative-group exclude N,0`) topologies; gated per-link via `link_tagging.exclude` and per-tenant via `community_stamping` - Updates `e2e/compatibility_test.go` to set the mandatory upgrade boundary for RFC-18 interface/link operations to `before: "0.18.0"` ## Diff Breakdown | Category | Files | Lines (+/-) | Net | |-------------|-------|--------------|-------| | Core logic | 6 | +368 / -6 | +362 | | Scaffolding | 2 | +22 / -0 | +22 | | Tests | 4 | +482 / -2 | +480 | | Fixtures | 2 | +415 / -0 | +415 | | Docs | 1 | +4 / -0 | +4 | Most of the weight is tests and fixtures; core logic is the controller cache/template and Go SDK deserialization. > **Note on diff size:** The full diff vs `main` is large because this branch is stacked on PR 1 (#3497), which has not yet merged. The table above reflects only PR 5's contribution. After PR 1 merges and this branch is rebased, the diff shrinks to the ~15 files shown here. <details> <summary>Key files (click to expand)</summary> - [`controlplane/controller/internal/controller/server.go`](https://github.com/malbeclabs/doublezero/pull/3515/files#diff-f5e715ed1377ac408a4ecde2c60cbcb98327512c7d1496114ef143cb27e7afaf) — populates `Topologies` map in state cache; resolves `LinkTopologies` pubkeys to topology names and `UnicastDrained` from `LinkFlags`; computes `TenantTopologyColors`; `resolveTenantColors` falls back to `unicast-default` when tenant has no explicit topology assignments - [`controlplane/controller/internal/controller/templates/tunnel.tmpl`](https://github.com/malbeclabs/doublezero/pull/3515/files#diff-d67a980326aca35fdabdd5445fbbab4716f1ae412af4540770d5a9ee116b04d9) — IS-IS flex-algo node segment config, link admin-group tagging, BGP `next-hop resolution ribs` and `set extcommunity` stamping, and full rollback (`no` commands) when disabled; handles both `include-any` and `exclude` constraint topologies - [`controlplane/controller/internal/controller/features_config.go`](https://github.com/malbeclabs/doublezero/pull/3515/files#diff-24f09aab331b4d309662e5ee6550fd38fa173676ae868db381d9af23cf645f1d) — `FeaturesConfig` struct with `flex_algo.enabled`, `link_tagging.exclude`, and `community_stamping` controls; auto-loaded from `/etc/doublezero-controller/features.yaml` - [`controlplane/controller/internal/controller/models.go`](https://github.com/malbeclabs/doublezero/pull/3515/files#diff-bdc4e34592a7435dcde7dab78ffd735dcb9847447f0113bd9f3afc8d3d01f864) — adds `LinkTopologies`, `UnicastDrained`, `FlexAlgoNodeSegments`, `TenantTopologyColors` to controller model types; `TopologyModel` and `FlexAlgoEnabled()` for template rendering - [`smartcontract/sdk/go/serviceability/state.go`](https://github.com/malbeclabs/doublezero/pull/3515/files#diff-057f627e5f2629b35f1976fba6eaa1b3c507c511b0f47d2f80a3a1d0c684c4a1) — adds `TopologyType`, `TopologyConstraint`, `TopologyInfo`, `LinkFlagUnicastDrained`; extends `Link` with `LinkTopologies`/`LinkFlags` and `Tenant` with `IncludeTopologies` - [`smartcontract/sdk/go/serviceability/deserialize.go`](https://github.com/malbeclabs/doublezero/pull/3515/files#diff-b71f90b1b3e00de9bd820920e63f5615edb9b7d110c2b39f38895847509a7844) — adds `DeserializeTopologyInfo`; extends `DeserializeLink` with `LinkTopologies`/`LinkFlags` and `DeserializeTenant` with `IncludeTopologies` </details> ## Testing Verification - `TestGetConfig_FlexAlgo` (new): render tests for flex-algo disabled, enabled with both constraint types (`include-any` and `exclude`), and link excluded from tagging — all pass - `Test_resolveTenantColors` (new): 5 table-driven cases covering fallback to `unicast-default`, missing topology, single/multiple known pubkeys, and unknown pubkey — all pass - `TestE2E_IBRL`, `TestE2E_IBRL_WithAllocatedAddr`, `TestE2E_Multicast` — all pass; flex-algo config is suppressed in e2e (no `features.yaml` present) - `exclude` constraint flex-algo syntax validated on physical lab switches (chi-dn-dzd5–dzd8) --------- Co-authored-by: Greg Mitchell <greg@malbeclabs.com>

ben-malbeclabs force-pushed the bc/rfc18-pr2 branch from 834ffe0 to ed49424 Compare April 14, 2026 01:10

ben-malbeclabs force-pushed the bc/rfc18-pr4 branch from af4189e to be77997 Compare April 14, 2026 01:32

ben-malbeclabs force-pushed the bc/rfc18-pr2 branch from ed49424 to 93ecaeb Compare April 15, 2026 01:09

ben-malbeclabs force-pushed the bc/rfc18-pr4 branch from be77997 to dc2ef9c Compare April 15, 2026 01:10

ben-malbeclabs marked this pull request as ready for review April 15, 2026 04:00

ben-malbeclabs requested a review from elitegreg as a code owner April 15, 2026 04:00

ben-malbeclabs requested review from juan-malbeclabs and nikw9944 April 15, 2026 04:01

elitegreg force-pushed the bc/rfc18-pr2 branch from 93ecaeb to 51c4779 Compare April 21, 2026 01:42

Base automatically changed from bc/rfc18-pr2 to main April 21, 2026 20:40

elitegreg changed the title ~~activator,sdk: add topology event handler and update Python/TypeScript SDKs for RFC-18~~ sdk: add topology support to Python/TypeScript SDKs for RFC-18 Apr 22, 2026

ben-malbeclabs added 6 commits April 22, 2026 20:18

e2e,changelog: document mandatory CLI upgrade for RFC-18 InterfaceV3 …

c777ef7

…format change

sdk: add topology support to Python/TypeScript SDKs

48270e2

sdk: fix getProgramData

2ac9fa3

compat test to use 120s request timeout

sdk/ts: remove AbortController timeout from rpc.ts fetch wrapper

44c4572

sdk/ts: guard flexAlgoNodeSegments loop against pre-RFC-18 mainnet ac…

d784ca3

…counts

sdk/ts: fix tryReadString to check content length before reading

580e96b

elitegreg force-pushed the bc/rfc18-pr4 branch from dc2ef9c to 580e96b Compare April 22, 2026 20:20

elitegreg approved these changes Apr 22, 2026

View reviewed changes

elitegreg enabled auto-merge (squash) April 22, 2026 20:22

elitegreg merged commit dfd989a into main Apr 23, 2026
36 checks passed

elitegreg deleted the bc/rfc18-pr4 branch April 23, 2026 05:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sdk: add topology support to Python/TypeScript SDKs for RFC-18#3514

sdk: add topology support to Python/TypeScript SDKs for RFC-18#3514
elitegreg merged 7 commits intomainfrom
bc/rfc18-pr4

ben-malbeclabs commented Apr 10, 2026 •

edited by elitegreg

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ben-malbeclabs commented Apr 10, 2026 • edited by elitegreg Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of Changes

Diff Breakdown

Testing Verification

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ben-malbeclabs commented Apr 10, 2026 •

edited by elitegreg

Loading