Skip to content

[feat] PIP-468: Add scalable topics and segments admin APIs with CLI#25565

Merged
merlimat merged 1 commit intoapache:masterfrom
merlimat:st-admin
Apr 24, 2026
Merged

[feat] PIP-468: Add scalable topics and segments admin APIs with CLI#25565
merlimat merged 1 commit intoapache:masterfrom
merlimat:st-admin

Conversation

@merlimat
Copy link
Copy Markdown
Contributor

Stacked on top of #25564 (which is stacked on #25559). Opened as a
draft; will be rebased on master once the upstream PRs land. Reviewers:
the new code in this PR is the single commit titled "PIP-468: Add
scalable topics and segments admin APIs with CLI" — everything else
comes from #25559 and #25564.

Summary

Third PR in the PIP-468 series. Wires up the admin/management surface
for scalable topics:

REST APIs (broker)

  • ScalableTopics REST resource — create / list / get-metadata / get-stats /
    delete / create-subscription / delete-subscription / split / merge.
    Includes redirectToControllerLeaderIfNeeded for write operations so
    non-leader brokers automatically 307 to the leader.
  • Segments REST resource — cross-broker segment-topic
    create / terminate / delete with ownership-based routing.

Client-admin API

  • ScalableTopics interface on PulsarAdmin with sync and async
    counterparts for every operation.
  • ScalableTopicsImpl — REST client wiring.
  • Data types under org.apache.pulsar.common.policies.data:
    ScalableTopicMetadata (plus nested SegmentInfo and HashRange),
    ScalableTopicStats (plus nested SegmentStats and SubscriptionStats
    records), ScalableSubscriptionType enum.

CLI

  • CmdScalableTopics pulsar-admin subcommand with list, create,
    get-metadata, stats, delete, split-segment, merge-segments.

Broker integration

Tests

New test classes (24 tests)

  • ScalableTopicStatsTest (10 tests) — construction (no-args /
    @Builder / all-args), @Builder.Default fresh-per-instance
    behavior, equals/hashCode, and the nested SegmentStats /
    SubscriptionStats records.
  • ScalableTopicMetadataTest (11 tests) — same construction patterns,
    plus SegmentInfo.isActive / isSealed (including the unknown-state
    branch), full getters on a sealed segment, and HashRange.toString()
    format (lowercase 4-digit hex).
  • ScalableSubscriptionTypeTest (3 tests) — locks down the enum values
    so a rename breaks the build; valueOf roundtrip and rejection of
    unknown names.

Expanded in this commit

  • ScalableTopicControllerTest — 24 tests covering all controller
    methods including createSubscription / deleteSubscription /
    getStats / stats-after-split-and-subscriptions / etc.
  • ScalableTopicServiceTest — 16 tests covering service lifecycle,
    create/delete topic, get-or-create-controller (caching + failure +
    eviction), release, delegation to split / merge / subscriptions /
    stats / register / disconnect, and topic cleanup.

Coverage on this branch

Test class Tests
CommandsScalableTopicTest (from #25564) 8
ConsumerSessionTest (from #25564) 15
DagWatchSessionTest (from #25564) 12
ScalableTopicControllerTest 24
ScalableTopicServiceTest 16
SegmentLayoutTest (from #25559) 15
SubscriptionCoordinatorTest (from #25559) 11
ScalableTopicStatsTest 10
ScalableTopicMetadataTest 11
ScalableSubscriptionTypeTest 3
Total 125

Test plan

  • ./gradlew :pulsar-client-admin-api:test --tests "org.apache.pulsar.common.policies.data.Scalable*Test" — 24/24 pass.
  • ./gradlew :pulsar-broker:test --tests "org.apache.pulsar.broker.service.scalable.*" — 93/93 pass.
  • ./gradlew :pulsar-common:test --tests "org.apache.pulsar.common.protocol.CommandsScalableTopicTest" — 8/8 pass.
  • ./gradlew :pulsar-client-admin-api:checkstyleMain :pulsar-client-admin-api:checkstyleTest :pulsar-broker:checkstyleMain :pulsar-broker:checkstyleTest :pulsar-common:checkstyleMain :pulsar-common:checkstyleTest :pulsar-client-admin-original:checkstyleMain :pulsar-client-tools:checkstyleMain — clean.

@merlimat merlimat added type/feature The PR added a new feature or issue requested a new feature area/broker area/admin labels Apr 22, 2026
@merlimat merlimat changed the title PIP-468: Add scalable topics and segments admin APIs with CLI [feat] PIP-468: Add scalable topics and segments admin APIs with CLI Apr 22, 2026
Copy link
Copy Markdown
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I guess authnz will come later.

Add ScalableTopics REST resource for CRUD, split, and merge operations on
scalable topics. Add Segments REST resource for cross-broker segment topic
create/terminate/delete with ownership-based routing. Implement client-admin
ScalableTopics interface and ScalableTopicsImpl with async support. Add
CmdScalableTopics CLI commands (list, create, get-metadata, delete,
split-segment, merge-segments). Wire into PulsarAdmin and PulsarAdminTool.
@merlimat merlimat marked this pull request as ready for review April 24, 2026 21:50
@merlimat merlimat merged commit 3c5c866 into apache:master Apr 24, 2026
80 of 82 checks passed
@merlimat merlimat deleted the st-admin branch April 24, 2026 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/admin area/broker type/feature The PR added a new feature or issue requested a new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants