Improvement/CLDSRV-825 drop ceph support by benzekrimaha · Pull Request #6097 · scality/cloudserver

benzekrimaha · 2026-03-09T16:19:11Z

bert-e · 2026-03-09T16:19:23Z

Hello benzekrimaha,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-03-09T16:37:59Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

claude · 2026-03-09T16:47:33Z

LGTM

Clean removal of all Ceph-related code: Docker files, location config, CI_CEPH env var checks, isCEPH/itSkipCeph/describeSkipIfCeph conditionals, and capability flags. No leftover Ceph references found in JS, JSON, or YAML files. The test-utils exports are consistent after the change.

Review by Claude Code

francoisferrand · 2026-04-01T19:40:07Z

        if (config.capabilities) {
            if (config.capabilities.locationTypes) {
-                this.supportedLocationTypes = new Set(config.capabilities.locationTypes);
+                this.supportedLocationTypes = new Set(


why?
this should not be needed, unless there is an issue in tests.... so the tests should be updated instead: we should not adapt production code to tests, but tests to match how it happens in production

francoisferrand · 2026-04-01T19:46:10Z

@@ -531,7 +523,7 @@ describe('Multipart Upload API with AWS Backend', function mpuTestSuite() {
                })).then(() => {
                    assert.fail('Expected an error listing parts of aborted MPU');
                }).catch(err => {
-                    const wantedError = isCEPH ? 'NoSuchKey' : 'NoSuchUpload';
+                    const wantedError = 'NoSuchUpload';


variable probably not needed here

francoisferrand · 2026-04-01T19:47:25Z

                awsIngestLocation: false,
            });
+            assert.strictEqual(
+                Object.prototype.hasOwnProperty.call(caps, 'locationTypeCephRadosGW'),


useless: redundant with assert.deepStrictEqual above, can never fail...

francoisferrand · 2026-04-01T19:47:35Z

                customCapability: 'test-value',
            });
+            assert.strictEqual(
+                Object.prototype.hasOwnProperty.call(caps, 'locationTypeCephRadosGW'),


useless: redundant with assert.deepStrictEqual above, can never fail...

francoisferrand

a few small things, but more importantly it seems to me this cleanup is not complete: this is modifying only the tests, but we keep the code handling specific corner cases for cepth (which may be either/both in cloudserver and arsenal).

this code MUST be cleaned as well, otherwise we just continue maintaining the same code, just with less coverage

bert-e · 2026-04-08T13:05:51Z

Branches have diverged

This pull request's source branch improvement/CLDSRV-825-drop-ceph-support has diverged from
development/9.3 by more than 50 commits.

To avoid any integration risks, please re-synchronize them using one of the
following solutions:

Merge origin/development/9.3 into improvement/CLDSRV-825-drop-ceph-support
Rebase improvement/CLDSRV-825-drop-ceph-support onto origin/development/9.3

Note: If you choose to rebase, you may have to ask me to rebuild
integration branches using the reset command.

benzekrimaha · 2026-04-08T16:44:35Z

@francoisferrand Thanks for the careful review — I agree with your point that removing tests without removing related code would leave us with untested legacy behavior.
I re-audited this PR specifically with that objective:
I listed CEPH-conditioned test removals (isCEPH / itSkipCeph / describeSkipIfCeph / CEPH-only expected error variants).
I mapped those removed tests to the corresponding runtime paths.
I removed the CEPH-specific runtime handling that was still present.
What was actually CEPH-specific in CloudServer:
=> Capability-key cleanup in reportHandler getCapabilities() (locationTypeCephRadosGW / cephIngestLocation) was the remaining CEPH-specific logic.
This has now been removed, and the matching unit test coverage was removed accordingly.
What remains and why:
The remaining behaviors touched by removed CEPH-conditioned tests are generic S3/MPU API semantics, not CEPH branches. Examples:
Internal MPU metadata NoSuchKey mapped to public NoSuchUpload.
Multipart delete behavior controlled by legacyAwsBehavior.
Standard versioning/object-lock semantics (NoSuchKey / NoSuchVersion / MethodNotAllowed).
These paths are still covered by active non-CEPH tests.
So this is not a tests-only cleanup anymore:
CEPH-specific runtime handling in CloudServer has been removed.
Remaining logic is generic compatibility behavior with active coverage.
If you want, I can also do the same explicit mapping pass on Arsenal in a follow-up to ensure both repos are aligned.

francoisferrand · 2026-04-09T07:36:57Z

-    delete caps.locationTypeCephRadosGW;
-    delete caps.cephIngestLocation;


this was an earlier comment, we need to keep these!
cloudserver will not support ceph anymore, so we don't want to advertise that we still support it

francoisferrand · 2026-04-09T07:40:57Z

Capability-key cleanup in reportHandler getCapabilities() (locationTypeCephRadosGW / cephIngestLocation) was the remaining CEPH-specific logic.

this is not "ceph-specific" logic, this is logic which was introduced earlier on purpose to correctly report that we DO NOT support ceph anymore. So it must be kept.

I can also do the same explicit mapping pass on Arsenal in a follow-up to ensure both repos are aligned.

yes, as discussed on Monday we need to check in Arsenal as well : where the actual specifics of each (data) backend are implemented

bert-e · 2026-04-30T14:30:04Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

claude · 2026-04-30T14:32:28Z

LGTM

Clean removal of Ceph support across Docker infrastructure, config, capabilities, and tests. A few observations (non-blocking):

- Tests that were previously always-skipped via describeSkipIfNotMultipleOrCeph (which resolved to describe.skip unconditionally) are now gated with describeSkipIfNotMultiple, meaning they will actually run when S3BACKEND=multiple. This is the right behavior but is a meaningful change in test coverage.
- rangeTest.js includes AWS SDK v3 compatibility fixes (e.g. transformToByteArray, relaxed ContentType assertion, uploadParts return values) beyond just Ceph removal — looks correct but is a separate concern bundled in.
- The delete caps.locationTypeCephRadosGW / delete caps.cephIngestLocation approach in reportHandler.js correctly handles backward compatibility with existing config files that may still reference these fields.
- Arsenal pinned to tag 8.4.1 — good.

Review by Claude Code

claude · 2026-04-30T14:34:26Z


-const describeSkipIfNotMultipleOrCeph = config.backends.data !== 'multiple'
-    ? describe.skip : describe.skip;



The objectPutCopyPartAzure.js file has a blank line left behind after removing the old local describeSkipIfNotMultipleOrCeph definition (line 23-24). Cosmetic, not a blocker.

— Claude Code

claude · 2026-04-30T14:34:39Z

LGTM

Clean removal of Ceph support. The key changes are well-scoped:
- Ceph Docker infra, location config, and CI_CEPH env var references removed
- describeSkipIfNotMultipleOrCeph / itSkipCeph / isCEPH helpers eliminated from test utils and all consumers
- Tests previously always-skipped (describeSkipIfNotMultipleOrCeph was describe.skip in all branches) are now properly gated by describeSkipIfNotMultiple
- reportHandler correctly deletes Ceph capability fields as a safety net for existing configs
- Arsenal bumped from 8.3.9 to 8.4.1 (pinned to tag, good)
- rangeTest.js fixes for AWS SDK v3 stream handling (transformToByteArray) and uploadParts returning ETags are correct
- routeBackbeat abort MPU test appropriately changed from bare it to itIfLocationAws since it requires the AWS location

One minor cosmetic nit noted inline (extra blank line in objectPutCopyPartAzure.js).

Review by Claude Code

Drop the Ceph CI service, helper scripts, and dedicated test location config now that CloudServer no longer supports Ceph RadosGW. Remove the CI_CEPH runtime overrides from Config so location constraints follow the normal validation path, and keep report capabilities from advertising Ceph support even if an external config still provides those legacy flags. Issue: CLDSRV-825

Delete the Ceph-only Mocha helpers and replace Ceph-conditioned skips with the regular multiple-backend guards so the remaining suites run for supported backends. Clean up Ceph-specific expected errors, comments, and fixture references from functional and multiple-backend tests while preserving the generic S3 behavior coverage. Issue: CLDSRV-825

Adapt the range functional test to the SDK v3 response body API by materializing the returned byte array before writing it to disk. Return uploaded part responses so CompleteMultipartUpload receives the expected ETags, and tolerate NoSuchUpload during cleanup after the upload has already completed. Issue: CLDSRV-825

Update the Arsenal dependency to 8.4.1 so CloudServer consumes the backend capability changes that complete the Ceph support removal. Refresh yarn.lock for the new Arsenal revision and its hdclient dependency requirement. Issue: CLDSRV-825

claude · 2026-04-30T14:41:18Z

LGTM

Clean removal of Ceph (RADOS Gateway) support throughout the codebase. All changes are consistent:

- Ceph Docker infrastructure, CI config, and location config deleted
- CI_CEPH env var checks removed from Config.js
- Ceph capability fields stripped in reportHandler.js with backward-compat delete for old configs that still set them
- All test skip utilities (isCEPH, itSkipCeph, describeSkipIfCeph, describeSkipIfNotMultipleOrCeph) removed and replaced with describeSkipIfNotMultiple or plain describe/it — note that describeSkipIfNotMultipleOrCeph was always describe.skip, so these tests now correctly run when the backend is multiple
- rangeTest.js includes SDK v3 fixes (Body stream handling via transformToByteArray, uploadParts return values for CompleteMultipartUpload) needed now that the test is no longer ceph-skipped
- Arsenal bumped from 8.3.9 to 8.4.1 (pinned to tag)

Review by Claude Code

benzekrimaha changed the title ~~Improvement/cldsrv 825 drop ceph support~~ Improvement/CLDSRV-825 drop ceph support Mar 9, 2026