fix: support multi-mount OSS secret projections in jindo#5778
fix: support multi-mount OSS secret projections in jindo#5778CAICAIIs wants to merge 4 commits intofluid-cloudnative:masterfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @CAICAIIs. Thanks for your PR. I'm waiting for a fluid-cloudnative member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
Signed-off-by: CAICAIIs <3360776475@qq.com>
Signed-off-by: CAICAIIs <3360776475@qq.com>
13f43d6 to
7a06f0f
Compare
There was a problem hiding this comment.
Code Review
This pull request introduces support for multiple OSS bucket credential projections in both the jindocache and jindofsx engines. It updates Helm templates to handle projected secret volumes and modifies the transformation logic to generate these projections from dataset mount options. The review feedback highlights a security inconsistency where jindofsx still persists inline credentials in plain text, suggests optimizing the grouping of secret projections to reduce pod spec verbosity, and identifies several bugs related to error messages and logging consistency.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #5778 +/- ##
==========================================
+ Coverage 60.47% 60.53% +0.06%
==========================================
Files 465 465
Lines 31148 31209 +61
==========================================
+ Hits 18837 18893 +56
- Misses 10764 10767 +3
- Partials 1547 1549 +2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: CAICAIIs <3360776475@qq.com>
8bd230b to
7c6841e
Compare
There was a problem hiding this comment.
Pull request overview
Fixes Jindo multi-mount OSS secret handling so that different OSS mounts (buckets) can use different AK/SK pairs by switching from a single “last-one-wins” secret to bucket-scoped projected secrets and per-bucket provider configuration.
Changes:
- Introduces bucket-scoped secret projections and per-bucket secret URIs in both
jindocacheandjindofsxtransforms. - Updates Fuse/Token properties to configure OSS credentials provider per bucket (endpoint/format/provider), avoiding persisting inline credentials.
- Updates Helm templates to mount credential volumes when either a single secret or
secretProjectionsare present, and adds projected-volume support.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| pkg/ddc/jindofsx/types.go | Adds fields to carry secret projections + bucket-to-secret URI mapping. |
| pkg/ddc/jindofsx/transform.go | Builds bucket-scoped projections/URIs and injects per-bucket provider properties (Fuse/Token). |
| pkg/ddc/jindofsx/transform_master_test.go | Adds unit tests for multi-mount OSS projections and non-persistence of inline creds. |
| pkg/ddc/jindocache/types.go | Adds fields to carry secret projections + bucket-to-secret URI mapping. |
| pkg/ddc/jindocache/transform.go | Builds bucket-scoped projections/URIs and injects per-bucket provider properties (Fuse/Token). |
| pkg/ddc/jindocache/transform_test.go | Adds unit tests for multi-mount OSS projections and non-persistence of inline creds. |
| charts/jindofsx/templates/worker/statefulset.yaml | Mounts secret volume when secret or secretProjections is set. |
| charts/jindofsx/templates/master/statefulset.yaml | Mounts secret volume when secret or secretProjections is set. |
| charts/jindofsx/templates/fuse/daemonset.yaml | Mounts secret volume when secret or secretProjections is set. |
| charts/jindofsx/templates/_helpers.tpl | Adds projected secret volume rendering supporting secretProjections. |
| charts/jindocache/templates/worker/statefulset.yaml | Mounts secret volume when secret or secretProjections is set. |
| charts/jindocache/templates/master/statefulset.yaml | Mounts secret volume when secret or secretProjections is set. |
| charts/jindocache/templates/fuse/daemonset.yaml | Mounts secret volume when secret or secretProjections is set. |
| charts/jindocache/templates/_helpers.tpl | Adds projected secret volume rendering supporting secretProjections. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: CAICAIIs <3360776475@qq.com>
|



Ⅰ. Describe what this PR does
This PR fixes multi-mount OSS secret handling for Jindo when different mounts use different AK/SK pairs.
Before this change, if a
Datasetcontained multiple OSS mounts with differentencryptOptions, only the last OSS secret could effectively take effect. This PR changes the secret handling to bucket-scoped secret projections, so each OSS bucket gets its own projected secret path and provider configuration.The fix covers both
jindocacheandjindofsx, and updates the related Helm templates for master / worker / fuse.Ⅱ. Does this pull request fix one issue?
fixes #4255
Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
Added unit tests:
TestJindoCacheEngine_transformMasterWithMultipleOSSEncryptOptionsTestJindoFSxEngine_transformMasterWithMultipleOSSEncryptOptionsTestJindoCacheEngine_transformMasterDoesNotPersistInlineOSSCredentialsThese tests cover:
Ⅳ. Describe how to verify it
Datasetwith two OSS mounts, for example:oss://caicaiis-a/->/partAoss://caicaiis-b/->/partBJindoRuntime.I also verified this with real OSS buckets:
caicaiis-acaicaiis-bAfter the fix:
/data/partAcan be read successfully/data/partBcan be read successfullyⅤ. Special notes for reviews