feat: remove product config & clean up#790
Open
maltesander wants to merge 19 commits into
Open
Conversation
Patches stackable-operator to the operator-rs smooth-operator branch (provides the v2::config_overrides framework) as the prerequisite for removing the product-config dependency, mirroring trino-operator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copies product-config's writer functions (to_hadoop_xml, to_java_properties_string) into config::writer, backed directly by the xml and java-properties crates so output stays byte-identical with the kuttl ConfigMap snapshots. Removes all product_config::writer usage; the product-config dependency itself is dropped in a later phase. Adds unit tests pinning the exact XML/properties on-wire format. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduces the controller/build/properties module structure (mirroring trino-operator) and a local ConfigFileName enum that replaces the bare filename string constants as the single source of truth for the rolegroup ConfigMap keys. Scaffolding for moving per-file rendering out of the monolithic rolegroup_config_map in the next phase. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves rendering of hadoop-policy.xml, ssl-server.xml, ssl-client.xml and security.properties out of the monolithic rolegroup_config_map match into dedicated controller/build/properties modules, each with unit tests pinning the exact output. Logic is copied verbatim (overrides applied last), so the rendered ConfigMap is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the two large inline config renderings out of rolegroup_config_map
into controller/build/properties/{core_site,hdfs_site}, each with unit
tests driven by a shared MINIMAL_HDFS_YAML fixture. Logic is copied
verbatim (overrides applied last), so the rendered ConfigMap is
unchanged. The match arm in rolegroup_config_map is now a thin dispatch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves the recommended networkaddress.cache.ttl=30 and networkaddress.cache.negative.ttl=0 values out of the product-config properties.yaml and into security_properties::build directly. While product-config is still active these arrive identically via the override map applied last, so the rendered file is unchanged; this makes the builder self-sufficient ahead of removing product-config validation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the product-config transform/validate steps with native config
merging, mirroring trino-operator:
- validate_cluster no longer takes a ProductConfigManager; it merges the
typed config fragments (HdfsNodeRole::merged_config) and the per-file
configOverrides + envOverrides (role <- role group) into the extended
ValidatedRoleGroupConfig { merged_config, config_overrides, env_overrides }.
- HdfsConfigOverrides now uses typed KeyValueConfigOverrides fields (v2,
deriving Merge) instead of Options; the per-file builders pull their
overrides via resolved_overrides and apply them last.
- Operator-injected values previously provided by properties.yaml /
Configuration impls are now set directly in the builders/validate:
security.properties DNS cache TTLs, dfs.replication, and the rack-aware
net.topology / TOPOLOGY_LABELS (namenode).
- Deletes build_role_properties, the Configuration impls, PropertyNameKind
usage, and the ProductConfigManager plumbing (Ctx, main); the CLI arg is
kept but ignored.
- Filenames now come from the ConfigFileName enum everywhere (discovery,
kerberos, jvm), dropping the *_XML / *_PROPERTIES_FILE constants.
The product-config crate dependency is now unused and removed separately.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The product-config crate is no longer used by the operator. Removes it from the workspace and operator-binary manifests and empties the config-spec properties.yaml (the schema now lives in the typed CRD and the per-file builders), mirroring trino-operator. product-config remains only as a transitive dependency of stackable-operator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Moves rolegroup_config_map out of hdfs_controller into controller/build/config_map::build_rolegroup_config_map with its own Error enum, mirroring trino-operator. The function now takes the ValidatedCluster and looks up the role group's merged config and overrides internally instead of receiving them pre-destructured. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirrors trino-operator: ValidatedCluster now carries a ValidatedClusterConfig (name, namespace, dfs_replication, https/kerberos/ authentication/authorization flags, rack awareness, OPA authorization) resolved once during validation. The standalone hdfs_opa_config field is folded into it. The build steps no longer take the raw HdfsCluster: - hdfs_site/core_site builders and build_rolegroup_config_map consume &ValidatedClusterConfig / &ValidatedCluster. - The lower-level HdfsSiteConfigBuilder/CoreSiteConfigBuilder and their kerberos security_config impls take primitives (a KerberosConfig struct and bools) to keep the config layer free of controller types. Behavior-preserving: the flags are computed by the same HdfsCluster predicates as before, just resolved up-front. The discovery path keeps using the raw HdfsCluster as it does not go through validation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a top-level name: ClusterName field to ValidatedCluster (mirroring trino-operator), constructed and validated in the validate step. The per-file builders and build_rolegroup_config_map now take the whole &ValidatedCluster, reading cluster.name and cluster.cluster_config. product_version is intentionally not added: unlike Trino (integer versions parsed to u16), HDFS uses semver product versions, so the image keeps carrying the version string. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the pod_refs lookups for namenodes and journalnodes out of reconcile_hdfs and into validate_cluster, storing the results on ValidatedCluster. Build steps that already receive the validated cluster (config_map and hdfs_site) now read the refs from it instead of taking them as separate parameters. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
16 tasks
Replace the vendored java-properties/Hadoop-XML writer (rust/operator-binary/src/config/writer.rs) with stackable_operator::v2::config_file_writer, which hosts this exact file (moved there verbatim via operator-rs #1217 on the smooth-operator branch; hdfs's copy was the canonical source). Drop the now-unused java-properties and xml dependencies. No behaviour change; rendered output is byte-identical by construction (same code, new home). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Definition of Done Checklist
Author
Reviewer
Acceptance
type/deprecationlabel & add to the deprecation scheduletype/experimentallabel & add to the experimental features tracker