feat(hadoop): Add 3.5.0 and 3.4.3

[sbernauer]

Description

Part of #1494
Needs stackabletech/hdfs-utils#62

• Add Hadoop 3.4.3 and 3.5.0
• As all of the aws and azure lib collection is a clusterfuck, I remodeled it: The hdfs/hdfs image takes care of placing all cloud libs in /stackable/hadoop-cloud-libraries/, so that dependant images can easily pull them in. Side node: We now support 3 different Hadoop versions with 3 different naming conventions, argh.
• I tried to be a good citizen and bump Hadoop 3.4.2 -> 3.4.3 in downstream images. However, this caused compilation failures. As this PR is big enough I left it for the downstream version bumps.

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

• Changes are OpenShift compatible
• All added packages (via microdnf or otherwise) have a comment on why they are added
• Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
• All packages should have (if available) signatures/hashes verified
• Add an entry to the CHANGELOG.md file
• Integration tests ran successfully

TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

boil build <IMAGE> --image-version <RELEASE_VERSION> --strip-architecture --load
kind load docker-image <MANIFEST_URI> --name=<name-of-your-test-cluster>

See the output of boil to retrieve the image manifest URI for <MANIFEST_URI>.

[sbernauer]

marker

[NickLarsenNZ]

Could we use patchable instead and avoid uploading sources to Nexus?

[NickLarsenNZ]

As in, it becomes an image layer with a boil-config.toml and patches

[NickLarsenNZ]

It looks like you did, since there is "shared/boost". We just need to mirror the repo (if not done).

[sbernauer]

I copied the Dockerfile from @razvan and he documented it here:

docker-images/shared/boost/Dockerfile

Lines 10 to 14 in 61741f6

	# NOTE: We use a published source bundle instead of the patchable workflow
	# because boost uses git submodules for its build system
	# and patchable doesn't support these.
	# The source bundle contains everything needed to build b2 (the build system)
	# and boost except for dependencies which are automatically discovered.

[NickLarsenNZ]

I think Druid should be its own PR

[NickLarsenNZ]

I see all the other products were update in this PR too.
I guess it is fine, but maybe find the tracking issues to link this PR in them (did you do the whole task for all, or only hdfs related stuff?)

[NickLarsenNZ]

The reminder only have TODOs. How was druid different (why could it have the bump while the others couldn't)?

[sbernauer]

I kept the ones where the compilation worked. All the ones where I was running into compilation errors which I couldn't solve in a sane timeframe I left as a TODO.
Such big changes in a patch level release suck... Any maven dependency resolution sucks, argh

[NickLarsenNZ]

Looks fine. I didn't check in detail, but that should come out via testing.

I left a few comments for clarifications.