25.8.15 Backport of #92250 - Reduce INSERT/merges memory usage with wide parts for very wide tables by enabling adaptive write buffers

[zvonand]

Changelog category (leave one):

• Performance Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Reduce INSERT/merges memory usage with wide parts for very wide tables by enabling adaptive write buffers. Add support of adaptive write buffers for encrypted disks (ClickHouse#92250 by @azat)

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All with Aarch64
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [ff06714]

[zvonand]

S3 fails: Altinity/clickhouse-regression#93

Aggregate fails: uniqTheta was updated in upstream, not related, test will be updated separately by QA team

Not sure about /iceberg/iceberg engine/rest catalog/datatypes/data types check where. @CarlosFelipeOR maybe you know whether it is OK or not?

[Selfeer]

Failed Jobs Analysis

Job	Status	Cause	PR-Related
Regression release iceberg_1	failure	timestamptz equality comparison bug in Iceberg integration	❌ No
Regression release aggregate_functions_1	failure	uniqTheta test needs changes unrelated to the PR	❌ No
Regression aarch64 aggregate_functions_1	failure	uniqTheta test needs changes unrelated to the PR	❌ No
Regression release s3_aws_s3_2	failure	Zero-copy replication issue - fix pending upstream	❌ No
Regression release s3_azure_2	failure	Zero-copy replication issue - fix pending upstream	❌ No
Regression release s3_minio_2	failure	Zero-copy replication issue - fix pending upstream	❌ No
Regression aarch64 s3_aws_s3_2	failure	Zero-copy replication issue - fix pending upstream	❌ No
Regression aarch64 s3_azure_2	failure	Zero-copy replication issue - fix pending upstream	❌ No
Regression aarch64 s3_minio_2	failure	Zero-copy replication issue - fix pending upstream	❌ No

────────────────────────────────────────

Detailed Failure Analysis

1. Iceberg timestamptz equality comparison (NEW)

• Reports: x86_64
• Failed test: data types check where
• Reason: WHERE timestamptz = toDateTime64(...) returns 0 rows when 1 row should match. We can see that this test sometimes passes and sometimes doesn't even before this PR
• Results from the database: iceberg_1

2. uniqTheta aggregate function test (FLAKY)

• Reports: x86_64 | aarch64
• Reason: Aggregate function test completely unrelated to adaptive write buffer changes
• Verdict: Test needs to be updated because of other changes made upstream related to aggregate functions.

3. Zero-copy replication tests

Failed tests across AWS S3, Azure, and MinIO backends:
• zero copy replication/add remove replica parallel
• zero copy replication/add remove one replica
• zero copy replication/bad detached part

The issue: #1338
The fix: ClickHouse#95597

Reports:
• s3_aws_s3_2 x86_64
• s3_minio_2 x86_64
• s3_azure_2 x86_64
• s3_aws_s3_2 aarch64
• s3_minio_2 aarch64
• s3_azure_2 aarch64

[Selfeer]

On further investigation it does seem like the iceberg issue mostly appears in versions 25.8.15 and 25.8.12:

Version	OK	Fail	Pass Rate
25.8.14.20001.altinityantalya	90	0	100%
25.8.14.17-alpine	2	0	100%
25.8.12.20747	2	10	17%
25.8.15.35-alpine	0	8	0%
25.8.15.35	0	1	0%

[alsugiliazova]

Regression iceberg_1 fail is unreleated to this pr. It happened because of float division introducing a tiny rounding error so the constructed DateTime64 value doesn’t exactly match the stored microsecond timestamp. I fixed the seed and checked that now test is stable.