[SPARK-55189][BUILD] Upgrade lz4-java to 1.10.3#53971
Closed
LuciferYang wants to merge 1 commit intoapache:masterfrom
Closed
[SPARK-55189][BUILD] Upgrade lz4-java to 1.10.3#53971LuciferYang wants to merge 1 commit intoapache:masterfrom
LuciferYang wants to merge 1 commit intoapache:masterfrom
Conversation
JIRA Issue Information=== Improvement SPARK-55189 === This comment was automatically generated by GitHub Actions |
LuciferYang
changed the title
dongjoon-hyun
approved these changes
Jan 26, 2026
Member
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM (Pending CIs).
Could you add a comment about the running CI, @LuciferYang ? It seems that it's missing currently.
Contributor
Author
sarutak
approved these changes
Jan 26, 2026
zhengruifeng
approved these changes
Jan 26, 2026
Member
|
Thank you, @LuciferYang and all. Merged to master for Apache Spark 4.2.0. |
pan3793
added a commit
that referenced
this pull request
Feb 27, 2026
…gression ### What changes were proposed in this pull request? Previously, lz4-java was upgraded to 1.10.x to address CVEs, - #53327 - #53347 - #53971 while this casues significant performance drop, see the benchmark report at - #53453 this PR follows the [suggestion](#53290 (comment)) to migrate to safeDecompressor. ### Why are the changes needed? Mitigate performance regression. ### Does this PR introduce _any_ user-facing change? No, except for performance. ### How was this patch tested? GHA for functionality, [benchmark](#53453 (comment)) for performance. > TL;DR - my test results show lz4-java 1.10.1 is about 10~15% slower on lz4 compression than 1.8.0, and is about ~5% slower on lz4 decompression even with migrating to suggested safeDecompressor ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53454 from pan3793/SPARK-54571. Lead-authored-by: Cheng Pan <chengpan@apache.org> Co-authored-by: pan3793 <pan3793@users.noreply.github.com> Signed-off-by: Cheng Pan <chengpan@apache.org>
SteNicholas
added a commit
to apache/celeborn
that referenced
this pull request
Mar 3, 2026
… CVE‐2025‐12183 and CVE-2025-66566 ### What changes were proposed in this pull request? - Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566. - `Lz4Decompressor` follows the [suggestion](apache/spark#53290 (comment)) to move from `fastDecompressor` to `safeDecompressor` to mitigate the performance. Backport: - apache/spark#53327 - apache/spark#53347 - apache/spark#53971 - apache/spark#53454 - apache/spark#54585 ### Why are the changes needed? - [CVE‐2025‐12183](https://sites.google.com/sonatype.com/vulnerabilities/cve-2025-12183): Various lz4-java compression and decompression implementations do not guard against out-of-bounds memory access. Untrusted input may lead to denial of service and information disclosure. Vulnerable Maven coordinates: org.lz4:lz4-java up to and including 1.8.0. - [CVE-2025-66566](GHSA-cmp6-m4wj-q63q): Insufficient clearing of the output buffer in Java-based decompressor implementations in lz4-java 1.10.0 and earlier allows remote attackers to read previous buffer contents via crafted compressed input. In applications where the output buffer is reused without being cleared, this may lead to disclosure of sensitive data. JNI-based implementations are not affected. Therefore, lz4-java version should upgrade to 1.10.4. ### Does this PR resolve a correctness bug? No. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? CI. Closes #3555 from SteNicholas/CELEBORN-2218. Lead-authored-by: SteNicholas <programgeek@163.com> Co-authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: SteNicholas <programgeek@163.com>
SteNicholas
added a commit
to apache/celeborn
that referenced
this pull request
Mar 3, 2026
… CVE‐2025‐12183 and CVE-2025-66566 - Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566. - `Lz4Decompressor` follows the [suggestion](apache/spark#53290 (comment)) to move from `fastDecompressor` to `safeDecompressor` to mitigate the performance. Backport: - apache/spark#53327 - apache/spark#53347 - apache/spark#53971 - apache/spark#53454 - apache/spark#54585 - [CVE‐2025‐12183](https://sites.google.com/sonatype.com/vulnerabilities/cve-2025-12183): Various lz4-java compression and decompression implementations do not guard against out-of-bounds memory access. Untrusted input may lead to denial of service and information disclosure. Vulnerable Maven coordinates: org.lz4:lz4-java up to and including 1.8.0. - [CVE-2025-66566](GHSA-cmp6-m4wj-q63q): Insufficient clearing of the output buffer in Java-based decompressor implementations in lz4-java 1.10.0 and earlier allows remote attackers to read previous buffer contents via crafted compressed input. In applications where the output buffer is reused without being cleared, this may lead to disclosure of sensitive data. JNI-based implementations are not affected. Therefore, lz4-java version should upgrade to 1.10.4. No. No. CI. Closes #3555 from SteNicholas/CELEBORN-2218. Lead-authored-by: SteNicholas <programgeek@163.com> Co-authored-by: Cheng Pan <chengpan@apache.org> Signed-off-by: SteNicholas <programgeek@163.com> (cherry picked from commit dca3749) Signed-off-by: SteNicholas <programgeek@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This pr aims to upgrade lz4-java from 1.10.1 to 1.10.3
Why are the changes needed?
The new version has fixed an NPE issue related to
LZ4FrameInputStream:Full release notes as follows:
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass Github Actions
Was this patch authored or co-authored using generative AI tooling?
No