Skip to content

Reproduce license cache inheritance failure #4709#5237

Draft
hakandilek wants to merge 1 commit into
aboutcode-org:developfrom
hakandilek:test/4709-license-cache-inheritance
Draft

Reproduce license cache inheritance failure #4709#5237
hakandilek wants to merge 1 commit into
aboutcode-org:developfrom
hakandilek:test/4709-license-cache-inheritance

Conversation

@hakandilek

Copy link
Copy Markdown

Summary

Add a lightweight reproducer for the multiprocessing license-cache
inheritance failure reported in #4709, and a possible fix following a short
discussion.

Root cause: ScanCode warms the license cache in the parent process before
creating its worker pool. Linux Python 3.13 uses fork, so workers inherit
the warmed cache through copy-on-write. Python 3.14 uses forkserver by
default, so workers start without that state and load private copies of the
license index.

Reproducer

The test substitutes a small sentinel for the real license cache and creates
a worker through the production scancode.pool.get_pool() function.

This exercises the same inheritance boundary without:

  • Loading the approximately 1 GB license index
  • Measuring unstable RSS values
  • Requiring a large input scan
  • Consuming significant memory or runtime

Expected behavior on the current code:

Platform Python Start method Result
Linux 3.13 fork Pass
Linux 3.14 forkserver Fail
macOS/Windows Any spawn Skipped

The test is collected by the
core_tests / misc_and_scancode
job defined in azure-pipelines.yml, running on Ubuntu 24.04 with Python
3.14.

Scope

This Draft PR initially contains only the reproducer and no production fix.

I would like maintainer feedback on the preferred fix direction before
adding one.

Tests do not currently pass on Linux Python 3.14. This failure is
intentional while the PR is a Draft and demonstrates the regression.

Reproduces #4709.

Verification

venv/bin/pytest -vvs tests/scancode/test_pool.py

ScanCode warms the license cache before creating its process pool. Linux Python 3.14 uses forkserver by default, so workers do not inherit that state and load private copies.

Add a lightweight sentinel test for this boundary without loading the real license index. The test intentionally fails under the current Python 3.14 Linux pool behavior.

Signed-off-by: Hakan Dilek <hakandilek@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant