feat(pypi): support importing uv.lock file by aignas · Pull Request #3785 · bazel-contrib/rules_python

aignas · 2026-05-16T17:07:35Z

This is vibe coded, but I thought that the approach might have been rigorous
enough to submit a PR.

The strategy was:

First add a way for us to create a uv.lock file from the lock rule.
Then add a uv.lock file to JSON converter.
Then add a way to read the uv.lock file together with the requirements file
and verify things are OK.
Reuse most of the code.

Extra things that we could do:

Use https://github.com/jvolkman/toml.bzl. However we need to send a few
upstream patches to remove the bazel_lib dependency (which I think could
become a dev_dependency). This way we won't transitively depend on
rules_go and we won't need to download the interpreter to read the
uv.lock file.
Full test suite for various uv.lock scenarious and ensure parity with
requirements.txt files.
Call the PyPI index to understand if the packages are yanked or not - lock
file does not have that information.
Read the pyproject.toml file to get the index values for each package.

Summary:

feat(pypi): add toml2json tool for converting uv.lock TOML to JSON
test(pypi): add tests for toml2json tool
feat(pypi): add uv_lock utility for uv.lock to JSON conversion
feat(uv): update lock rule with uv.lock JSON conversion support
feat(pypi): add uv.lock parsing support to parse_requirements
test(pypi): add tests for uv.lock parsing in parse_requirements
test(uv): add lock rule integration tests for uv.lock format
test: add uv_pypi end-to-end integration test
docs: add uv.lock documentation and sample files

Closes #3557
Work towards #2787

gemini-code-assist

Code Review

This pull request adds support for uv.lock files in rules_python, introducing a toml2json conversion utility and updating the lock rule and requirement parsing logic. The review identifies critical compatibility issues, specifically the toml2json tool's reliance on Python 3.11's tomllib and missing serialization for date/time objects. Furthermore, the feedback highlights logic errors in how package extras are handled—which could lead to dependency bloat or failed consistency checks—and suggests improvements for platform resolution and path handling in shell scripts.

gemini-code-assist · 2026-05-16T17:12:54Z

+    """Parse requirements using uv.lock as the primary source."""
+    ret = _parse_uv_lock_json(
+        uv_lock_json = uv_lock_json,
+        all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],


If requirements_by_platform is empty (which is common when using uv.lock as the primary source), all_platforms will be an empty list. This results in all packages having empty target_platforms, which will likely cause issues in downstream rules that expect platform information for wheel selection.

Suggested change

all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],

all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()),

gemini-code-assist · 2026-05-16T17:12:55Z

+        for extra in pkg.get("provides-extras", pkg.get("extras", [])):
+            if extra not in entry["extras"]:
+                entry["extras"][extra] = None


Including all provides-extras in the requirement line for every package is likely incorrect. provides-extras lists all extras a package defines, not necessarily what was resolved or requested. Including all of them will force the installation of all optional dependencies for every package in the lock file, leading to significant dependency bloat. It might be better to omit extras from the requirement line if the lock file already provides the specific version and URL, or only include the extras that were part of the resolution.

gemini-code-assist · 2026-05-16T17:12:55Z

+        for dep in pkg.get("dependencies", []):
+            extra = dep.get("extra")
+            if extra and extra not in uv_packages[name]["extras"]:
+                uv_packages[name]["extras"][extra] = None


The logic for collecting extras in the consistency check appears to be flawed. It iterates over the dependencies of pkg and adds the extra name to uv_packages[name]["extras"], where name is the name of the requiring package. Later, when checking a package item, it looks for extras in uv_packages[item.name]["extras"]. This will fail to find extras that were requested by other packages.

gemini-code-assist · 2026-05-16T17:12:55Z

+import datetime
+import json
+import sys
+import tomllib


tomllib was introduced in Python 3.11. Since rules_python supports older versions (e.g., 3.9 as seen in the docs build), this tool will crash on those environments. Consider adding a fallback to the tomli library or requiring a minimum Python version of 3.11 for this specific tool.

gemini-code-assist · 2026-05-16T17:12:55Z

+        python_run = getattr(python, "short_path", python)
+
+        # Make python path absolute since uv changes directory via --directory
+        python_abs = "$PWD/" + python_path


Prepending $PWD/ to python_path assumes that the path is relative to the execution root. If python_path is already an absolute path, this will result in an invalid path. It is safer to check if the path is absolute before prepending.

Suggested change

python_abs = "$PWD/" + python_path

python_abs = python_path if python_path.startswith("/") else "$PWD/" + python_path

gemini-code-assist · 2026-05-16T17:12:55Z

+def json_serializer(obj):
+    if isinstance(obj, datetime.datetime):
+        return obj.isoformat()
+    raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")


The json_serializer only handles datetime.datetime. However, tomllib can also return datetime.date and datetime.time objects for TOML date and time literals. These should also be serialized to ISO format to avoid TypeError.

Suggested change

def json_serializer(obj):

if isinstance(obj, datetime.datetime):

return obj.isoformat()

raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")

def json_serializer(obj):

if isinstance(obj, (datetime.datetime, datetime.date, datetime.time)):

return obj.isoformat()

raise TypeError(f"Object of type {type(obj).__name__} is not JSON serializable")

- tomllib: try/except fallback to tomli for Python <3.11 - json_serializer: add datetime.date and datetime.time support - all_platforms: use sorted(platforms.keys()) fallback - $PWD/ path: check if python_path is absolute - extra_pip_args: pass through to _parse_uv_lock_json - Add uv_lock tests: multiple packages, extra_pip_args, multi-os - Update plan.md with review cycle instructions

aignas · 2026-05-16T18:33:12Z

Addressed all review comments. Please re-review when you get a chance. @gemini-code-assist

gemini-code-assist

Code Review

This pull request introduces support for uv.lock files in rules_python, adding a new lock_format attribute to the lock rule and a toml2json utility for parsing. It also implements a consistency check between uv.lock and requirements files. The reviewer identified a potential runtime error in the uv.lock parser due to unsafe dictionary access for optional hash keys and suggested simplifying redundant logic during hash extraction.

gemini-code-assist · 2026-05-16T18:36:22Z

+
+        seen = {}
+        for wheel in pkg.get("wheels", []):
+            sha256 = wheel["hash"].replace("sha256:", "")


The hash key is optional for wheels in the uv.lock file format. Using direct dictionary access wheel["hash"] will cause a runtime error if the key is missing. You should use .get() for safe access, similar to how it's used for sdist hashes.

Suggested change

sha256 = wheel["hash"].replace("sha256:", "")

sha256 = wheel.get("hash", "").replace("sha256:", "")

gemini-code-assist · 2026-05-16T18:36:22Z

+
+        sdist = pkg.get("sdist")
+        if sdist:
+            sha256 = sdist.get("hash", "").replace("sha256:", "") if sdist.get("hash") else ""


This line can be simplified. The if sdist.get("hash") else "" is redundant. sdist.get("hash", "").replace("sha256:", "") correctly handles cases where the hash is missing or empty.

Suggested change

sha256 = sdist.get("hash", "").replace("sha256:", "") if sdist.get("hash") else ""

sha256 = sdist.get("hash", "").replace("sha256:", "")

- Add python_3_14 toolchain in MODULE.bazel - Always use Python 3.14 interpreter for uv.lock JSON conversion - Remove tomli fallback from toml2json.py (Python 3.14 has tomllib) - Simplify convert_uv_lock_to_json to use fixed interpreter

- Add bazel-in-bazel integration test under tests/integration/uv_lock/ - Test converts uv.lock TOML to JSON using toml2json tool - Remove dead uv_pypi test data (placeholder only) - Restore tomli fallback in toml2json.py for backward compatibility

aignas · 2026-05-16T18:43:40Z

Addressed all review comments and added Python 3.14 toolchain + integration test. Please re-review. @gemini-code-assist

gemini-code-assist

Code Review

This pull request introduces support for uv.lock files as a primary metadata source in rules_python. Key additions include a toml2json tool for parsing, a new uv_lock format for the lock rule, and consistency checks within parse_requirements. Reviewer feedback suggests adopting more idiomatic Starlark dictionary access, implementing safer hash prefix stripping, and improving the portability of shell commands by replacing hardcoded environment variables with Bazel-provided paths.

gemini-code-assist · 2026-05-16T18:48:04Z

+    uv_packages = {}
+    for pkg in packages:
+        name = pkg["name"]
+        if (name == "rules_python" and is_rules_python_root) or (pkg.get("source") or {}).get("virtual"):


Using pkg.get("source", {}) is more idiomatic in Starlark and Python than (pkg.get("source") or {}) when you want to provide a default dictionary for a missing or None key.

Suggested change

if (name == "rules_python" and is_rules_python_root) or (pkg.get("source") or {}).get("virtual"):

if (name == "rules_python" and is_rules_python_root) or pkg.get("source", {}).get("virtual"):

gemini-code-assist · 2026-05-16T18:48:05Z

+
+        seen = {}
+        for wheel in pkg.get("wheels", []):
+            sha256 = wheel["hash"].replace("sha256:", "")


It is safer to check if the hash string actually starts with the sha256: prefix before attempting to replace it, or use a more robust way to strip the algorithm prefix if uv ever supports other hash types in the lock file.

Suggested change

sha256 = wheel["hash"].replace("sha256:", "")

sha256 = wheel["hash"][7:] if wheel["hash"].startswith("sha256:") else wheel["hash"]

gemini-code-assist · 2026-05-16T18:48:05Z

+
+        sdist = pkg.get("sdist")
+        if sdist:
+            sha256 = sdist.get("hash", "").replace("sha256:", "") if sdist.get("hash") else ""


Similar to the wheel hash processing, it's better to explicitly check for the sha256: prefix.

Suggested change

sha256 = sdist.get("hash", "").replace("sha256:", "") if sdist.get("hash") else ""

sha256 = sdist["hash"][7:] if sdist.get("hash", "").startswith("sha256:") else ""

gemini-code-assist · 2026-05-16T18:48:05Z

+                filename = filename,
+                yanked = None,
+            ))
+        elif (pkg.get("source") or {}).get("git"):


Prefer pkg.get("source", {}) for consistency and readability.

Suggested change

elif (pkg.get("source") or {}).get("git"):

elif pkg.get("source", {}).get("git"):

gemini-code-assist · 2026-05-16T18:48:05Z

+        python_run = getattr(python, "short_path", python)
+
+        # Make python path absolute since uv changes directory via --directory
+        python_abs = python_path if python_path.startswith("/") else "$PWD/" + python_path


Hardcoding $PWD/ for absolute path construction in a shell command might be problematic on some platforms or environments. Consider using ctx.bin_dir.path or similar Bazel-provided paths if possible, or ensure this is only executed in a bash-compatible environment where PWD is guaranteed to be set correctly.

jvolkman · 2026-05-17T14:39:43Z

The latest release of toml.bzl makes bazel_lib a dev dependency. It now uses skylib for the bzl_library rules.

aignas added 9 commits May 17, 2026 01:56

feat(pypi): add toml2json tool for converting uv.lock TOML to JSON

a58cf50

test(pypi): add tests for toml2json tool

363c141

feat(pypi): add uv_lock utility for uv.lock to JSON conversion

6f4e925

feat(uv): update lock rule with uv.lock JSON conversion support

29a7993

feat(pypi): add uv.lock parsing support to parse_requirements

a063d53

test(pypi): add tests for uv.lock parsing in parse_requirements

9fea66c

test(uv): add lock rule integration tests for uv.lock format

aa19ceb

test: add uv_pypi end-to-end integration test

ee069e4

docs: add uv.lock documentation and sample files

dd53e29

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

aignas added 2 commits May 17, 2026 02:18

remove dead code and add plan.md for iterating

d5709ea

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

aignas added 3 commits May 17, 2026 03:38

docs: update plan.md with completed stages

6db3345

gemini-code-assist Bot reviewed May 16, 2026

View reviewed changes

	all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else [],
	all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()),

	python_abs = "$PWD/" + python_path
	python_abs = python_path if python_path.startswith("/") else "$PWD/" + python_path

	sha256 = wheel["hash"].replace("sha256:", "")
	sha256 = wheel.get("hash", "").replace("sha256:", "")

	sha256 = sdist.get("hash", "").replace("sha256:", "") if sdist.get("hash") else ""
	sha256 = sdist.get("hash", "").replace("sha256:", "")

	if (name == "rules_python" and is_rules_python_root) or (pkg.get("source") or {}).get("virtual"):
	if (name == "rules_python" and is_rules_python_root) or pkg.get("source", {}).get("virtual"):

	sha256 = wheel["hash"].replace("sha256:", "")
	sha256 = wheel["hash"][7:] if wheel["hash"].startswith("sha256:") else wheel["hash"]

	sha256 = sdist.get("hash", "").replace("sha256:", "") if sdist.get("hash") else ""
	sha256 = sdist["hash"][7:] if sdist.get("hash", "").startswith("sha256:") else ""

	elif (pkg.get("source") or {}).get("git"):
	elif pkg.get("source", {}).get("git"):

Uh oh!

Conversation

aignas commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

aignas commented May 16, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

aignas commented May 16, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 16, 2026

Choose a reason for hiding this comment

Uh oh!

jvolkman commented May 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aignas commented May 16, 2026 •

edited

Loading