-
-
Notifications
You must be signed in to change notification settings - Fork 705
feat: updates to the py_extension rule for building C extensionsPy extension #3875
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
rickeylev
merged 50 commits into
bazel-contrib:py-extension
from
rsartor-cmd:py-extension
Jul 16, 2026
Merged
Changes from all commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
8214422
Don't leave a trailing slash for the root package.
rsartor-cmd 1e89b99
Remove duplicate assignment.
rsartor-cmd d921d20
Remove typical C-style integer suffixes before parsing.
rsartor-cmd f318173
Fix check for filename.
rsartor-cmd 80a9aae
Remove unnecessary guard.
rsartor-cmd 9bfc331
Re-format docstring.
rsartor-cmd e85e06b
Get the platform from the constraints.
rsartor-cmd 590b667
Adjust formatting syntax.
rsartor-cmd 15c95f8
Change the default value.
rsartor-cmd daa2455
Add more test cases for py_limited_api.
rsartor-cmd 030c879
Use RunfilesBuilder instead of manual.
rsartor-cmd 73eb7c0
Add abi_tag to PyCcToolchainInfo.
rsartor-cmd 5cb7c90
Use the PyCcToolchainInfo to derive the filename instead of the runti…
rsartor-cmd 797bf98
Remove the fallback. Determine platform tag solely from constraints.
rsartor-cmd 09a2e0a
Remove the version compatibility check, as it has performance implica…
rsartor-cmd 5bbe09f
Fix header filename.
rsartor-cmd 475ed3d
Update python/private/cc/py_extension_rule.bzl
rsartor-cmd 903c96a
ruff
rsartor-cmd 9570a98
Derive the values for the _constraints attr programmatically, instead…
rsartor-cmd 7179c8a
Take glibc-vs-musl into account, so we get the right platform and name.
rsartor-cmd b353be6
Clean up as per the linter.
rsartor-cmd ee6395d
Remove print()s per buildifier.
rsartor-cmd 8d80783
Remove unused variable (buildifier).
rsartor-cmd 451ea48
Sort keys [buildifier]
rsartor-cmd a40ae2b
Remove print()s per buildifier.
rsartor-cmd 90ad765
Remove unused load().
rsartor-cmd 708dac4
Call out unused parameters [buildifier]
rsartor-cmd 830723f
Add some basic tests of different dependency situations, so we can co…
rsartor-cmd 37f5eaa
Replace the custom compiling and linking logic with internal cc_libra…
rsartor-cmd 4596ae2
Add missing files.
rsartor-cmd 933f69f
Add a data argument to the macro.
rsartor-cmd 6f05ae0
Add missing file
rsartor-cmd 8f2fa54
Add a couple more examples for PyInit_* defined in dependency targets.
rsartor-cmd 908ccd5
Add missing files
rsartor-cmd eb4b8a8
Expand the analysis tests with more details assertions.
rsartor-cmd 12bc59e
Re-format files
rsartor-cmd b158975
Address buildifier and buildifier-lint warnings.
rsartor-cmd 6e1763e
Use text instead of code file for data.
rsartor-cmd d324300
Ignore unusual edge case.
rsartor-cmd 0f4edda
Restore missing file.
rsartor-cmd bae75f7
Remove the CcSharedLibraryInfo provider for now.
rsartor-cmd b7f8112
Calculate the platform tag from the constraints directly, instead of …
rsartor-cmd 1b31b7e
Remove unused load()
rsartor-cmd 32c37e5
Fix tests.
rsartor-cmd 421eb5c
Include free-threading 't' flag in the abi_tag, if the interpreter is…
rsartor-cmd 320c25a
Calculate the platform tag and include it as part of the toolchain info.
rsartor-cmd 4183634
Pass the right set of additional kwargs to the internal generated tar…
rsartor-cmd aab3cee
Adjust import logic to account for package names.
rsartor-cmd 652a50a
Add some C-specific linking attributes to pass to the internal genera…
rsartor-cmd f5cbb95
Adjust dep handling so we can include headers correctly.
rsartor-cmd File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,33 +1,126 @@ | ||
| """Wrapper macro for the py_extension rule.""" | ||
| """Macro for creating Python extensions.""" | ||
|
|
||
| load("@rules_cc//cc:cc_library.bzl", "cc_library") | ||
| load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") | ||
| load("//python/private:util.bzl", "add_tag") | ||
| load( | ||
| ":py_extension_rule.bzl", | ||
| _py_extension = "py_extension", | ||
| ##_py_extension_csl_rule = "py_extension_csl", | ||
| ) | ||
| load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") | ||
| load(":py_extension_rule.bzl", "py_extension_wrapper") | ||
|
|
||
| def py_extension(**kwargs): | ||
| """A macro that calls the py_extension rule and adds a tag. | ||
| def py_extension( | ||
| name, | ||
| srcs = None, | ||
| hdrs = None, | ||
| copts = None, | ||
| defines = None, | ||
| includes = None, | ||
| linkopts = None, | ||
| linkshared = None, | ||
| linkstatic = None, | ||
| deps = None, | ||
| dynamic_deps = None, | ||
| exports_filter = None, | ||
| user_link_flags = None, | ||
| visibility = None, | ||
| data = None, | ||
| **kwargs): | ||
| """Creates a Python extension module. | ||
|
|
||
| By default, extensions are created within their workspace package directory | ||
| (e.g., `pkg/ext.so`) and imported using standard Python package paths | ||
| (e.g., `from pkg import ext`). | ||
|
|
||
| To customize import path behavior: | ||
| - `imports`: Pass `imports = ["..."]` to append custom search directories to | ||
| `sys.path` (matching `py_library`). | ||
| - `module_name`: Pass `module_name = "custom_name"` to override the base module | ||
| filename. | ||
|
|
||
| Args: | ||
| **kwargs: Additional arguments to pass to the rule. | ||
| name: Target name. | ||
| srcs: Optional C/C++ source files to compile directly for this extension. | ||
| hdrs: Optional header files for the srcs. | ||
| copts: Optional compiler flags for srcs. | ||
| defines: Optional preprocessor defines for srcs. | ||
| includes: Optional header include search paths passed to internal cc_library. | ||
| linkopts: Optional link options passed to internal cc_library and cc_shared_library. | ||
| linkshared: Deprecated and ignored. Extensions are always linked dynamically. | ||
| linkstatic: Optional linkstatic flag passed to internal cc_library. | ||
| deps: cc_library targets to statically link into the extension. | ||
| dynamic_deps: cc_shared_library targets to dynamically link. | ||
| exports_filter: Filter for exported symbols passed to cc_shared_library. | ||
| user_link_flags: Additional link flags passed to cc_shared_library. | ||
| visibility: Target visibility. | ||
| data: Optional list of files or targets needed by this extension at runtime. | ||
| **kwargs: Additional arguments passed to the underlying wrapper rule. | ||
| """ | ||
| add_tag(kwargs, "@rules_python//python/cc:py_extension") | ||
| _ = linkshared # buildifier: disable=unused-variable | ||
|
|
||
| csl_deps = [] | ||
|
|
||
| use_csl = kwargs.pop("use_csl", False) | ||
| if use_csl: | ||
| _py_extension_csl(**kwargs) | ||
| else: | ||
| _py_extension(**kwargs) | ||
| # 1. If srcs or hdrs are specified, create an implicit cc_library for them | ||
| if srcs or hdrs: | ||
| impl_lib_name = "_" + name + "_impl" | ||
| impl_lib_kwargs = copy_propagating_kwargs(kwargs) | ||
| if includes: | ||
| impl_lib_kwargs["includes"] = includes | ||
| if linkopts: | ||
| impl_lib_kwargs["linkopts"] = linkopts | ||
| if linkstatic != None: | ||
| impl_lib_kwargs["linkstatic"] = linkstatic | ||
| cc_library( | ||
| name = impl_lib_name, | ||
| srcs = srcs, | ||
| hdrs = hdrs, | ||
| copts = (copts or []) + ["-fPIC"], | ||
| defines = defines, | ||
| deps = (deps or []) + ["@rules_python//python/cc:current_py_cc_headers"], | ||
| visibility = ["//visibility:private"], | ||
| **impl_lib_kwargs | ||
| ) | ||
| csl_deps.append(":" + impl_lib_name) | ||
| elif deps: | ||
| csl_deps.extend(deps) | ||
|
|
||
| def _py_extension_csl(*, name, module_name = None, **kwargs): | ||
| if not module_name: | ||
| module_name = name | ||
| # 2. If no static deps or sources were specified, use empty target for CSL requirement | ||
| if not csl_deps: | ||
| csl_deps.append("//python/private/cc:empty") | ||
|
|
||
| # 4. Create the underlying cc_shared_library | ||
| csl_name = "_" + name + "_csl" | ||
| csl_kwargs = copy_propagating_kwargs(kwargs) | ||
| if exports_filter: | ||
| csl_kwargs["exports_filter"] = exports_filter | ||
| effective_user_link_flags = user_link_flags or linkopts | ||
| if effective_user_link_flags: | ||
| csl_kwargs["user_link_flags"] = effective_user_link_flags | ||
|
|
||
| cc_shared_library( | ||
| name = csl_name, | ||
| deps = csl_deps, | ||
| dynamic_deps = dynamic_deps, | ||
| visibility = ["//visibility:private"], | ||
| **csl_kwargs | ||
| ) | ||
|
|
||
| # 5. Select default libc constraint if not provided | ||
| if "libc" not in kwargs: | ||
| kwargs["libc"] = select({ | ||
| "@rules_python//python/config_settings:_is_py_linux_libc_glibc": "glibc", | ||
| "@rules_python//python/config_settings:_is_py_linux_libc_musl": "musl", | ||
| "//conditions:default": "glibc", | ||
| }) | ||
|
|
||
| if data != None: | ||
| kwargs["data"] = data | ||
|
|
||
| # 6. Filter out C++ specific compilation/linking attributes before invoking wrapper rule | ||
| for cc_attr in ("includes", "linkopts", "linkshared", "linkstatic", "features"): | ||
| kwargs.pop(cc_attr, None) | ||
|
|
||
| # 7. Wrap with py_extension_wrapper for PEP 3149 naming & PyInfo | ||
| py_extension_wrapper( | ||
| name = name, | ||
| shared_lib_name = module_name + ".so", | ||
| src = ":" + csl_name, | ||
| visibility = visibility, | ||
| **kwargs | ||
| ) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.