Expand fbcode buck test coverage for arm backend (58→162 test targets) (#18817) by digantdesai · Pull Request #18817 · pytorch/executorch

digantdesai · 2026-04-10T16:09:23Z

Summary:

Expand the arm backend test coverage in fbcode by adding all available
test files from ops/, quantizer/, and misc/ directories to targets.bzl.

Reviewed By: 3l1

Differential Revision: D96521268

pytorch-bot · 2026-04-10T16:09:28Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18817

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

[CI[B200] Smoke test encounters CUDA Unknown error for dgxb200-03 and dgxb200-04

✅ You can merge normally! (4 Unrelated Failures)

As of commit 4141453 with merge base 75ba558 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

Cadence Build & Test / cpu-test / test-ops / test-ops (gh) (detected as infra flaky with no log or failing log classifier)
Test Vulkan Backend / test-vulkan / test-backend-linux (vulkan, models) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.
pull / unittest-editable / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

meta-codesync · 2026-04-10T16:09:33Z

@digantdesai has exported this pull request. If you are a Meta employee, you can view the originating Diff in D96521268.

github-actions · 2026-04-10T16:10:15Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Copilot

Pull request overview

Expands the Arm backend’s Buck-based pytest coverage in fbcode by enumerating many more existing test files, and aligns Conv3D lowering with the common tosa_serializer import style used throughout the Arm backend.

Changes:

Greatly expands backends/arm/test/targets.bzl to include many more ops/, quantizer/, and misc/ pytest targets (with some explicitly disabled and annotated).
Updates Conv3D TOSA lowering to import tosa_serializer directly.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
backends/arm/test/targets.bzl	Adds many more pytest targets for Arm backend tests and documents known exclusions.
backends/arm/operators/op_tosa_conv3d.py	Switches Conv3D visitor to import `tosa_serializer` directly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-10T16:17:08Z

    # Operators
    test_files += [
+        "ops/test_abs.py",
        "ops/test_add.py",
        "ops/test_addmm.py",
+        "ops/test_amax.py",


The PR description/title claims this adds “all available” tests from ops/, quantizer/, and misc/, but this target list still omits many existing tests in those directories (e.g., ops/test_acos.py, ops/test_any.py, misc/test_high_rank_permute_view_invariants.py, etc.). Either update the description to reflect that this is a curated subset, or switch to native.glob(["ops/test_.py"], ...) / native.glob(["misc/test_.py"], ...) with an explicit exclusion list so coverage doesn’t silently drift as new tests are added.

Copilot · 2026-04-10T16:17:08Z

+        "quantizer/test_conv_relu_fusing.py",
        "quantizer/test_generic_annotater.py",
+        "quantizer/test_partial_quantization.py",
+        # "quantizer/test_preserve_kwargs.py",  # needs executorch.backends.test.harness.stages (no BUCK target)


This exclusion reason appears incorrect: executorch.backends.test.harness.stages is included in the public Buck target //executorch/backends/test/harness:tester (which is already a dependency of //executorch/backends/arm/test:arm_tester_lib). If quantizer/test_preserve_kwargs.py is being skipped for another reason (e.g., failures), please update the comment; otherwise it looks like it should be enabled to increase coverage.

Suggested change

# "quantizer/test_preserve_kwargs.py", # needs executorch.backends.test.harness.stages (no BUCK target)

"quantizer/test_preserve_kwargs.py",

Copilot · 2026-04-10T16:17:08Z

+        "misc/test_extract_io_params_tosa.py",
+        # "misc/test_int64.py",  # 5 failures: int64 overflow/chain handling issues
+        # "misc/test_lifted_tensor.py",  # needs executorch.backends.test.harness.stages (no BUCK target)
+        "misc/test_mixed_fp_bf16_partition.py",


Same as above for this exclusion: executorch.backends.test.harness.stages has a Buck target (//executorch/backends/test/harness:tester) and is already pulled in transitively via //executorch/backends/arm/test:arm_tester. If this test is disabled due to failures/flake, please update the comment accordingly; otherwise consider enabling it to match the stated goal of expanding coverage.

pytorch#18817) Summary: Expand the arm backend test coverage in fbcode by adding all available test files from ops/, quantizer/, and misc/ directories to targets.bzl. Reviewed By: 3l1 Differential Revision: D96521268

Summary: The pybindings code handled bool and int scalar inputs but was missing support for float (Python float → C++ double). This caused failures when running models like Addmm that take float alpha/beta parameters, throwing 'Unsupported python type <class float>'. Added py::isinstance<py::float_> handling to convert Python floats to EValue(double) in both the portable and XNNPACK input processing paths. Differential Revision: D99845426

pytorch#18817) Summary: Expand the arm backend test coverage in fbcode by adding all available test files from ops/, quantizer/, and misc/ directories to targets.bzl. Reviewed By: 3l1 Differential Revision: D96521268

pytorch#18817) Summary: Pull Request resolved: pytorch#18817 Expand the arm backend test coverage in fbcode by adding all available test files from ops/, quantizer/, and misc/ directories to targets.bzl. Reviewed By: 3l1 Differential Revision: D96521268

Copilot

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

Comments suppressed due to low confidence (2)

extension/pybindings/pybindings.cpp:1145

Same issue here: the error message says inputs should be a flat list of tensors, but this conversion path supports None/bool/int/float scalars too. Updating the message to reflect supported types will make failures easier to debug.

        cpp_inputs.push_back(EValue(py::cast<double>(python_input)));
      } else {
        throw std::runtime_error(
            "Unsupported python type " + type_str +
            ". Ensure that inputs are passed as a flat list of tensors.");

extension/pybindings/pybindings.cpp:815

The thrown error message is now misleading: this input conversion accepts tensors plus None/bool/int/float, but the message says "Ensure that inputs are passed as a flat list of tensors." Consider updating it to list the supported scalar types (and/or mention that non-tensor scalars are supported) so users aren’t misdirected when they pass an unsupported type.

        cpp_inputs.push_back(EValue(py::cast<double>(python_input)));
      } else {
        throw std::runtime_error(
            "Unsupported python type " + type_str +
            ". Ensure that inputs are passed as a flat list of tensors.");

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-16T14:41:01Z

+      } else if (py::isinstance<py::float_>(python_input)) {
+        cpp_inputs.push_back(EValue(py::cast<double>(python_input)));
      } else {


This PR’s description is focused on expanding Arm backend Buck test targets, but this hunk also changes the Python bindings’ accepted input types (python float → EValue(double)). Please update the PR description to mention this behavior change, or split it into a separate PR if it’s unrelated to the test target expansion.

Copilot · 2026-04-16T14:41:01Z

        "misc/test_bn_relu_folding_qat.py",
+        "misc/test_call_operator_submodule.py",
+        "misc/test_compile_spec.py",


misc/test_compile_spec.py is listed twice in test_files (once earlier in the misc section and again here). Because target names are derived from the basename, this will attempt to define the same Buck target (compile_spec) twice and fail. Remove the duplicate or de-duplicate test_files before the loop.

Copilot · 2026-04-16T14:41:01Z

        "deprecation/test_arm_compile_spec_deprecation.py",
+        "misc/test_partition_decomposed_quantized_ops.py",
+        "misc/test_pass_pipeline_config.py",
+        "misc/test_pass_required_order.py",
+        "misc/test_qat_training_loop.py",


misc/test_pass_pipeline_config.py is already included in the earlier "Misc tests" list, and is added again here. Since test_name is based on the basename, this will create a duplicate Buck rule name (pass_pipeline_config) and break the build. Remove one occurrence or ensure test_files is unique before generating targets.

Copilot · 2026-04-16T14:41:02Z

+        "misc/test_tosa_dialect_conv2d.py",
+        "misc/test_tosa_dialect_dw_conv2d.py",
+        "misc/test_tosa_dialect_shape_ops.py",
+        "misc/test_tosa_spec.py",
    ]


misc/test_tosa_spec.py is already included in the "Misc tests" list and is added again here. This results in defining the same Buck target name (tosa_spec) twice. Remove the duplicate entry or de-duplicate test_files before iterating.

Copilot · 2026-04-16T14:41:02Z

    def _get_tosa_op(self):
-        import serializer.tosa_serializer as ts  # type: ignore
+        import tosa_serializer as ts

        return ts.Op.CONV3D


This PR description says it only updates targets.bzl to expand test coverage, but this file also changes runtime behavior by switching the Conv3D lowering import path. Please reflect this in the PR description (or move it into a separate change) so reviewers understand the additional functional impact.

Copilot · 2026-04-16T14:41:02Z

    # Operators
    test_files += [
+        "ops/test_abs.py",
        "ops/test_add.py",
        "ops/test_addmm.py",
+        "ops/test_amax.py",
+        "ops/test_amin.py",


The PR description says it adds all available tests from ops/, quantizer/, and misc/, but there are existing test files in backends/arm/test/ops/ (e.g. ops/test_acos.py, ops/test_arange.py, etc.) that are not included or called out as intentionally excluded. Either add the remaining tests (with comments for known exclusions) or adjust the description to match what’s actually being covered.

Copilot AI review requested due to automatic review settings April 10, 2026 16:09

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 10, 2026

meta-codesync Bot added fb-exported meta-exported labels Apr 10, 2026

Copilot started reviewing on behalf of digantdesai April 10, 2026 16:10 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

meta-codesync Bot changed the title ~~Expand fbcode buck test coverage for arm backend (58→162 test targets)~~ Expand fbcode buck test coverage for arm backend (58→162 test targets) (#18817) Apr 13, 2026

digantdesai force-pushed the export-D96521268 branch from 68a6157 to d6432e8 Compare April 13, 2026 19:38

Copilot AI review requested due to automatic review settings April 16, 2026 14:35

digantdesai force-pushed the export-D96521268 branch from d6432e8 to d59afd6 Compare April 16, 2026 14:35

digantdesai requested review from JacobSzwejbka and larryliu0820 as code owners April 16, 2026 14:35

Copilot started reviewing on behalf of digantdesai April 16, 2026 14:35 View session

digantdesai force-pushed the export-D96521268 branch from d59afd6 to 4141453 Compare April 16, 2026 14:38

Copilot AI reviewed Apr 16, 2026

View reviewed changes

	# "quantizer/test_preserve_kwargs.py", # needs executorch.backends.test.harness.stages (no BUCK target)
	"quantizer/test_preserve_kwargs.py",

Conversation

digantdesai commented Apr 10, 2026 • edited by meta-codesync Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Apr 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18817

❗ 1 Active SEVs

✅ You can merge normally! (4 Unrelated Failures)

Uh oh!

meta-codesync Bot commented Apr 10, 2026

Uh oh!

github-actions Bot commented Apr 10, 2026

This PR needs a release notes: label

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

digantdesai commented Apr 10, 2026 •

edited by meta-codesync Bot

Loading

pytorch-bot Bot commented Apr 10, 2026 •

edited

Loading

This PR needs a `release notes:` label