Skip to content

[ET Device Support] Annotate device attributes of CUDA backend IO tensors cuda device#18080

Open
Gasoonjia wants to merge 13 commits intogh/gasoonjia/137/basefrom
gh/gasoonjia/137/head
Open

[ET Device Support] Annotate device attributes of CUDA backend IO tensors cuda device#18080
Gasoonjia wants to merge 13 commits intogh/gasoonjia/137/basefrom
gh/gasoonjia/137/head

Conversation

@Gasoonjia
Copy link
Copy Markdown
Contributor

@Gasoonjia Gasoonjia commented Mar 10, 2026

Stack from ghstack (oldest at bottom):

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: D96010436

…sors cuda device

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Mar 10, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18080

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 2 New Failures, 3 Unrelated Failures

As of commit f5f20d9 with merge base 81bc830 (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Gasoonjia added a commit that referenced this pull request Mar 10, 2026
…sors cuda device

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

ghstack-source-id: 350230761
Pull Request resolved: #18080
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 10, 2026
@github-actions
Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
Gasoonjia added a commit that referenced this pull request Mar 13, 2026
…sors cuda device

Pull Request resolved: #18080

Update cuda backend partitioner to annotate its IO tensors as cuda device
ghstack-source-id: 351558872
@exported-using-ghexport

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
Gasoonjia added a commit that referenced this pull request Mar 17, 2026
…sors cuda device

Pull Request resolved: #18080

Update cuda backend partitioner to annotate its IO tensors as cuda device
ghstack-source-id: 353202795
@exported-using-ghexport

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
Gasoonjia added a commit that referenced this pull request Mar 19, 2026
…sors cuda device

Pull Request resolved: #18080

Update cuda backend partitioner to annotate its IO tensors as cuda device
ghstack-source-id: 354478933
@exported-using-ghexport

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
Gasoonjia added a commit that referenced this pull request Apr 14, 2026
…sors cuda device

Pull Request resolved: #18080

Update cuda backend partitioner to annotate its IO tensors as cuda device, and add checks in cuda backend to guarantee it works
ghstack-source-id: 366850769
@exported-using-ghexport

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)
Gasoonjia added a commit that referenced this pull request Apr 14, 2026
…ensorSpecs (#18078)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #18080
* #18328
* #18079
* __->__ #18078

Add end-to-end device type annotation support from export to runtime.
Currently we only support one device per graph

The overall pipeline is:
a. Partitioner use `compile_spec` to determine which device the
partitoned blob is runing on
b. after lowered partitioned graph to backend, the new-introed
propagate_device_pass will annotate the input and output tensors of
delegate blob as target device.

Differential Revision:
[D95842511](https://our.internmc.facebook.com/intern/diff/D95842511/)
Gasoonjia added a commit that referenced this pull request Apr 14, 2026
…ized Tensor (#18079)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #18080
* #18328
* __->__ #18079
* #18078

Propagate device information from `TensorSpec.device` (set by
`PropagateDevicePass`) to
the serialized `schema.Tensor` in the emitted PTE file, to make runtime
further aware of it.

Differential Revision:
[D95899706](https://our.internmc.facebook.com/intern/diff/D95899706/)
…kend IO tensors cuda device"

Update cuda backend partitioner to annotate its IO tensors as cuda device

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)

[ghstack-poisoned]
Gasoonjia added a commit that referenced this pull request Apr 16, 2026
…sors cuda device

Pull Request resolved: #18080

Update cuda backend partitioner to annotate its IO tensors as cuda device, and add checks in cuda backend to guarantee it works
ghstack-source-id: 368551184
@exported-using-ghexport

Differential Revision: [D96010436](https://our.internmc.facebook.com/intern/diff/D96010436/)
Gasoonjia added a commit that referenced this pull request Apr 17, 2026
…r_parser (#18328)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at
bottom):
* #18080
* __->__ #18328

Parse device info (device_type, device_index) from the serialized
ExtraTensorInfo in .pte files into TensorImpl at runtime.
When a tensor's extra_tensor_info contains device annotations (e.g.,
CUDA), the tensor parser now reads and propagates them to the TensorImpl
constructor. Tensors without extra_tensor_info default to CPU/0 for
backward compatibility with older PTE files.、

Differential Revision:
[D97199497](https://our.internmc.facebook.com/intern/diff/D97199497/)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/cuda CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. fb-exported meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants