Skip cibuildwheel for dev-requirement wheels with C extensions

[l0lawrence]

Problem

The internal storage SDK pipeline fails on every PR that touches a package whose dev_requirements.txt includes azure-storage-extensions (e.g. azure-storage-queue, azure-storage-blob, azure-storage-file-datalake).

The failure occurs in dispatch_checks.py → replace_dev_reqs → build_whl_for_req → create_package. Because azure-storage-extensions defines a crc64 C extension module, create_package routes the wheel build through cibuildwheel. On Windows, cibuildwheel downloads its own CPython from api.nuget.org via NuGet, but the internal 1ES pool agents block outbound to api.nuget.org (DNS returns RFC-5737 sinkhole 192.0.2.9 and the firewall denies the connection):

Unable to load the service index for source https://api.nuget.org/v3/index.json.
  An error occurred while sending the request.
  Unable to connect to the remote server
  An attempt was made to access a socket in a way forbidden by its access permissions 192.0.2.9:443

Fix

cibuildwheel is designed for producing release artifacts that span multiple Python versions and ABIs. For dev-requirement builds, the wheel only needs to install into the current test venv with the current Python — so the cibuildwheel-driven download of an isolated CPython is unnecessary.

This PR adds a use_cibuildwheel parameter to create_package (defaulting to True so release builds keep using cibuildwheel) and has build_whl_for_req (the dev-requirements path) pass use_cibuildwheel=False. When the flag is off and the package has ext_modules, the build falls through to setup.py bdist_wheel with the invoking Python, which works fine in-venv and needs no nuget.org access.

Impact

• Release pipeline (build_packages): unchanged — still produces the abi3 wheel for azure-storage-extensions via cibuildwheel on the public-CI agent that can reach nuget.org.
• Dev-requirements installs (build_whl_for_req): now build the local ext-module package with the invoking Python — no nuget download, works on the internal 1ES pool.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

This PR adjusts the Azure SDK Python tooling used in CI to avoid using cibuildwheel when building wheels for dev-requirements that include C extensions (notably azure-storage-extensions), addressing failures on internal Windows agents that cannot reach api.nuget.org.

Changes:

• Added a use_cibuildwheel flag to ci_tools.build.create_package (defaulting to True to preserve release behavior).
• Updated dev-requirements wheel building (build_whl_for_req) to call create_package(..., use_cibuildwheel=False) so C-extension wheels are built with the invoking Python instead of cibuildwheel.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
eng/tools/azure-sdk-tools/ci_tools/scenario/generation.py	Disables cibuildwheel when building dev-requirements wheels so builds work on restricted internal agents.
eng/tools/azure-sdk-tools/ci_tools/build.py	Introduces use_cibuildwheel and gates the cibuildwheel path for extension-module wheels behind this flag.

[l0lawrence]

/azp run python - storage - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).