mgmt eng, avoid double TypeSpec generation by resolving sdk_folder from tspconfig.yaml

[XiaofeiCao]

What this PR does

Fixes #47775

When generating SDK from TypeSpec via eng/automation/generate.py, the code previously ran tsp-client init twice for mgmt-plane libraries (remove_before_regen=True):

1. First generation: only to discover sdk_folder via git status on tsp-location.yaml
2. Second generation: the actual generation with correct version/api-version options

For large TypeSpec projects where TCGC is slow, this doubled generation time (e.g., 35min → 1:15h).

Solution

Parse tspconfig.yaml before generation to determine the SDK output folder, eliminating the need for the first generation pass entirely. The code now always generates only once.

Changes to eng/automation/generate_utils.py

1. _resolve_tspconfig_variables() (new): General-purpose template variable resolver that follows the same resolution order as the spec validation rules:

   • Emitter options → Parameters → Global config
   • Iterative resolution (up to 10 iterations) for nested variables (e.g., service-dir containing {output-dir})
   • Handles runtime variables ({output-dir}, {project-root}) that are provided by tsp-client, not defined in tspconfig.yaml

2. resolve_sdk_folder_from_tspconfig() (new): Reads tspconfig.yaml (remote URL or local path), resolves emitter-output-dir from @azure-tools/typespec-java options using the variable resolver, and returns the SDK folder path relative to sdk_root. Returns None with a clear error if resolution fails.

3. _read_tspconfig() (new): Reads tspconfig.yaml content from either a GitHub blob URL (converted to raw URL) or a local filesystem path.

4. TSPCONFIG_URL_PATTERN (new): Module-level compiled regex for matching GitHub tspconfig.yaml blob URLs, extracted from the duplicate patterns that existed in the codebase.

5. generate_typespec_project() (modified): Simplified to a single linear flow — resolve sdk_folder from tspconfig → optionally prep (remove old code, set version) → generate once. Fails fast with a RuntimeError if tspconfig resolution fails.

Additional fix

The resolve_sdk_folder_from_tspconfig() approach also fixes the edge case where git status does not detect tsp-location.yaml changes when commit-id is unchanged (e.g., regen due to emitter bug fix with no spec changes, as noted in this comment).

How to verify

The change fails fast if tspconfig.yaml cannot be parsed — resolve_sdk_folder_from_tspconfig() returns None and a RuntimeError is raised with a clear error message. This is by design: all valid TypeSpec projects should have a parseable tspconfig.yaml with Java emitter options including emitter-output-dir and service-dir.

[Copilot]

Pull request overview

This PR optimizes TypeSpec SDK generation by eliminating double generation for management-plane libraries. Previously, generate.py ran tsp-client init twice when remove_before_regen=True: once to discover the SDK folder via git status, and again to perform the actual generation. For large TypeSpec projects, this doubled generation time (e.g., 35min → 1:15h).

Changes:

• Added resolve_sdk_folder_from_tspconfig() to parse tspconfig.yaml and determine the SDK output folder without running generation
• Added _read_tspconfig() to fetch tspconfig.yaml from either remote GitHub URLs or local paths
• Modified generate_typespec_project() to use the optimized path when resolution succeeds, falling back to the original double-generation approach when it fails

[weidongxu-microsoft]

please check copilot review message

wondered how risky if we just depend on the new check (without the fallback) -- it can make the code cleaner, but can have problem on corner cases (what case could happen?)

[XiaofeiCao]

wondered how risky if we just depend on the new check (without the fallback) -- it can make the code cleaner, but can have problem on corner cases (what case could happen?)

The corner cases I have in mind is when tspconfig.yaml format got changed (e.g. something like package-dir replaced with emitter-output-dir, etc).
This may not be a big issue, since tsp-client might also need to be bumped in this case.

azure-sdk-for-java/eng/common/tsp-client/package.json

Line 3 in f032f0c

"@azure-tools/typespec-client-generator-cli": "0.31.0"

[weidongxu-microsoft]

wondered how risky if we just depend on the new check (without the fallback) -- it can make the code cleaner, but can have problem on corner cases (what case could happen?)

The corner cases I have in mind is when tspconfig.yaml format got changed (e.g. something like package-dir replaced with emitter-output-dir, etc). This may not be a big issue, since tsp-client might also need to be bumped in this case.

azure-sdk-for-java/eng/common/tsp-client/package.json

Line 3 in f032f0c

"@azure-tools/typespec-client-generator-cli": "0.31.0"

we may also see typespec-validation, and handling case that would pass it should be enough
https://github.com/Azure/azure-rest-api-specs/blob/main/eng/tools/typespec-validation/src/rules/sdk-tspconfig-validation.ts#L212

[XiaofeiCao]

we may also see typespec-validation, and handling case that would pass it should be enough https://github.com/Azure/azure-rest-api-specs/blob/main/eng/tools/typespec-validation/src/rules/sdk-tspconfig-validation.ts#L212

Thanks, done. Removed fallback logic as well.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 9 comments.