refactor: Migrate Content Understanding from preview to GA and consolidate AI Services account

[Harsh-Microsoft]

Summary

Migrate Azure AI Content Understanding from preview 2024-12-01-preview to GA 2025-11-01, and consolidate the standalone Content Understanding Cognitive Services account into the existing unified Azure AI Services account (which now hosts both Azure OpenAI and Content Understanding).

Why two changes in one PR

In preview, CU was only available in 3 regions, which forced us to provision it as a separate Microsoft.CognitiveServices/accounts (aicu-*) alongside the OpenAI account (aif-*). With GA, CU's region footprint expanded enough that a single unified account can host both services in 11 common regions, so the migration and the consolidation make sense to ship together.

Infra changes

• Drop avmAiServices_cu module, contentUnderstandingPrivateEndpoint, and the contentUnderstandingLocation parameter from main.bicep and main_custom.bicep; mirror in main.json.
• Restrict azureAiServiceLocation @allowed to the 11-region intersection where both CU GA and gpt-5.1 GlobalStandard are available.
• Add two Cognitive Services User role assignments (API and Workflow managed identities) on the unified account so CU calls don't 403.
• Re-route APP_CONTENT_UNDERSTANDING_ENDPOINT to the unified account (CU lives at the same cognitiveservices.azure.com host, only the path differs).
• Drop AZURE_ENV_CU_LOCATION mapping from main.parameters.json and main.waf.parameters.json.
• Remove contentUnderstandingLocation override from .github/workflows/deploy.yml.

Application code changes

• Bump api-version to 2025-11-01 and switch to the GA REST surface in src/ContentProcessor/src/libs/azure_helper/content_understanding.py: :analyzeBinary for stream payloads, knowledgeSources[] for training data, and /files/{id} for figure retrieval.
• Update Pydantic models for GA: add Warning, relax Page optionals (angle/spans/words/lines), and surface the new top-level DocumentContent.paragraphs field.
• Add unit tests for the new Warning model and relaxed Page optionals; bump existing apiVersion fixtures.

Docs

• CustomizingAzdParameters.md: drop AZURE_ENV_CU_LOCATION row, rewrite AZURE_ENV_AI_SERVICE_LOCATION row, and append a usageName note for the Standard deployment type.
• LocalDevelopmentSetup.md: replace stale aicu-{suffix} reference with aif-{suffix}.
• TroubleShootingSteps.md: update the CU 403 row for the consolidated account name and DNS zones.

Validation

• ruff check passes on touched files.
• pytest tests/unit/azure_helper/test_content_understanding_model.py — 15/15 passed.
• az bicep build clean on both main.bicep and main_custom.bicep.
• End-to-end test on the GA-deployed environment confirms functional parity with the preview baseline (same fields populated, same fields missed).

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
libs/azure_helper/model
content_understanding.py	92	2	97%	71, 94
TOTAL	1217	161	86%

Tests	Skipped	Failures	Errors	Time
244	0 💤	0 ❌	0 🔥	4.788s ⏱️

[Copilot]

Pull request overview

This PR migrates the Content Processor’s Azure AI Content Understanding integration from the preview API to the GA API version and updates the infrastructure to consolidate Content Understanding into the existing unified Azure AI Services account (hosting both Azure OpenAI and Content Understanding).

Changes:

• Update the Content Understanding REST client + Pydantic response models to the GA API surface (2025-11-01) and new endpoints/actions.
• Refactor infra parameters/templates to remove the standalone CU account + location override and route CU endpoint to the unified AI Services account, including required RBAC assignments.
• Update tests and docs to reflect GA model shape changes and the consolidated account/private networking guidance.

Reviewed changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/ContentProcessor/src/libs/azure_helper/content_understanding.py	Switch to GA API version and new :analyzeBinary/knowledgeSources//files/{id} behaviors.
src/ContentProcessor/src/libs/azure_helper/model/content_understanding.py	Update response models for GA (structured warnings, relaxed Page fields, add top-level paragraphs).
src/ContentProcessor/tests/unit/azure_helper/test_content_understanding_model.py	Update unit tests for GA apiVersion and structured warnings / optional Page fields.
src/tests/ContentProcessor/azure_helper/test_content_understanding_model.py	Mirror GA model/unit test updates in the parallel test suite.
infra/main.bicep	Remove CU-specific account/location + private endpoint, restrict AI Services regions, add RBAC on unified account, reroute CU endpoint.
infra/main_custom.bicep	Same as main.bicep for custom deployment path.
infra/main.json	Regenerated ARM template reflecting consolidation/removals and updated parameters/role assignments.
infra/main.parameters.json	Remove contentUnderstandingLocation parameter mapping.
infra/main.waf.parameters.json	Remove contentUnderstandingLocation parameter mapping for WAF deployments.
.github/workflows/deploy.yml	Stop passing contentUnderstandingLocation override during deployments.
docs/CustomizingAzdParameters.md	Remove AZURE_ENV_CU_LOCATION, update AI Services location guidance and Standard/GlobalStandard note.
docs/LocalDevelopmentSetup.md	Update example CU endpoint host to aif-{suffix} (unified account).
docs/TroubleShootingSteps.md	Update CU 403 troubleshooting guidance for unified account and DNS/private endpoint expectations.

Comments suppressed due to low confidence (1)

src/ContentProcessor/src/libs/azure_helper/content_understanding.py:327

• get_image_from_analyze_operation() is now documented as retrieving a generic generated file via /files/{id}, but it still (a) uses the parameter name image_id and (b) asserts Content-Type == "image/jpeg". That assertion can fail for non-JPEG file types (or if the service changes MIME types) and would raise unexpectedly in production. Either narrow the method back to images-only (and document that guarantee) or relax the check / return the content without asserting a specific MIME type.

    def get_image_from_analyze_operation(
        self, analyze_response: Response, image_id: str
    ):
        """Retrieves a generated file (e.g., a rendered page image) from a
        completed analyze operation by its file id / path.

        In Content Understanding GA the file-retrieval URL changed from
        ``{operationLocation}/images/{imageId}`` to
        ``{operationLocation}/files/{fileId}`` (where ``operationLocation`` now
        ends in ``/analyzerResults/{operationId}``).

        Args:
            analyze_response (Response): The response object from the analyze operation.
            image_id (str): The id (or path) of the file to retrieve.
        Returns:
            bytes: The file content as a byte string.
        """
        operation_location = analyze_response.headers.get("operation-location", "")
        if not operation_location:
            raise ValueError(
                "Operation location not found in the analyzer response header."
            )
        operation_location = operation_location.split("?api-version")[0]
        image_retrieval_url = (
            f"{operation_location}/files/{image_id}?api-version={self._api_version}"
        )
        try:
            response = requests.get(url=image_retrieval_url, headers=self._headers)
            response.raise_for_status()

            assert response.headers.get("Content-Type") == "image/jpeg"

            return response.content

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

src/ContentProcessor/src/libs/azure_helper/content_understanding.py:327

• get_image_from_analyze_operation() now targets the GA /files/{id} endpoint and the docstring describes retrieving a generic generated file, but the code still asserts Content-Type == image/jpeg. If the service returns a different media type (or omits the header), this will raise AssertionError and bypass the RequestException handler. Consider removing the assert or validating/handling content types more robustly (and raising a normal exception on unexpected types).

        try:
            response = requests.get(url=image_retrieval_url, headers=self._headers)
            response.raise_for_status()

            assert response.headers.get("Content-Type") == "image/jpeg"

            return response.content

[Copilot]

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/ContentProcessor/src/libs/azure_helper/content_understanding.py:326

• get_image_from_analyze_operation() now describes retrieving a generic generated file via the GA /files/{id} route, but it still asserts Content-Type == 'image/jpeg'. This will raise an AssertionError if the service returns a different image type (or any non-JPEG file), even when the download is successful. Consider removing the strict assert or relaxing it (e.g., validate an image/ prefix only when the caller expects an image).

            response = requests.get(url=image_retrieval_url, headers=self._headers)
            response.raise_for_status()

            assert response.headers.get("Content-Type") == "image/jpeg"