Adding more scenario-based tests#49568
Open
FabianMeiswinkel wants to merge 15 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new set of Cosmos “customer workflow” scenario-based live tests (fault-injection focused), wires them into Maven profiles/TestNG suites, and updates the Cosmos live test matrix to run these workflows in CI.
Changes:
- Added new
com.azure.cosmos.workflows.customer.*workflow test suite covering request options, latest-committed, session token behavior, CFP restart, availability fault matrices, circuit breaker, timeouts, stored procedures, and single-master multi-region scenarios. - Introduced new TestNG suite XMLs + Maven profiles (
fi-customer-workflows,fi-sm-customer-workflows) to run these workflows independently. - Updated Cosmos live platform matrix and suite setup/shutdown groups to include the new workflow groups.
Show a summary per file
| File | Description |
|---|---|
| sdk/cosmos/live-platform-matrix.json | Adds new CI matrix legs / display names to run the new workflow profiles (including single-master multi-region). |
| sdk/cosmos/azure-cosmos-tests/src/test/resources/fi-sm-customer-workflows-testng.xml | New TestNG suite for single-master customer workflows group. |
| sdk/cosmos/azure-cosmos-tests/src/test/resources/fi-customer-workflows-testng.xml | New TestNG suite for multi-master customer workflows group. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowTestBase.java | New shared base providing region discovery, container helpers, diagnostics assertions, and FI rule helpers for workflow tests. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowStoredProcedureTest.java | Adds stored procedure workflow test with script logging and metadata delay FI coverage. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowSingleMasterAvailabilityTest.java | Adds single-master multi-region workflows validating routing/availability behavior under faults and excluded regions. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowSessionTokenTest.java | Adds workflow validating advanced session token behavior for readMany. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowRequestOptionsTest.java | Adds chained CRUD/query/readMany workflow validating request option propagation via diagnostics. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowPartitionLevelCircuitBreakerTest.java | Adds PCLB-oriented workflow validating circuit breaker and query plan diagnostics shape. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowLatestCommittedTest.java | Adds latest-committed workflows across read/query/readMany/change feed, plus FI fault coverage. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowHighE2ETimeoutTest.java | Adds response-delay + E2E policy workflows over multiple operation types. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowDaoStyleOperationsTest.java | Adds DAO-style workflow coverage for CRUD/readAll/patch/batch/bulk with diagnostics assertions. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowChangeFeedProcessorTest.java | Adds latest-version CFP workflow with restart and injected read-feed delay, using polling instead of sleeps. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowAvailabilityFaultMatrixTest.java | Adds representative FI scenario matrix across operations/error families with diagnostics assertions. |
| sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/TestSuiteBase.java | Includes new workflow groups in suite-level setup/shutdown. |
| sdk/cosmos/azure-cosmos-tests/pom.xml | Adds new Maven profiles to run the new workflow suites via failsafe/TestNG XML. |
| sdk/cosmos/azure-cosmos-tests/CUSTOMER_WORKFLOW_COVERAGE_MAP.md | Documents coverage mapping and porting rules for the new customer workflow suite. |
Copilot's findings
- Files reviewed: 17/17 changed files
- Comments generated: 8
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Member
Author
|
/azp run java - cosmos – tests |
|
No pipelines are associated with this pull request. |
Member
Author
|
/azp run java - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run java - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run java - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Member
Author
|
/azp run java - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
jeet1995
approved these changes
Jun 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR adds scenario-based tests - the original test-suite was provided by a customer who used these for release valdiations. Below is an analysis of the tests and triage of gap analysis and brief summary what additional coverage these tests provide.
test/CosmosDriverDynamicRequestOptionTest.javaOperationPoliciesTest; initial triage:partialworkflow gap.azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/CustomerWorkflowRequestOptionsTest.javatest/CosmosMultiFeatureTests.javaCosmosDiagnosticsTest,ExcludeRegionTests, andSessionConsistencyWithRegionScopingTests; initial triage:partial.CustomerWorkflowRequestOptionsTest.java,CustomerWorkflowSessionTokenTest.javatest/CosmosDaoTest.javaCosmosItemTest,DocumentCrudTest,CosmosBulkTest,CosmosBulkAsyncTest,TransactionalBatchTest, andCosmosItemSerializerTest; initial triage:partial/wrapper-specific.CustomerWorkflowDaoStyleOperationsTest.javatest/Latest_Committed_Tests.javaCosmosLatestCommittedItemTests,GatewayReadConsistencyStrategyE2ETest, andClientRetryPolicyE2ETests; initial triage:partial.CustomerWorkflowLatestCommittedTest.javaregression/direct/*.javaCosmosLatestCommittedItemTestsplus change feed tests; initial triage:partial.CustomerWorkflowLatestCommittedTest.javaregression/gateway/*.javaGatewayReadConsistencyStrategyE2ETestand spy-wire tests; initial triage:duplicate/partial.CustomerWorkflowLatestCommittedTest.javatest/CosmosHighE2ETimeoutTest.javaEndToEndTimeOutValidationTests,EndToEndTimeOutWithAvailabilityTest, andFaultInjectionWithAvailabilityStrategyTestsBase; initial triage:partial.CustomerWorkflowHighE2ETimeoutTest.javatest/CosmosStoredProcedureTest.javagap/partial.CustomerWorkflowStoredProcedureTest.javatest/ChangeFeedProcessorTest.javapartial.CustomerWorkflowChangeFeedProcessorTest.javatest/PartitionLevelCircuitBreakerTests.javaPerPartitionCircuitBreakerE2ETestsandPerPartitionAutomaticFailoverE2ETests; initial triage:partial.CustomerWorkflowPartitionLevelCircuitBreakerTest.javatest/Cosmos429test.javaduplicate/partial.CustomerWorkflowAvailabilityFaultMatrixTest.java,CustomerWorkflowSingleMasterAvailabilityTest.java,CustomerWorkflowHighE2ETimeoutTest.javatest/CosmosConflictResolutionTest.javaCosmosConflictsTest,ConflictTests, andMultiMasterConflictResolutionTest; initial triage:duplicate/partial.multimaster/direct/*.javaFaultInjectionWithAvailabilityStrategyTestsBase,FITests_*, and PPAF/PCLB tests; initial triage:partial.CustomerWorkflowAvailabilityFaultMatrixTest.javamultimaster/gateway/*.javapartial.CustomerWorkflowAvailabilityFaultMatrixTest.javamultimaster/direct/BasicBehaviorMultiMasterTest.javaduplicate/partial.CustomerWorkflowDaoStyleOperationsTest.java,CustomerWorkflowRequestOptionsTest.java,CustomerWorkflowLatestCommittedTest.java,CustomerWorkflowSessionTokenTest.java,CustomerWorkflowAvailabilityFaultMatrixTest.javamultimaster/direct/CosmosFeatureValidationMultiMasterTest.javaduplicate/partial.azure-cosmos-tests/src/test/java/com/azure/cosmos/workflows/customer/*multimaster/direct/CosmosSessionTokenMismatchMultiMasterTest.javaduplicate/partial.CustomerWorkflowSessionTokenTest.javaAll SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines