Add GlobalSecondaryIndexDefinition for GSI container support#47468
Add GlobalSecondaryIndexDefinition for GSI container support#47468mbhaskar wants to merge 5 commits into
Conversation
Implement client-side support for Global Secondary Index (GSI) containers in the Python Cosmos SDK, mirroring the Java SDK implementation. Changes: - Add GlobalSecondaryIndexDefinition class with serialization/deserialization - Add global_secondary_index_definition keyword to create_container, create_container_if_not_exists, and replace_container (sync and async) - Implement dual-write pattern: writes to both globalSecondaryIndexDefinition and materializedViewDefinition keys for backward compatibility - Add comprehensive unit tests (20 tests) - Update CHANGELOG Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add 'cosmosGSI' pytest mark to pytest.ini - Add GSITestConfig matrix entry in live-platform-matrix.json - Add GSI_ACCOUNT_HOST/GSI_ACCOUNT_KEY env var mappings from Key Vault secrets (gsi-pipeline-uri, gsi-pipeline-key) in tests.yml - Create test_global_secondary_index_live.py with live tests for: - Creating GSI container with GlobalSecondaryIndexDefinition class - Creating GSI container with raw dict - create_container_if_not_exists with GSI definition Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
fef6964 to
713f56a
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds client-side support for Cosmos DB Global Secondary Index (GSI) containers in azure-cosmos, including a new GlobalSecondaryIndexDefinition model, plumbing through container create/replace APIs (sync + async) with a dual-write wire format, and CI/live-test wiring for a dedicated GSI-enabled account.
Changes:
- Added
GlobalSecondaryIndexDefinition(validation + (de)serialization) and exported it fromazure.cosmos. - Added
global_secondary_index_definitionkwarg support tocreate_container,create_container_if_not_exists, andreplace_containerfor both sync and async database clients (writing bothglobalSecondaryIndexDefinitionandmaterializedViewDefinition). - Added unit + live tests plus pipeline/matrix/marker updates to run GSI live coverage.
Show a summary per file
| File | Description |
|---|---|
| sdk/cosmos/tests.yml | Injects GSI account secrets as env vars for pipeline live runs. |
| sdk/cosmos/live-platform-matrix.json | Adds a dedicated matrix entry to run cosmosGSI marked live tests. |
| sdk/cosmos/azure-cosmos/tests/test_global_secondary_index.py | Unit tests for constructor validation and dict round-tripping. |
| sdk/cosmos/azure-cosmos/tests/test_global_secondary_index_live.py | Live tests that exercise creating GSI containers (class + raw dict). |
| sdk/cosmos/azure-cosmos/pytest.ini | Registers the new cosmosGSI pytest marker. |
| sdk/cosmos/azure-cosmos/CHANGELOG.md | Documents the new GSI feature in the Unreleased section. |
| sdk/cosmos/azure-cosmos/azure/cosmos/database.py | Adds sync API kwarg plumbing + dual-write of GSI definition in create/replace. |
| sdk/cosmos/azure-cosmos/azure/cosmos/aio/_database.py | Async mirror of sync GSI kwarg plumbing + dual-write behavior. |
| sdk/cosmos/azure-cosmos/azure/cosmos/_global_secondary_index.py | Introduces GlobalSecondaryIndexDefinition model implementation. |
| sdk/cosmos/azure-cosmos/azure/cosmos/init.py | Exports GlobalSecondaryIndexDefinition from the public package namespace. |
Copilot's findings
Comments suppressed due to low confidence (1)
sdk/cosmos/azure-cosmos/CHANGELOG.md:11
- The new changelog entry under "Features Added" is missing the usual reference/link to the PR (most other bullets in this section end with "See PR ####" or similar). Keeping the same format makes it easier to trace changes back to discussions and reviews.
#### Bugs Fixed
#### Other Changes
- Files reviewed: 10/10 changed files
- Comments generated: 1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
@sdkReviewAgent-2 |
|
/azp run python - cosmos - tests |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
✅ Review complete (38:22) Posted 5 inline comment(s). Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage |
|
✅ Review complete (38:58) Posted 6 inline comment(s). Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage |
|
✅ Review complete (47:54) Posted 7 inline comment(s). Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage |
|
✅ Review complete (05:21) Posted 7 inline comment(s). Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage |
allenkim0129
left a comment
There was a problem hiding this comment.
Thank you for adding this. Overall, the updates look good to me, and Annie left good comments, but I have a comment related to the typehint. Please address it. Thanks!
- Move CHANGELOG entry to 4.16.2 (Unreleased) section - Add missing global_secondary_index_definition to sync replace_container overload 2 - Fix type hints: Optional[Any] -> Optional[Union[GlobalSecondaryIndexDefinition, dict[str, Any]]] - Add **provisional** marker to class and all kwarg docstrings - Add GSI docstrings to all async overloads (not just implementation methods) - Use public import in live tests - Add dual-write unit test Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wrap :keyword docstring lines to stay within 120 char limit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Summary
Implements Global Secondary Index (GSI) container support in the Python Cosmos SDK, mirroring the Java SDK implementation (Azure/azure-sdk-for-java#48480).
A GSI container is a derived container built from a source container using a SQL projection query. The service already supports this; this PR adds the client-side support.
Changes
New files
azure/cosmos/_global_secondary_index.py—GlobalSecondaryIndexDefinitionclass with constructor validation, serialization (_to_dict/_from_dict), and read-only server-populated properties (source_container_rid,status)tests/test_global_secondary_index.py— 20 unit tests covering constructor validation, serialization, deserialization, forward-compatibilitytests/test_global_secondary_index_live.py— Live integration tests using dedicated GSI test accountModified files
azure/cosmos/__init__.py— ExportGlobalSecondaryIndexDefinitionazure/cosmos/database.py— Addedglobal_secondary_index_definitionkwarg tocreate_container,create_container_if_not_exists, andreplace_container(all overloads) with dual-write patternazure/cosmos/aio/_database.py— Async mirror of all sync changesCHANGELOG.md— Feature entrypytest.ini— AddedcosmosGSImarker for live testslive-platform-matrix.json— AddedGSITestConfigmatrix entrytests.yml— Mappedgsi-pipeline-uri/gsi-pipeline-keyKV secrets to env varsDesign Decisions
Dual-Write Pattern
For backward compatibility with older service versions, the SDK writes the GSI definition under both:
"globalSecondaryIndexDefinition"(new key)"materializedViewDefinition"(legacy key)On read, the new key is preferred with fallback to the legacy key.
Status as String Constants (not enum)
Status values (
"Initializing","InitialBuildAfterCreate","InitialBuildAfterRestore","Active","DeleteInProgress") are kept as plain strings for forward-compatibility with future service-defined values.Pattern Consistency
Follows the same integration pattern as
computed_properties,vector_embedding_policy,full_text_policy, andchange_feed_policy— kwarg extraction viakwargs.pop(), added to definition dict if not None.Testing
gsi-pipeline-uri/gsi-pipeline-keyfrom KV (same secrets used by Java SDK pipeline)