Fixes 28727: flatten testCaseResolutionStatusDetails index mapping to match its schema (Part A)

[TeddyCr]

Describe your changes:

Fixes #28727

Part A only. Issue #28727 has two parts. This PR fixes Part A (the testCaseResolutionStatusDetails mapping↔schema mismatch). Part B — validating highlight fields at Search Settings save time by reusing SearchSourceBuilderFactory.isHighlightUnsafeField — is explicitly out of scope here and remains open. Please do not close the issue on merge beyond Part A.

I flattened the testCaseResolutionStatusDetails index mapping so it matches its JSON Schema, and repointed the resolution-comment search boost at the path documents actually carry.

The mapping declared a resolved wrapper object that the schema makes structurally impossible. testCaseResolutionStatus.json models the field as oneOf [assigned.json, resolved.json], and both branches are "additionalProperties": false — resolved.json declares testCaseFailureReason / testCaseFailureComment / resolvedBy directly. So a resolved key could never legally appear in a document, and indeed never did: doc building is generic (SearchIndex → JsonUtils.getMap(entity); TestCaseResolutionStatusIndex only adds fqnParts, @timestamp, and parent relationships). The consequence was a dead 10× boost — testCaseResolutionStatusDetails.resolved.testCaseFailureComment in TestCaseResolutionStatusIndex.getFields() matched nothing, in every language.

"dynamic": true on that node was masking the bug: it indexed the real flat fields anyway, but as standard-analyzer text rather than the declared types. That broke exact match on testCaseFailureReason (analyzed text never matches an exact term) and broke CJK/Cyrillic comment search (the standard analyzer splits Japanese/Chinese into single characters and does not stem Russian).

Type of change:

• Bug fix

High-level design:

Approach — the issue's option 1: correct the mapping to the schema, don't reshape the document. The schema and the document are already flat and are the source of truth; only the mapping was wrong. Reshaping the document to match the mapping would mean writing a wrapper key that additionalProperties: false forbids.

Files changed (7):

File	Change
openmetadata-spec/.../elasticsearch/{en,jp,ru,zh}/test_case_resolution_status_index_mapping.json	Move testCaseFailureReason / testCaseFailureComment / resolvedBy up one level, out of the resolved wrapper. Pure unwrap — no other edit.
openmetadata-service/.../search/indexes/TestCaseResolutionStatusIndex.java	Boost repointed to testCaseResolutionStatusDetails.testCaseFailureComment (one line).
openmetadata-service/src/test/.../IndexMappingNestedFieldConsistencyTest.java	New mapping↔schema invariant.
openmetadata-integration-tests/.../SearchConsumerFieldBehaviorIT.java	Resolved-variant document + 3 behavioral tests.

There is no separate OpenSearch resource tree — the same four files serve both engines, with OsUtils.enrichIndexMappingForOpenSearch rewriting at runtime.

Per-locale analyzers deliberately preserved, not homogenized. en/ru keep om_analyzer; jp keeps om_analyzer_jp (kuromoji) and its fields.ngram; zh keeps ik_max_word and its fields.ngram. Verified byte-for-byte: en ≡ ru in this subtree, and unwrapping the base resolved object with jq yields output identical to this branch in all four locales.

"dynamic": true kept — deliberate. After flattening, every property of both oneOf branches is mapped explicitly, so dynamic is no longer load-bearing for correctness. It is retained as the forward-compatibility net: a property added to assigned.json / resolved.json still gets indexed rather than silently dropped, and the new unit test fires immediately in that case. Flipping it to false is a separate hardening decision and would couple a silent-data-loss behavior change to a mapping correction.

Alternative rejected: rewriting the document in TestCaseResolutionStatusIndex to emit a resolved wrapper. That would contradict the schema, break the UI and Python (both already read/write the flat shape), and invalidate the existing testCaseResolutionStatusDetails.assignee.name filter pattern.

---

⚠️ Upgrade note — this requires a reindex

This change requires a reindex of test_case_resolution_status_search_index. migrate alone is not sufficient and will log a mapping-update failure for that index.

migrate reaches searchRepository.updateIndexes() (OpenMetadataOperations.java:1196), which issues PUT _mapping. On any cluster that has ever resolved an incident, dynamic: true has already created those fields as text, so declaring them keyword is an incompatible type change and the request is rejected 400:

illegal_argument_exception: mapper [testCaseResolutionStatusDetails.<field>]
cannot be changed from type [text] to [keyword]

Three things worth knowing:

1. The rejection is atomic — no property in that request lands, not just the four here. Any unrelated mapping change shipped for this index in the same release is discarded with it. (Verified by adding an unrelated probe field to the same request and confirming it did not land either.)
2. Which field the engine names first is engine-dependent — OpenSearch 3.4.0 reported …testCaseFailureReason, Elasticsearch 9.3.0 reported …resolvedBy.id for the identical request. Same 400, same atomicity; a different field name is not a different bug.
3. The cause is swallowed on the OpenSearch path — OpenSearchIndexManager logs "Failed to Update Open Search index {}" without the exception, so an operator sees one bare error line and a silently stale mapping.

The supported path is ./bootstrap/openmetadata-ops.sh reindex. It is also the only path that drops the stale resolved sub-object, since PUT _mapping cannot remove a field. IndexMappingVersionTracker hashes each mapping, so the smart-reindex plan picks up testCaseResolutionStatus and recreates just that index — no DB migration is involved.

Mitigating point: because "dynamic": true was retained, un-reindexed clusters still get a partial improvement immediately — the repointed boost now targets a path dynamic mapping has already created (as standard-analyzer text) rather than a path with no values at all. Full correctness (declared keyword, per-locale analyzers) arrives with the reindex.

Tests:

Use cases covered

• Searching the Incident Manager index for a word that appears only in an incident's resolution comment returns that incident (previously returned nothing, in every language).
• The Resolved branch of testCaseResolutionStatusDetails is indexed and searchable at all — previously no test anywhere indexed a Resolved-variant document.
• Comment search works in Japanese, Chinese and Russian, not just English, because the per-locale analyzer is applied instead of the standard-analyzer fallback.
• testCaseFailureReason supports exact-match term lookup, i.e. its declared keyword type is actually in effect.
• Existing Assigned-branch behavior (status filter, assignee.name filter, test-case/origin-entity filters) is unchanged.

Unit tests

• I added unit tests for the new/changed logic.
• Files updated: openmetadata-service/src/test/java/org/openmetadata/service/search/IndexMappingNestedFieldConsistencyTest.java
• resolutionStatusDetailsMappingMustMatchSchemaProperties asserts the mapped subfield set equals the union of the properties of every oneOf branch, read from the JSON Schema at runtime rather than hardcoded — so adding a property to assigned.json / resolved.json fails this test until the mappings follow. It guards the class of bug, not just this instance.
• Result: Tests run: 7, Failures: 0, Errors: 0, Skipped: 0 (1 failure before the fix).

Coverage on the changed class — measured, and honestly short of the 90% target. Ran mvn -P static-code-analysis -pl openmetadata-service test jacoco:report scoped to the search tests. TestCaseResolutionStatusIndex.java line coverage is 62.8% (27/43):

Method	Lines
<init>	1/1	100%
buildSearchIndexDocInternal	5/5	100%
getEntityTypeName	1/1	100%
setParentRelationships	20/28	71.4%
getEntity	0/1	0%
getFields — the method this PR changes	0/7	0%

The 0% on getFields is a measurement artifact, not an untested line: getFields() is executed directly by the new integration test (incidentResolutionCommentSearchReturnsIncidentInAllLanguages builds its query from TestCaseResolutionStatusIndex.getFields()), and that test lives in openmetadata-integration-tests — a separate Maven module whose execution this -pl openmetadata-service jacoco run does not observe. The changed line is covered by an automated test that fails without the fix; it is simply not covered by a unit test in the measured module. The remaining gaps (setParentRelationships, getEntity) are pre-existing and untouched by this PR.

Backend integration tests

• I added integration tests in openmetadata-integration-tests/.
• File updated: openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/SearchConsumerFieldBehaviorIT.java

This class boots a real OpenSearch 3.4.0 testcontainer with analysis-kuromoji and analysis-ik, creates the real mapping for every language in IndexMappingLanguage (en/jp/ru/zh), indexes real documents and runs the actual consumer queries. I added a Resolved-variant document alongside the existing Assigned one, plus three tests:

Test	What it asserts	Failed before the fix?
incidentResolutionCommentSearchReturnsIncidentInAllLanguages	The production consumer query — a most_fields / operator:and multi_match built from TestCaseResolutionStatusIndex.getFields() — returns the incident when searching a word unique to its comment. Because the field list comes from production code, repointing a boost at a dead path fails this test.	Yes — is broken in language(s): [en, jp, zh, ru], 0 hits everywhere
failureReasonSupportsExactMatchTermInAllLanguages	A term on testCaseResolutionStatusDetails.testCaseFailureReason returns the incident, i.e. the declared keyword is in effect. A type probe, not a product feature — nothing filters on the reason today; it is what distinguishes "explicitly mapped" from "accidentally indexed by dynamic:true".	Yes — is broken in language(s): [en, jp, zh, ru]
resolutionCommentUsesLanguageAnalyzerInAllLanguages	_analyze on the comment field yields each locale's expected token, proving the per-locale analyzer survived the flatten.	Yes — ru expected 'продаж' but got [ежемесячные, продажи], zh expected '销售' but got [北, 京, 大, 学, 的, 销, 售, 报, 表], jp expected '東京' but got [東, 京, タワー, の, 売, 上]

• Result: Tests run: 20, Failures: 0, Errors: 0, Skipped: 0 (3 failures before the fix; the 17 pre-existing tests passed throughout, so the added document perturbs nothing).

Elasticsearch — verified by hand only. The automated IT runs OpenSearch 3.4.0. On a real ES 9.3.0 container the branch mapping creates cleanly, the flat Resolved document indexes, and both the boost query and the testCaseFailureReason term filter return 1 hit — against 0 hits on the base mapping. This was done manually during review; no automated ES coverage exists for this index, so a future ES-only regression would not be caught by CI.

Ingestion integration tests

• Not applicable — no ingestion changes. ingestion/.../sample_data.py already writes the flat shape and needed no change.

Playwright (UI) tests

• Not applicable — no UI changes. The UI already reads the flat shape (InlineTestCaseIncidentStatus.component.tsx), which is part of the evidence that the mapping, not the document, was wrong.

Manual testing performed

Steps 1 and the reindex/mapping inspection below are the reproducible procedure; steps marked (not verified) were not executed against a running OpenMetadata stack — the equivalent effects are covered by the automated IT against a real engine, and the PUT _mapping rejection in step 3a was reproduced directly against both engines rather than through openmetadata-ops.sh.

1. Build and install: mvn install -pl openmetadata-spec -DskipTests then mvn install -nsu -pl openmetadata-service -DskipTests.
2. (not verified) Bring up a stack: docker compose -f docker/development/docker-compose.yml up -d; log in as admin.
3. Reindex: ./bootstrap/openmetadata-ops.sh reindex. Expect Index mapping changed for entity: testCaseResolutionStatus and only that entity in the plan; a second reindex reports no changes.
   • 3a. Confirm the migrate-only path fails as described in the upgrade note above (400, atomic, cause swallowed on OpenSearch).
4. Confirm the live mapping is flat and correctly typed:

   curl -s 'http://localhost:9200/test_case_resolution_status_search_index/_mapping' \
     | jq '..|.testCaseResolutionStatusDetails? // empty | .properties | keys'
   

   Expect ["assignee","resolvedBy","testCaseFailureComment","testCaseFailureReason"] and no resolved. Then check testCaseFailureComment carries "type":"text" with the locale's analyzer (om_analyzer en/ru, om_analyzer_jp jp, ik_max_word zh) and, for jp/zh, a fields.ngram subfield.
5. (not verified) Produce a resolved incident: Data Quality → a failing test case → Incident Manager → status Resolved, reason FalsePositive, comment flakyupstream feed was backfilled.
6. (not verified) See the boost take effect — query by a word appearing only in the comment:

   curl -s -u admin:admin \
     'http://localhost:8585/api/v1/search/query?index=test_case_resolution_status_search_index&q=flakyupstream&from=0&size=10'
   

   Pre-fix hits.total.value: 0; post-fix the incident is returned. Re-run with &explain=true and confirm the winning clause is testCaseResolutionStatusDetails.testCaseFailureComment^10.0, ranking a comment match above a testCaseResolutionStatusType^1.0 match.
7. (not verified) Confirm no Assigned-branch regression: the Incident Manager list still filters by assignee (SearchListFilter → testCaseResolutionStatusDetails.assignee.name) and the incident detail panel still renders reason/comment/resolvedBy.

Gates run on the rebased branch (spec + service installed from the working tree first):

Gate	Result
mvn spotless:check -nsu -pl openmetadata-service,openmetadata-integration-tests	0 needs changes to be clean in both modules, BUILD SUCCESS
mvn test -nsu -pl openmetadata-service -Dtest='org.openmetadata.service.search.**.*Test,SearchListFilterTest'	Tests run: 2231, Failures: 0, Errors: 0, Skipped: 0
mvn test -nsu -pl openmetadata-integration-tests -Dtest=SearchConsumerFieldBehaviorIT	Tests run: 20, Failures: 0, Errors: 0, Skipped: 0

UI screen recording / screenshots:

Not applicable — this PR contains no UI changes. The diff touches only ES/OS mapping resources, one backend boost map, and two test files; nothing under openmetadata-ui/.

Checklist:

• I have read the CONTRIBUTING document.
• My PR title is Fixes <issue-number>: <short explanation>
• My PR is linked to a GitHub issue via Fixes #<issue-number> above.
• I have commented on my code, particularly in hard-to-understand areas.
• For JSON Schema changes: no JSON Schema changed. The four edited files are ES/OS index-mapping resources under openmetadata-spec/src/main/resources/elasticsearch/, not JSON Schemas under json/schema/ — they drive no code generation, so make generate does not apply. No SQL migration is needed either: mapping drift is tracked by content hash in IndexMappingVersionTracker and resolved by reindex (see the upgrade note). Precedent: 14b3a57 (fix(search): add nested customPropertiesTyped mapping to 5 entity indexes #30501) changed 20 mapping JSONs with no SQL and no version bump.
• For UI changes: not applicable — no UI changes.
• I have added tests (unit / integration) and listed them above.

• I have added a test that covers the exact scenario we are fixing. All four new tests fail on main and pass here; the RED output is quoted in the tables above.

🤖 Generated with Claude Code

Greptile Summary

The PR aligns the test-case resolution-status search mapping with the schema’s flat resolved-details shape and updates the resolution-comment boost to target the field documents actually contain.

• Flattens testCaseFailureReason, testCaseFailureComment, and resolvedBy in all four locale mappings while preserving locale-specific analyzers.
• Adds a schema-to-mapping consistency invariant across supported languages.
• Adds real OpenSearch coverage for resolved incidents, exact failure-reason matching, multilingual analysis, and resolution-comment search.

Confidence Score: 5/5

The PR appears safe to merge, with the documented reindex requirement remaining necessary for existing deployments.

The flattened mappings match the closed schema branches and serialized document shape, the search boost targets the resulting live field, and the added tests exercise the corrected behavior across every supported mapping language.

Important Files Changed

Filename	Overview
openmetadata-service/src/main/java/org/openmetadata/service/search/indexes/TestCaseResolutionStatusIndex.java	Repoints the resolution-comment search boost from the impossible wrapped path to the flat schema-backed field.
openmetadata-service/src/test/java/org/openmetadata/service/search/IndexMappingNestedFieldConsistencyTest.java	Adds a deterministic invariant comparing each locale mapping’s resolution-detail children with the union of the schema’s closed oneOf branches.
openmetadata-integration-tests/src/test/java/org/openmetadata/it/tests/SearchConsumerFieldBehaviorIT.java	Adds a resolved document and behavioral checks for boosted comment search, keyword matching, and locale-specific analysis.
openmetadata-spec/src/main/resources/elasticsearch/en/test_case_resolution_status_index_mapping.json	Flattens resolved-detail properties and preserves the English analyzer and keyword subfields.
openmetadata-spec/src/main/resources/elasticsearch/jp/test_case_resolution_status_index_mapping.json	Flattens resolved-detail properties while preserving Japanese analysis and the ngram subfield.
openmetadata-spec/src/main/resources/elasticsearch/ru/test_case_resolution_status_index_mapping.json	Flattens resolved-detail properties while preserving the Russian analyzer configuration.
openmetadata-spec/src/main/resources/elasticsearch/zh/test_case_resolution_status_index_mapping.json	Flattens resolved-detail properties while preserving IK analysis and the ngram subfield.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[TestCaseResolutionStatus entity] --> B[Generic JSON search document]
  B --> C[testCaseResolutionStatusDetails]
  C --> D[assignee]
  C --> E[testCaseFailureReason]
  C --> F[testCaseFailureComment]
  C --> G[resolvedBy]
  F --> H[Locale-specific analyzer]
  F --> I[Boosted incident search query]

Loading

_{Reviews (1): Last reviewed commit: "Fixes #28727: name the failure-reason te..." | Re-trigger Greptile}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

• Issue Search follow-ups: testCaseResolutionStatusDetails mapping↔schema mismatch & save-time highlight-field validation #28727 is missing Shipping → Source.
• Issue Search follow-ups: testCaseResolutionStatusDetails mapping↔schema mismatch & save-time highlight-field validation #28727 is missing Shipping → Priority.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

[gitar-bot]

Code Review ✅ Approved

Flattens the testCaseResolutionStatusDetails index mapping to match its JSON schema, repointing the incident comment boost and enabling multi-language search support. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}