Expose system index metadata in CAT indices

[cwperks]

Description

Adds system-index discovery to /_cat/indices using the system metadata and descriptors already maintained by OpenSearch.

The system query parameter filters results:

• /_cat/indices?system=true returns only system indices.
• /_cat/indices?system=false returns only non-system indices.
• Omitting system preserves the existing behavior and returns both.

The response supports two columns:

• system (sys): whether IndexMetadata.isSystem() is true.
• system.description (sysdesc): the description from the matching SystemIndexDescriptor, when available.

These columns do not change the default output when system is omitted. They can be selected explicitly with h=system,system.description, and are shown by default when the system filter is specified.

Rationale

OpenSearch previously had no API for operators to identify indices registered through SystemIndexPlugin.getSystemIndexDescriptors() or inspect their descriptions without reading plugin source.

Since this PR was originally opened, OpenSearch added and maintains IndexMetadata.isSystem(). This update uses that metadata as the filtering source of truth, addressing the earlier review concern that system status should first be represented in index metadata. The descriptor registry remains the source for the human-readable description.

This reports OpenSearch Core's system-index classification. Security-plugin protection and authorization remain separate concerns and continue to determine which indices a caller may access.

Changes

• Register system as a supported CAT response/query parameter.
• Document it in the REST API specification.
• Filter from IndexMetadata.isSystem() for consistent cluster-state behavior.
• Expose stable system and system.description columns.
• Cover system-only, non-system-only, descriptor, and default-output behavior.
• Sync the branch with current main.

Testing

./gradlew :server:test \
  --tests org.opensearch.rest.action.cat.RestIndicesActionTests \
  --tests org.opensearch.action.RenamedTimeoutRequestParameterTests \
  :server:precommit \
  :rest-api-spec:validateRestSpec \
  :rest-api-spec:validateNoKeywords

All listed checks pass locally.

Checklist

• New functionality includes focused tests.
• REST API specification updated.
• Server precommit checks pass.
• Commits are signed off for DCO.

[github-actions]

Compatibility status:

Checks if related components are compatible with change 7ad8bc7

Incompatible components

Incompatible components: [https://github.com/opensearch-project/performance-analyzer.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/performance-analyzer-rca.git]

[github-actions]

❌ Gradle check result for 4af2dd5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 8b35852: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 7ad8bc7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[opensearch-trigger-bot]

This PR is stalled because it has been open for 30 days with no activity.

[peternied]

@cwperks Thanks for creating this PR, but I don't think this is the right kind of change. AFAIK OpenSearch doesn't expose the notion of system index in any of the other APIs so only exposing it via a filtering parameter on cat indices it.

To move forward with this change can you map out all the externally facing places that should be updated and this change would be part of that whole. IMO the first incremental change should be to return the property as part of the index settings then plumb it elsewhere.

I'm going to close this PR, after creating an issue that lays out how this would work as a whole feel free to reopen.

[cwperks]

I will draft up an issue for this to solicit comments. There's been a lot of discussion around system indices and I was surprised to find that there wasn't any way to list out all system indices and their descriptors that are defined by SystemIndexPlugin.getSystemIndexDescriptor. I think there should be some sort of API where its possible to list them out and where users can see the descriptions w/o perusing the code of each plugin/module that defines them.

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 3b04ea1)

Here are some key observations to aid the review process:

🧪 PR contains tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ No major issues detected

[github-actions]

PR Code Suggestions ✨

Latest suggestions up to 3b04ea1

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Avoid cluttering output when filtering non-system When a user passes system=false to filter to non-system indices, the system and system.description columns become visible by default, but every row will trivially show false/null, cluttering output. Consider only enabling the columns by default when system=true is explicitly requested, otherwise keep them hidden unless requested via h. server/src/main/java/org/opensearch/rest/action/cat/RestIndicesAction.java [437-442] -final String systemColumnDefault = request.hasParam("system") ? "" : "default:false;"; +final boolean showSystemColumns = request.paramAsBoolean("system", false); +final String systemColumnDefault = showSystemColumns ? "" : "default:false;"; table.addCell("system", "alias:sys;" + systemColumnDefault + "desc:whether the index is a system index"); table.addCell( "system.description", "alias:sysdesc;" + systemColumnDefault + "desc:description from the matching system index descriptor" ); Suggestion importance[1-10]: 5 __ Why: Valid UX observation: showing trivial false/null columns for system=false is noisy. However, this is a design choice and the current behavior of showing columns whenever system param is present is defensible.	Low
	Simplify optional boolean parameter handling Using request.hasParam("system") combined with paramAsBoolean bypasses the standard behavior of marking the parameter as consumed when it's absent, but more importantly, it fails to handle the case where the user explicitly passes an empty value. Consider using paramAsBoolean("system", null) directly (which returns null when the param is not present) for cleaner semantics and consistent parameter consumption tracking. server/src/main/java/org/opensearch/rest/action/cat/RestIndicesAction.java [913] -final Boolean systemParam = request.hasParam("system") ? request.paramAsBoolean("system", false) : null; +final Boolean systemParam = request.paramAsBoolean("system", (Boolean) null); Suggestion importance[1-10]: 4 __ Why: The simplification is reasonable and could improve param consumption tracking, but the current implementation is functionally correct. Minor code-quality improvement.	Low

---

Previous suggestions

Suggestions up to commit 27195f6

Category	Suggestion	Impact
General	Align column visibility with parameter value The system column visibility depends on whether the system parameter is present, but getTableWithHeader is also invoked in contexts where the request may not carry that parameter (e.g., via RestIndicesListAction). Additionally, this differs from the row-filtering logic which uses paramAsBoolean; consider gating column display on an explicit truthy value or the h parameter so users who filter by system=false don't unexpectedly see the columns as default-visible. server/src/main/java/org/opensearch/rest/action/cat/RestIndicesAction.java [437-442] -final String systemColumnDefault = request.hasParam("system") ? "" : "default:false;"; +final boolean showSystemColumns = request.paramAsBoolean("system", false); +final String systemColumnDefault = showSystemColumns ? "" : "default:false;"; table.addCell("system", "alias:sys;" + systemColumnDefault + "desc:whether the index is a system index"); table.addCell( "system.description", "alias:sysdesc;" + systemColumnDefault + "desc:description from the matching system index descriptor" ); Suggestion importance[1-10]: 5 __ Why: The suggestion highlights a legitimate inconsistency between how column visibility (based on hasParam) and row filtering (based on paramAsBoolean) treat the system parameter, which could cause confusing UX. However, the impact is minor and the proposed change (paramAsBoolean default false) may itself hide columns when system=true is passed with wrong parsing semantics.	Low
	Make Boolean unboxing explicit Comparing Boolean to boolean via != triggers auto-unboxing here which is safe only because of the != null check, but using != on Boolean object references is fragile and easy to misread. Use Boolean.booleanValue() or != after unboxing to make the intent explicit and avoid any accidental reference comparison if the code is refactored. server/src/main/java/org/opensearch/rest/action/cat/RestIndicesAction.java [957-959] -if (systemParam != null && systemParam != indexMetadata.isSystem()) { +if (systemParam != null && systemParam.booleanValue() != indexMetadata.isSystem()) { continue; } Suggestion importance[1-10]: 2 __ Why: The existing code correctly auto-unboxes Boolean when comparing to boolean after a null check. The suggested change is a stylistic preference and offers minimal improvement.	Low

Suggestions up to commit 92dfbd8

Category	Suggestion	Impact
General	Simplify system index filtering logic The system parameter logic is inverted. When system=true, only system indices should be shown, but when system=false (default), all indices should be shown. Currently, non-system indices are always included when system=false, which is correct, but the filter should also allow showing all indices by default. server/src/main/java/org/opensearch/rest/action/cat/RestIndicesAction.java [956-961] -if (systemParam) { - if (!systemIndices.isSystemIndex(indexName)) { - // System filter is enabled but index is not a system index - continue; - } +if (systemParam && !systemIndices.isSystemIndex(indexName)) { + // System filter is enabled but index is not a system index + continue; } Suggestion importance[1-10]: 7 __ Why: The suggestion correctly identifies that the nested if statements can be simplified into a single condition. The logic is functionally equivalent but more concise and readable. However, this is primarily a code style improvement rather than a bug fix.	Medium
	Handle missing descriptor gracefully When system=true is set, the code already filters to only system indices. However, findMatchingDescriptor could still return null if the index matches the system index pattern but has no descriptor. This could cause inconsistent behavior where an index passes the filter but has no description. server/src/main/java/org/opensearch/rest/action/cat/RestIndicesAction.java [992-995] if (systemParam) { SystemIndexDescriptor descriptor = systemIndices.findMatchingDescriptor(indexName); - table.addCell(descriptor == null ? null : descriptor.getDescription()); + String description = (descriptor != null) ? descriptor.getDescription() : ""; + table.addCell(description); } Suggestion importance[1-10]: 5 __ Why: The suggestion proposes using an empty string instead of null when a descriptor is missing. While this could provide more consistent output, the current implementation with null is acceptable as table.addCell() likely handles null values appropriately. The change is minor and the concern about inconsistent behavior is not critical.	Low

[github-actions]

❌ Gradle check result for 92dfbd8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

Persistent review updated to latest commit 27195f6

[github-actions]

❌ Gradle check result for 27195f6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

Persistent review updated to latest commit 3b04ea1

[github-actions]

❌ Gradle check result for 3b04ea1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?