REDFISH-CLUSH

redfish-clush - Parallel Redfish REST Utility with ClusterShell Aggregation

Target Subsystem: Out-of-Band Management (BMC / Redfish API Infrastructure)

• 1 Overview
• 2 Key Features
• 3 Installation
• 4 Syntax & Usage
• 4.1 Arguments
• 5 Use Cases & Production Examples
• 5.1 Use Case 1: Cluster-Wide LED Indicator Audit
• 5.2 Use Case 2: Mass BIOS Configuration Deployment
• 5.3 Use Case 3: Cluster-Wide Firmware Inventory Audit
• 5.4 Use Case 4: Mass Power Control via Redfish Actions
• 5.5 Use Case 5: Thermal & Health Telemetry Snapshot
• 5.6 Use Case 6: BIOS Configuration Discovery & Diff
• 6 Error Handling & Debugging

Overview

redfish-clush is a high-performance administration tool engineered to execute Redfish REST API calls (GET, PATCH, POST) across large compute cluster node ranges simultaneously.

While native tools like clush (ClusterShell) operate smoothly over SSH within the operating system layer, managing hardware at the Baseboard Management Controller (BMC) layer via Redfish typically requires repetitive loop iterations, especially on vendor platforms such as Fujitsu that lack comprehensive OEM tooling. redfish-clush bridges this gap by expanding cluster node ranges natively, injecting JSON payloads for PATCH/POST operations, and aggregating identical BMC responses into a single consolidated view using an MD5 hashing algorithm — all with native JQ filtering to extract exactly the fields you need.

Key Features

• Native Node Range Expansion: Parses standard cluster bracket notation (e.g., node[01-50]) and sequentially maps them to target -bmc interfaces.
• Universal REST Methods: Supports GET, PATCH, and POST against any Redfish endpoint, covering inventory queries, configuration changes, and actions.
• Intelligent Output Deduplication: Hashes responses in real-time to group identical outputs, vastly reducing stdout noise.
• Native JQ Filtering: Injectable JQ expressions extract target fields directly from Redfish JSON responses without post-processing pipelines.
• True Parallel Execution: Utilizes background subshells to query hundreds of BMCs simultaneously, cutting execution time to seconds.
• Native JSON Diff Engine: Built-in diff subcommand compares two nodes or JSON files and identifies changed, added, or removed attributes — with --patch mode to generate a ready-to-deploy PATCH payload.
• ClusterShell Integration: Natively resolves @groups defined in /etc/clustershell/groups.conf via the nodeset command.

Installation

Since this is a private repository, ensure your environment has the proper Git credentials configured, then run the following one-liner to clone, deploy globally, and set execution permissions:

git clone -q https://github.com/doitnowgroup/redfish-clush.git /tmp/redfish-clush-tmp && sudo mv /tmp/redfish-clush-tmp/redfish-clush /usr/local/bin/redfish-clush && sudo chmod +x /usr/local/bin/redfish-clush && rm -rf /tmp/redfish-clush-tmp

Dependencies

Dependency	Required	Purpose
bash 4+	Yes	Associative arrays, advanced regex
curl	Yes	HTTPS Redfish REST transport
jq	Optional (required for diff)	JSON field filtering in GET queries; JSON diff engine
nodeset (ClusterShell)	Optional	@group resolution and compressed header folding

Syntax & Usage

Standard Mode

redfish-clush <clustershell_groups>,<node_ranges> <username> <password> <METHOD> </redfish/endpoint> [json_payload.json | jq_filter] [jq_filter]

Diff Mode

Compare two nodes directly or two saved JSON dumps to discover configuration differences and generate PATCH payloads.

redfish-clush diff <username> <password> </redfish/endpoint> <node_a> <node_b> [--filter <jq_filter>] [--patch]
redfish-clush diff <file_a.json> <file_b.json> [--filter <jq_filter>] [--patch]

Diff Options:

Option	Description
--filter <jq_filter>	Apply a JQ filter to both sources before diffing (e.g., .Attributes to isolate BIOS attributes).
--patch	Output only a JSON object with the differing keys from source A, ready to use as a PATCH payload.

File mode auto-detection: If the first argument is an existing file, the script switches to file mode. Files can be raw JSON or redfish-clush GET output (headers and [HTTP XXX] lines are stripped automatically).

Arguments

1. <node_ranges_or_@groups> (String): Comma-separated list of nodes, bracketed node matrices (e.g., cpu0[01-50],gpu0[01-10]), or native ClusterShell groups (e.g., @cpu,@vdi).
2. <username> (String): The active BMC user account (e.g., admin).
3. <password> (String): Password associated with the specified BMC user profile.
4. <METHOD> (String): HTTP method — GET, PATCH, or POST (case-insensitive).
5. </redfish/endpoint> (String): Target Redfish API URI (e.g., /redfish/v1/Systems/1).
6. [json_file \| jq_filter] (String, optional): A path to a JSON payload file (for PATCH/POST), or a JQ filter expression (for GET queries). The script auto-detects whether the argument is an existing file or a JQ expression.
7. [jq_filter] (String, optional): When argument 6 is a JSON file, argument 7 is treated as the JQ filter.

Note: All nodes are automatically suffixed with -bmc to target BMC management interfaces (e.g., cpu001 becomes cpu001-bmc).

Use Cases & Production Examples

Use Case 1: Cluster-Wide LED Indicator Audit

Scenario: Verify UID/LED indicator status across a heterogeneous compute fleet to locate physical hardware.

redfish-clush cpu0[01-50],gpu0[01-44] admin Sup3rP4ssword GET /redfish/v1/Systems/1 .IndicatorLED

Production Output:

---------------
cpu0[01-48,50],gpu0[03-44] (91)
---------------
"Off"
[HTTP 200]

---------------
cpu0[49] (1)
---------------
"Lit"
[HTTP 200]

---------------
gpu0[01-02] (2)
---------------
"Blinking"
[HTTP 200]

Critical Diagnostic Value: The aggregation instantly exposes that cpu0[49] has its UID LED lit and gpu0[01-02] are blinking — indicating active physical maintenance or pre-failure attention states that would otherwise go unnoticed.

Use Case 2: Mass BIOS Configuration Deployment

Scenario: Push a BIOS settings profile to an entire node group in a single command using a JSON payload file.

redfish-clush cpu0[01-20] admin Sup3rP4ssword PATCH /redfish/v1/Systems/1/Bios/Settings bios_profile.json

Expected Output:

---------------
cpu0[01-20] (20)
---------------
[HTTP 200]

Note: PATCH operations with a JSON payload discard the response body by default (keeping only the HTTP status code) to avoid overwhelming stdout with large Redfish task objects. The -w curl flag extracts the HTTP status code for verification.

Use Case 3: Cluster-Wide Firmware Inventory Audit

Scenario: Extract BIOS and BMC firmware versions across all compute nodes in a single command. The JQ filter combines multiple Redfish fields into a compact one-liner per node.

redfish-clush cpu0[01-50] admin Sup3rP4ssword GET /redfish/v1/Systems/1 '.BiosVersion + " | BMC: " + .FirmwareVersion'

Production Output:

---------------
cpu0[01-40] (40)
---------------
"R1.22.0 | BMC: 2.14.0"
[HTTP 200]

---------------
cpu0[41-45] (5)
---------------
"R1.20.0 | BMC: 2.12.1"
[HTTP 200]

---------------
cpu0[46-50] (5)
---------------
"R1.22.0 | BMC: 2.13.0"
[HTTP 200]

Critical Diagnostic Value: The aggregation immediately reveals a fleet split: 5 nodes on an older BIOS/BMC revision (R1.20.0/2.12.1) and another 5 with mismatched BMC firmware (2.13.0 vs the fleet standard 2.14.0). This identifies upgrade candidates that would be buried in manual per-node inspection.

Use Case 4: Mass Power Control via Redfish Actions

Scenario: Perform a graceful shutdown of a node group before scheduled datacenter maintenance, using the Redfish Reset action with a JSON payload.

Create graceful_shutdown.json:

{
    "ResetType": "GracefulShutdown"
}

Then execute:

redfish-clush cpu0[01-20] admin Sup3rP4ssword POST /redfish/v1/Systems/1/Actions/ComputerSystem.Reset graceful_shutdown.json

Expected Output:

---------------
cpu0[01-19] (19)
---------------
[HTTP 200]

---------------
cpu0[20] (1)
---------------
[HTTP 409]

Diagnostic Value: Node cpu0[20] returned HTTP 409 (Conflict) — the Redfish service rejected the shutdown because the system was already powered off or in a transitional state. The aggregation isolates this anomaly instead of burying it among 19 successes.

Use Case 5: Thermal & Health Telemetry Snapshot

Scenario: Collect real-time thermal readings and overall health status from all GPU nodes to detect pre-failure cooling conditions.

redfish-clush gpu0[01-44] admin Sup3rP4ssword GET /redfish/v1/Chassis/1/Thermal '.Temperatures[] | select(.ReadingCelsius > 60) | .Name + ": " + (.ReadingCelsius|tostring) + "C"'

Production Output:

---------------
gpu0[01-40],gpu0[42-44] (43)
---------------
"CPU0_TEMP: 52C"
"CPU1_TEMP: 54C"
"DIMM_A0: 48C"
[HTTP 200]

---------------
gpu0[41] (1)
---------------
"CPU0_TEMP: 78C"
"CPU1_TEMP: 81C"
"DIMM_A0: 72C"
"DIMM_B3: 71C"
[HTTP 200]

Critical Diagnostic Value: JQ's built-in select() filters only sensors above 60°C. Node gpu0[41] is immediately flagged with CPU temperatures approaching throttle thresholds and DIMM temps exceeding normal ranges — a failing fan module or obstructed airflow that demands urgent physical intervention.

Use Case 6: BIOS Configuration Discovery & Diff

Scenario: You manually configured a node to your desired BIOS settings via the web UI. Now you need to discover exactly which Redfish parameters changed (manufacturers often use cryptic names like Whitley0003 instead of human-readable labels) so you can replicate the configuration across the fleet.

Step 1 — Dump the configuration from both nodes:

redfish-clush golden01 admin Sup3rP4ssword GET /redfish/v1/Systems/1/Bios > golden.json
redfish-clush default01 admin Sup3rP4ssword GET /redfish/v1/Systems/1/Bios > default.json

Step 2 — Diff the dumps and generate a PATCH payload:

redfish-clush diff golden.json default.json --filter .Attributes --patch > bios_delta.json

Step 2 output (bios_delta.json):

{
  "BootMode": "UEFI",
  "MemoryTurbo": false,
  "Whitley0003": "SNC2",
  "Whitley0022": "Performance",
  "Whitley0065": "Enabled"
}

Step 3 — Deploy to the entire cluster:

redfish-clush cpu0[01-50] admin Sup3rP4ssword PATCH /redfish/v1/Systems/1/Bios/Settings bios_delta.json

Alternative — Direct node comparison (skip the file dumps):

redfish-clush diff admin Sup3rP4ssword /redfish/v1/Systems/1/Bios golden01 default01 --filter .Attributes --patch > bios_delta.json

Non-patch mode (see full breakdown):

Without --patch, the diff shows three categories of differences:

=== Only in golden01 (2) ===
{
  "FanMode": "Full",
  "Whitley0065": "Enabled"
}

=== Only in default01 (1) ===
{
  "PowerCap": 200
}

=== Different (4) ===
  BootMode:
    golden01: "UEFI"
    default01: "Legacy"
  MemoryTurbo:
    golden01: false
    default01: true
  Whitley0003:
    golden01: "SNC2"
    default01: "Disabled"
  Whitley0022:
    golden01: "Performance"
    default01: "PowerSave"

Critical Value: On platforms with opaque attribute naming (Whitley0003), manually mapping UI changes to Redfish parameters is impractical. This workflow discovers the exact PATCH payload via a single diff, then deploys it fleet-wide with the standard PATCH command — no guesswork, no manual JSON editing.

Error Handling & Debugging

When a node's BMC layer is unreachable, has invalid credentials, or returns an error, redfish-clush isolates it automatically under a separate summary block header with the raw curl error output:

---------------
cpu0[05] (1)
---------------
curl: (7) Failed to connect to cpu005-bmc port 443 after 15000 ms: Connection timed out
[HTTP 000]

---------------
cpu0[12] (1)
---------------
curl: (35) SSL connect error
[HTTP 000]

Failed nodes are never merged with successful responses, ensuring individual failures are immediately visible and actionable.

---

Author

SergioZ3R0 · @SergioZ3R0