Skip to content

request-tracking-api.adoc

Latest commit

 

History

History
402 lines (307 loc) · 13.6 KB

request-tracking-api.adoc

File metadata and controls

402 lines (307 loc) · 13.6 KB
Raw

Request Tracking API Guide

Table of Contents

1. For API Consumers

1.1. Submitting a Tracked Request

When a route has tracking-mode=simple, POST/PUT/PATCH requests return 202 Accepted with tracking information. For tracking-mode=attachments, see Attachments API.

Request
POST /api/orders HTTP/1.1
Authorization: Bearer <jwt-token>
Content-Type: application/json

{"item": "widget", "quantity": 5}
Response
HTTP/1.1 202 Accepted
Location: http://gateway:9443/status/550e8400-e29b-41d4-a716-446655440000
Content-Type: application/json

{
  "status": "accepted",
  "traceId": "550e8400-e29b-41d4-a716-446655440000",
  "_links": {
    "status": {
      "href": "/status/550e8400-e29b-41d4-a716-446655440000"
    }
  }
}
Note
 The Location header contains an absolute URI. The _links.status.href contains a relative URI (proxy-safe).

1.2. Polling for Status

GET /status/550e8400-e29b-41d4-a716-446655440000 HTTP/1.1

1.2.1. ACCEPTED

{
  "traceId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "ACCEPTED",
  "acceptedAt": "2026-03-13T10:00:00Z",
  "updatedAt": "2026-03-13T10:00:00Z"
}

1.2.2. COLLECTING_ATTACHMENTS

Only for routes with tracking-mode=attachments. The parent request is waiting for attachments to arrive (see Attachments API).

{
  "traceId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "COLLECTING_ATTACHMENTS",
  "acceptedAt": "2026-03-13T10:00:00Z",
  "updatedAt": "2026-03-13T10:00:00Z"
}

While in this status, POST /attachments/{traceId} uploads are accepted. Once the minimum attachment count (attachments-min-count) is reached, the gateway automatically transitions the status to PROCESSED. Attachments are still accepted up to attachments-max-count even after this transition. The attachment window is open only while the status is COLLECTING_ATTACHMENTS or PROCESSED; it closes when the status transitions to PROCESSING, REJECTED, ERROR, or any other status set by downstream flow logic (409 Conflict).

1.2.3. PROCESSING

{
  "traceId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "PROCESSING",
  "acceptedAt": "2026-03-13T10:00:00Z",
  "updatedAt": "2026-03-13T10:00:05Z"
}

1.2.4. PROCESSED

{
  "traceId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "PROCESSED",
  "acceptedAt": "2026-03-13T10:00:00Z",
  "updatedAt": "2026-03-13T10:00:10Z"
}

1.2.5. The error object

The error object is an RFC 9457 Problem Details object. It is emitted whenever any error* cache field yields a member, regardless of the entry’s status — REJECTED and ERROR are the common cases, but a PROCESSED entry carrying a warning detail returns the object just the same. It is absent when no error* field yields a member. A field that is present but malformed counts as absent for this purpose, so an entry whose only error* fields are malformed returns no error key rather than an empty object.

{
  "traceId": "550e8400-e29b-41d4-a716-446655440000",
  "status": "REJECTED",
  "acceptedAt": "2026-03-13T10:00:00Z",
  "updatedAt": "2026-03-13T10:00:10Z",
  "error": {
    "type": "https://example.com/problems/validation",
    "status": 422,
    "title": "Validation Failed",
    "detail": "The submitted order failed validation",
    "instance": "/status/550e8400-e29b-41d4-a716-446655440000",
    "violations": [
      {
        "pointer": "/item",
        "detail": "must not be blank"
      },
      {
        "pointer": "/quantity",
        "detail": "must be greater than 0"
      }
    ]
  }
}

Every member is optional and is emitted only when its producing cache field is populated. status is a JSON number and violations is a JSON array; the remaining members are strings. violations[].pointer values are RFC 6901 JSON Pointers into the submitted payload, passed through verbatim from the producer.

1.2.6. Unknown traceId (404)

Returns RFC 9457 Problem Detail:

{
  "type": "https://github.com/cuioss/nifi-extensions/blob/main/doc/reference/error-reference.adoc",
  "title": "Not Found",
  "status": 404,
  "detail": "No status found for traceId: 550e8400-e29b-41d4-a716-446655440000"
}

1.3. Chained Requests (X-Parent-Trace-Id)

To correlate related requests, include the X-Parent-Trace-Id header:

POST /api/payments HTTP/1.1
Authorization: Bearer <jwt-token>
Content-Type: application/json
X-Parent-Trace-Id: 550e8400-e29b-41d4-a716-446655440000

{"orderId": "ORD-123", "amount": 99.99}

The status response will include parentTraceId:

{
  "traceId": "660e8400-...",
  "status": "ACCEPTED",
  "parentTraceId": "550e8400-e29b-41d4-a716-446655440000",
  ...
}

1.4. Authentication

The /status endpoint follows the same auth configuration as other management endpoints. Default: local-only,bearer (loopback bypass or JWT required).

2. For NiFi Flow Designers

2.1. FlowFile Attributes

When request tracking is enabled, FlowFiles carry these additional attributes:

Attribute Description

rest.trace.id

The unique trace ID (UUID) for this request

rest.trace.parent.id

The parent trace ID (if X-Parent-Trace-Id header was provided)

rest.trace.accepted.at

ISO 8601 timestamp when the request was accepted (e.g., 2026-03-13T10:00:00Z)

2.2. Updating Status via PutDistributedMapCache

Downstream flows update the request status using NiFi’s standard PutDistributedMapCache processor.

2.2.1. Configuration

Property Value

Cache Entry Identifier

${rest.trace.id}

Cache Entry Value

JSON status string (see below)

Distributed Cache Service

Same DistributedMapCacheClient configured on the gateway

2.2.2. Status Update JSON Format

Update to PROCESSING at the start of your flow:

{"traceId":"${rest.trace.id}","status":"PROCESSING","acceptedAt":"${rest.trace.accepted.at}","updatedAt":"${now():format('yyyy-MM-dd''T''HH:mm:ss''Z''','UTC')}"}
Note
 The simplest approach is to use UpdateAttribute to build the JSON, then PutDistributedMapCache.

2.2.3. Example NiFi Flow

[RestApiGateway] --> [UpdateAttribute: set status=PROCESSING]
                --> [PutDistributedMapCache: update status]
                --> [Your Processing Logic]
                --> [UpdateAttribute: set status=PROCESSED or REJECTED]
                --> [PutDistributedMapCache: update final status]

2.2.4. Error Handling

Six optional cache fields populate the response’s error object. Writing any one of them makes the gateway emit error — no particular status value is required.

{
  "traceId": "${rest.trace.id}",
  "status": "REJECTED",
  "acceptedAt": "...",
  "updatedAt": "...",
  "errorType": "https://example.com/problems/validation",
  "errorStatus": "422",
  "errorTitle": "Validation Failed",
  "errorDetail": "Validation failed: ${error.message}",
  "errorInstance": "/status/${rest.trace.id}",
  "errorViolations": "[{\"pointer\":\"/item\",\"detail\":\"must not be blank\"}]"
}
Cache field Response member Notes

errorType

error.type (string)

URI reference identifying the problem type.

errorStatus

error.status (number)

Written as a String holding an integer; the gateway parses it into a JSON number. The value must fall within the RFC 9457 §3.1.2 range 100-599; a value outside it is treated exactly like a non-integer — the member is omitted and REST-125 is logged.

errorTitle

error.title (string)

Short, human-readable summary.

errorDetail

error.detail (string)

Explanation specific to this occurrence.

errorInstance

error.instance (string)

URI reference identifying this occurrence.

errorViolations

error.violations (array)

Written as a String holding a serialized JSON array; the gateway parses it into a real JSON array. Entries conventionally carry pointer (RFC 6901) and detail. Pointer conformance is the producer’s responsibility — the gateway does not validate it.

All six are written as JSON string scalars in the cache entry; only errorStatus and errorViolations are re-typed on the way out. A blank value counts as absent.

Important
 A malformed errorStatus (not an integer, or outside the 100-599 range) or errorViolations (not a serialized JSON array) omits only that one response member, without failing the response. Any sibling members that are well-formed are still returned, the request never fails, and the gateway records a WARN in the NiFi log — REST-125 for errorStatus, REST-126 for errorViolations. A malformed field counts as absent rather than as a populated component, so when it is the only error* field present the error key is omitted entirely instead of being returned empty. Check the NiFi log when a member you expected does not appear in the response.
Note
 These six keys are reserved. Unlike free-form top-level cache keys they are no longer echoed back as additional response fields — they feed the error object instead.

2.3. Best Practices

  1. Update to PROCESSING early: immediately after the gateway emits the FlowFile

  2. Update to PROCESSED/REJECTED at the end: after all processing is complete

  3. Use RETRY sparingly: only for genuinely transient errors that will be retried

  4. Populate the error fields meaningfully: errorTitle and errorDetail at minimum; add errorType, errorStatus, errorInstance and per-field errorViolations when the failure is structured, so consumers can act on it instead of parsing prose

3. Configuration Reference

3.1. Processor Properties

Property Default Description

Distributed Map Cache Client

(none)

Required for request tracking. Points to a DistributedMapCacheClient controller service.

Status Endpoint Enabled

true

Whether the /status/{traceId} endpoint is active

Status Endpoint Auth Mode

local-only,bearer

Authentication for the status endpoint

Status Endpoint Required Roles

(empty)

JWT roles required for the status endpoint

Status Endpoint Required Scopes

(empty)

JWT scopes required for the status endpoint

3.2. Route Properties

Route tracking is configured via restapi.<name>.tracking-mode (see Configuration Reference). For tracking-mode=attachments, see Attachments API.

3.3. DistributedMapCacheServer Setup

A DistributedMapCacheServer controller service must be running in NiFi for the cache client to connect to.

  1. Add DistributedMapCacheServer controller service

  2. Configure port (default: 4557)

  3. Enable the service

  4. Add DistributedMapCacheClientService controller service

  5. Configure server hostname and port

  6. Enable the service

  7. Reference it in the gateway processor’s "Distributed Map Cache Client" property

3.4. Cache Sizing and Eviction

The gateway writes one cache entry per tracked request and sets no expiry on it. Entries are removed explicitly only when a request never reaches a terminal state (for example, a queue-full 503, or in-flight containers discarded on processor shutdown). Entries for requests that complete normally — PROCESSED, REJECTED, ERROR — are never removed by the gateway.

The entry count therefore grows with total request volume, and the only bound is the eviction policy of the DistributedMapCacheServer. Sizing it is an operational decision, not a detail that can be left at its defaults.

3.4.1. Recommended Configuration

Configure these properties on the DistributedMapCacheServer controller service:

Property Recommended Rationale

Eviction Strategy

Least Recently Used

The default is Least Frequently Used, which is the wrong fit for this workload. A status entry is polled a handful of times while the request is fresh and then never again, so LFU retains long-finished entries that happened to be polled often and evicts recent entries that have only been polled once. Least Recently Used matches the actual access pattern: keep what is still being polled, discard what nobody has asked about.

Maximum Cache Entries

Peak accept rate x longest polling window, plus headroom

The default is 10000. Size it to the number of requests accepted during the longest window in which a consumer may still poll for status. At a sustained 50 requests/second with consumers polling for up to 10 minutes after acceptance, 50 x 600 = 30000 entries are live — three times the default, so the default would evict entries that are still being polled.

3.4.2. Consequence of Eviction

Eviction is silent and irreversible. Once an entry is evicted, GET /status/{traceId} cannot distinguish that trace ID from one that never existed: both return the same RFC 9457 404 Problem Detail documented under "Unknown traceId (404)" above.

{
  "type": "https://github.com/cuioss/nifi-extensions/blob/main/doc/reference/error-reference.adoc",
  "title": "Not Found",
  "status": 404,
  "detail": "No status found for traceId: 550e8400-e29b-41d4-a716-446655440000"
}
Important
 A 404 therefore does not mean the request was never accepted — it may mean the request completed and its entry has since been evicted. Consumers must not treat 404 as proof that a submission failed. Size the cache so that eviction cannot occur inside the window in which consumers are expected to poll.
Note
 The same 404 appears after a NiFi restart when the DistributedMapCacheServer has no "Persistence Directory" configured, because the cache is then held in memory only and starts empty.