Validate JSONPath reference paths (.$) in Step Functions state machine definitions

[ChrisJr404]

Issue #, if available: Fixes #4085

Description of changes:

Adds validation so that E3601 catches .$ reference-path fields whose value is not a valid JSONPath or intrinsic function call, before Step Functions rejects the template at deploy time.

In the Amazon States Language, inside a payload template (Parameters, ResultSelector, ItemSelector, and Assign) a field whose name ends with .$ is evaluated as a JSONPath or an intrinsic function rather than as a literal. Using a literal there fails at deploy time with, e.g.:

The value for the field 'Variable.$' must be a valid JSONPath or a valid intrinsic function call at /States/Pass/Assign

cfn-lint didn't flag this. This change teaches StateMachineDefinition (E3601) to walk payload templates recursively and report any .$ field whose value is a plain string that neither begins with $ (JSONPath, including $$ context references) nor looks like an intrinsic function call (States.*( ... )).

To avoid false positives it only inspects string values: non-string values (a resolved intrinsic function object, for example) and strings containing a ${...} substitution placeholder are left alone, since those can resolve to a valid path at deploy time. Nested objects and arrays inside a payload template are treated as payload templates too, matching the ASL semantics.

Testing:

• Added unit cases to test/unit/rules/resources/stepfunctions/test_state_machine_definition.py covering a valid mix of JSONPath / context / intrinsic / literal values (no error), the literal Assign case from the issue, and an invalid path nested inside Parameters.
• pytest test/unit/rules/resources/stepfunctions/ passes (23 tests).
• Ran cfn-lint over every fixture template under test/fixtures/templates that declares an AWS::StepFunctions::StateMachine; none produce a new E3601, so existing snapshots are unchanged.
• ruff check, ruff format --check, and mypy are clean on the changed files.

[kddejong]

The check also fires in JSONata mode. _validate_reference_paths runs inside _validate_step, which executes for every schema variant from _clean_schema, including JSONata. The .$ reference-path convention is JSONPath-only — under QueryLanguage: JSONata a literal .$ field gets flagged with a misleading JSONPath error:

Assign: { "foo.$": "plainliteral" }   # QueryLanguage: JSONata → E3601

Gate this to JSONPath/unset query language and add a JSONata case asserting no E3601. Otherwise looks good — inline notes below.

[kddejong]

This runs for both the JSONPath and JSONata schema variants. In JSONata mode the .$ convention isn't valid ASL, so literal .$ fields get flagged with a JSONPath-oriented message. Gate the call so it only runs when the resolved QueryLanguage is JSONPath (or unset).

[kddejong]

Minor: startswith("States.") and endswith(")") will accept malformed strings like States.Foo bar). Fine to keep (errs toward false negatives rather than false positives), but a short comment noting the intentional looseness would help future readers.

[ChrisJr404]

Good catch, thanks. I threaded the resolved query language out of _clean_schema (it already knows the JSONPath vs JSONata variant) and gated _validate_reference_paths so it only runs for the JSONPath/unset variant. Under QueryLanguage: JSONata the reference-path walk is now skipped, so a literal .$ field name no longer trips E3601.

Added a JSONata test case asserting no E3601 on a literal foo.$ in Assign; the existing JSONPath positive/negative cases are unchanged. Also dropped a short comment on the intentionally loose States.…) intrinsic check per your other note.

One thing worth flagging: the gate keys off the top-level QueryLanguage, matching how _clean_schema already selects the schema variant. A per-state query-language override isn't handled here (it isn't in the schema selection either), so that's the same scope as before rather than a new gap.

[kddejong]

Thanks for the follow-ups — the JSONata gating and the intentional-looseness comment both look good, and I confirmed the core behavior: the #4085 literal .$ case is flagged, and !Sub/!If intrinsics, ${...} substitutions, $$ context refs, and States.Format(...) are correctly left alone. 24 tests pass and existing SFN fixtures are unchanged.

One correctness issue before this can merge: a false positive.

_validate_reference_paths decides in_payload by matching the key name anywhere in the tree (key in _payload_template_fields). But that name isn't necessarily an ASL payload template — it can be a user-chosen state name, or nested literal data. A Pass Result is literal JSON (not a payload template), so a .$ key there is a literal and must not be flagged.

Repro:

Resources:
  SM:
    Type: AWS::StepFunctions::StateMachine
    Properties:
      RoleArn: arn:aws:iam::123456789012:role/r
      Definition:
        StartAt: Parameters
        States:
          Parameters:            # a state that happens to be named "Parameters"
            Type: Pass
            Result:              # Pass Result is literal, NOT a payload template
              "literalKey.$": "literal data"
            End: true

E3601 'literal data' is not a valid JSONPath string or intrinsic function for 'literalKey.$' at /States/Parameters/Result/literalKey.$

The same misfires for states named Assign/ResultSelector/ItemSelector. Since we treat false positives as worse than false negatives, this should be fixed before merge.

Suggested direction: scope payload detection structurally — only enter payload mode when Parameters/ResultSelector/ItemSelector/Assign appears as a direct field of a state (i.e. walk states and start the payload recursion from those specific fields), rather than matching the key name at any depth. That keeps Pass Result and state names from triggering it. A regression test with a state named Parameters (and a literal .$ under Result) asserting no E3601 would lock it in.

[ChrisJr404]

Good catch on the false positive. I reworked the payload detection to be structural instead of name-based: it now walks the States map and only starts payload-mode recursion from the payload template fields (Parameters/ResultSelector/ItemSelector/Assign) that appear as a direct field of a state, recursing into Parallel branches and Map ItemProcessor/Iterator so nested states keep their coverage. That means a Pass Result stays literal and a state that merely happens to be named Parameters (or Assign/ResultSelector/ItemSelector) no longer trips E3601.

Added a regression test for your repro: a state named Parameters whose Pass Result contains a literal "literalKey.$", asserting no E3601. Existing JSONPath positive/negative and JSONata cases are unchanged. 25 tests pass; ruff and isort are clean.

[kddejong]

The structural scoping fix looks good — my earlier false positive (state named after a payload field) is resolved, and nested Map/Parallel payloads are validated correctly.

One remaining false positive: the JSONata gate is only evaluated at the top level. _clean_schema reads QueryLanguage from the definition root, but QueryLanguage can be set per-state — a JSONPath (or unset) machine can have an individual JSONata state for incremental adoption (docs: "If the state machine level QueryLanguage is set to JSONPath, then any individual state-level QueryLanguage can be set to either JSONPath or JSONata"). Such a state still gets the .$ check:

Definition:
  StartAt: J
  States:
    J:
      Type: Pass
      QueryLanguage: JSONata      # valid per-state override
      Assign:
        foo.$: "plainliteral"     # legal in JSONata → wrongly flagged E3601
      End: true

This is the same class you flagged for the top-level JSONata case — just at state granularity.

Suggested fix in _validate_reference_paths: skip a state whose own QueryLanguage is JSONata, while still recursing into nested state machines (each nested state resolves its own):

            state_path = deque(path)
            state_path.extend(["States", state_name])

            # A state may override QueryLanguage to JSONata even when the
            # machine default is JSONPath (incremental adoption). The ".$"
            # convention is JSONPath-only, so skip its payload fields here.
            if state.get("QueryLanguage") != "JSONata":
                for field in self._payload_template_fields:
                    if field in state:
                        field_path = deque(state_path)
                        field_path.append(field)
                        yield from self._validate_payload(state[field], k, field_path)

I tried this locally: it clears the FP, keeps the issue repro and the nested-Map cases flagged, and all 25 tests still pass. Worth a regression test for a JSONPath machine + JSONata state asserting no E3601.

[ChrisJr404]

Good catch on the per-state case. _validate_reference_paths now resolves QueryLanguage per state instead of relying only on the top-level value: it carries an inherited QueryLanguage down through nested state machines (Map ItemProcessor/Iterator, Parallel Branches), and a state whose resolved QueryLanguage is JSONata (whether set on the state or inherited) is skipped, so a literal .$ field in a JSONata state inside a JSONPath machine is no longer flagged. A state can also opt back into JSONPath and still be validated. Added a test with a JSONPath machine that has one per-state JSONata state asserting no E3601. All 26 rule tests pass, ruff and isort are clean.