Skip to content

opentelemetry: validate JSON container value types - #12131

Merged
edsiper merged 5 commits into
masterfrom
otlp-json-anyvalue-noissue
Jul 30, 2026
Merged

opentelemetry: validate JSON container value types#12131
edsiper merged 5 commits into
masterfrom
otlp-json-anyvalue-noissue

Conversation

@edsiper

@edsiper edsiper commented Jul 22, 2026

Copy link
Copy Markdown
Member

Summary

  • validate resolved OTLP/JSON arrayValue and kvlistValue containers before conversion
  • reject scalar values fields before any MessagePack array/map union access
  • add type guards at the map, array, and kvlist conversion boundaries
  • remove the unused input-plugin copy of the JSON wrapper decoder
  • add integration regressions for malformed arrays, malformed kvlists, and map-form kvlists

Root cause

flb_otel_utils_json_payload_get_wrapped_value() assigned the semantic AnyValue type from the wrapper key, then returned the nested values object without consistently validating its actual MessagePack type. The downstream array and kvlist conversion paths could therefore interpret a scalar object's union as a container.

Related-path audit

The same missing validation affected kvlistValue.values, so this PR fixes both container paths and adds defensive checks at all conversion entry points. The metrics and traces callers share the corrected Fluent Bit utility. The cmetrics and ctraces bundled decoders use different conversion paths and do not require changes.

Compatibility

Valid OTLP/JSON arrays and kvlists continue to decode as before. Invalid scalar container payloads are rejected without terminating the worker. Existing map-form kvlists remain supported.

Validation

  • cmake --build build -j8 — passed
  • ctest --test-dir build -R flb-it-opentelemetry --output-on-failure — passed
  • tests/integration/.venv/bin/python -m pytest tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py::test_in_opentelemetry_handles_json_logs_with_invalid_any_value_container -q — 3 passed
  • VALGRIND=1 VALGRIND_STRICT=1 tests/integration/.venv/bin/python -m pytest tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py::test_in_opentelemetry_handles_json_logs_with_invalid_any_value_container -q — 3 passed
  • GITHUB_EVENT_NAME=pull_request GITHUB_BASE_REF=master tests/integration/.venv/bin/python .github/scripts/commit_prefix_check.py — passed

Summary by CodeRabbit

  • Bug Fixes
    • Strengthened OpenTelemetry JSON parsing for wrapped “AnyValue” container shapes, including stricter nested map validation.
    • Improved compatibility checks for array vs map payload encodings and corrected when type validation occurs.
    • More consistent handling of shape extraction errors and normalized conversions for wrapped empty containers.
    • Added safer argument/type validation for JSON payload encoding paths.
  • Tests
    • Added parameterized integration coverage for OTLP JSON ingestion with valid/invalid “AnyValue” arrayValue/kvlistValue body shapes, asserting HTTP status and emitted output.

edsiper added 2 commits July 21, 2026 21:17
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 954a6684-3fe6-442d-912b-3b10456a496a

📥 Commits

Reviewing files that changed from the base of the PR and between ce1cdec and f16e3f8.

📒 Files selected for processing (4)
  • plugins/in_opentelemetry/opentelemetry_utils.c
  • plugins/in_opentelemetry/opentelemetry_utils.h
  • src/opentelemetry/flb_opentelemetry_utils.c
  • tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py
💤 Files with no reviewable changes (2)
  • plugins/in_opentelemetry/opentelemetry_utils.h
  • plugins/in_opentelemetry/opentelemetry_utils.c
🚧 Files skipped from review as they are similar to previous changes (2)
  • tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py
  • src/opentelemetry/flb_opentelemetry_utils.c

📝 Walkthrough

Walkthrough

OpenTelemetry JSON wrapped-value parsing now validates nested container shapes and dispatches arrays and maps by their actual msgpack types. Conversion helpers reject invalid arguments, empty maps can represent empty arrays, and integration coverage exercises malformed AnyValue containers.

Changes

OpenTelemetry JSON validation

Layer / File(s) Summary
Wrapped-value validation and dispatch
plugins/in_opentelemetry/opentelemetry_utils.c, plugins/in_opentelemetry/opentelemetry_utils.h, src/opentelemetry/flb_otel...
Nested wrapped values are structurally and type-checked before output assignment; array/map values are dispatched according to their msgpack type, empty maps can encode as empty arrays, and the obsolete helper is removed.
Container guards and invalid-payload coverage
src/opentelemetry/flb_opentelemetry_utils.c, tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py
Container converters reject invalid inputs and propagate extraction errors, while integration tests submit malformed and valid JSON log body containers and verify the resulting behavior.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested reviewers: cosmo0920

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: validating OpenTelemetry JSON container value types.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch otlp-json-anyvalue-noissue

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@edsiper edsiper changed the title opentelemetry: validate JSON array value type opentelemetry: validate JSON container value types Jul 22, 2026
@edsiper edsiper added this to the Fluent Bit v5.1 milestone Jul 22, 2026
@edsiper
edsiper marked this pull request as ready for review July 22, 2026 03:37
@edsiper
edsiper requested a review from cosmo0920 as a code owner July 22, 2026 03:37

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1253cb3949

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +184 to +186
if (internal_type == MSGPACK_OBJECT_ARRAY &&
kv_value->type != MSGPACK_OBJECT_ARRAY) {
return -2;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve omitted values as an empty array

When an OTLP/JSON client sends an empty ArrayValue as {"arrayValue": {}}, the protobuf JSON mapping omits the repeated values field, so this is a valid empty array. This new check returns -2 whenever the resolved array wrapper is a map rather than an array, so the log body path no longer unwraps it as [] and instead falls back to serializing the raw wrapper map; the same wrapper is also rejected by shared AnyValue callers. Please keep the empty-map wrapper case for arrayValue and only reject scalar/non-container values payloads.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/opentelemetry/flb_opentelemetry_utils.c`:
- Around line 184-191: Update the nested-wrapper handling around
flb_otel_utils_json_payload_append_unwrapped_value() and
flb_otel_utils_json_payload_append_converted_map() so malformed wrapper shapes
rejected by the internal_type checks propagate a distinct error instead of being
treated as ordinary maps. Preserve the separate “not wrapped” result for valid
non-wrapper maps, and add a regression covering an arrayValue containing an
invalid values field that must be rejected rather than emitted verbatim.
- Around line 178-180: Update the key validation around the values-wrapper check
to require both an exact length of 6 and a case-insensitive match against
“values”; return -3 for shorter, longer, or otherwise different keys while
preserving acceptance of the exact key.

In
`@tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py`:
- Around line 854-900: Update
test_in_opentelemetry_handles_json_logs_with_invalid_any_value_container to
distinguish scalar invalid containers from the map-form kvlist case. Assert that
arrayValue and scalar kvlistValue inputs are dropped or emit no records (or
verify the relevant drop metric), while map-form kvlistValue produces the
expected emitted structure; retain the existing status and process-liveness
checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 61d61ef7-760e-42ef-bc7b-3f1f4d4aa340

📥 Commits

Reviewing files that changed from the base of the PR and between c830fb6 and 1253cb3.

📒 Files selected for processing (4)
  • plugins/in_opentelemetry/opentelemetry_utils.c
  • plugins/in_opentelemetry/opentelemetry_utils.h
  • src/opentelemetry/flb_opentelemetry_utils.c
  • tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py
💤 Files with no reviewable changes (2)
  • plugins/in_opentelemetry/opentelemetry_utils.h
  • plugins/in_opentelemetry/opentelemetry_utils.c

Comment thread src/opentelemetry/flb_opentelemetry_utils.c Outdated
Comment thread src/opentelemetry/flb_opentelemetry_utils.c

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/opentelemetry/flb_opentelemetry_utils.c (1)

171-193: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use the required Allman brace layout in the new C blocks.

  • src/opentelemetry/flb_opentelemetry_utils.c#L171-L193: move new if/else if opening braces to their own lines.
  • src/opentelemetry/flb_opentelemetry_utils.c#L345-L378: apply the same layout to array/map dispatch.
  • src/opentelemetry/flb_opentelemetry_utils.c#L407-L410: apply it to the argument guard.
  • src/opentelemetry/flb_opentelemetry_utils.c#L464-L467: apply it to the array guard.
  • src/opentelemetry/flb_opentelemetry_utils.c#L508-L511: apply it to the kvlist guard.
  • src/opentelemetry/flb_opentelemetry_utils.c#L898-L917: apply it to CFL array conversion branches.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/opentelemetry/flb_opentelemetry_utils.c` around lines 171 - 193, Reformat
the new conditional blocks to use Allman braces, placing every if, else if, and
else opening brace on its own line. Apply this consistently in
src/opentelemetry/flb_opentelemetry_utils.c at ranges 171-193, 345-378, 407-410,
464-467, 508-511, and 898-917, without changing behavior.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/opentelemetry/flb_opentelemetry_utils.c`:
- Around line 171-193: Reformat the new conditional blocks to use Allman braces,
placing every if, else if, and else opening brace on its own line. Apply this
consistently in src/opentelemetry/flb_opentelemetry_utils.c at ranges 171-193,
345-378, 407-410, 464-467, 508-511, and 898-917, without changing behavior.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a0531f26-b59c-4cec-b0aa-e0704ddbc734

📥 Commits

Reviewing files that changed from the base of the PR and between 1253cb3 and ce1cdec.

📒 Files selected for processing (4)
  • plugins/in_opentelemetry/opentelemetry_utils.c
  • plugins/in_opentelemetry/opentelemetry_utils.h
  • src/opentelemetry/flb_opentelemetry_utils.c
  • tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py
💤 Files with no reviewable changes (2)
  • plugins/in_opentelemetry/opentelemetry_utils.c
  • plugins/in_opentelemetry/opentelemetry_utils.h
🚧 Files skipped from review as they are similar to previous changes (1)
  • tests/integration/scenarios/in_opentelemetry/tests/test_in_opentelemetry_001.py

Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
edsiper added 2 commits July 29, 2026 12:26
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
Signed-off-by: Eduardo Silva <eduardo@chronosphere.io>
@edsiper
edsiper force-pushed the otlp-json-anyvalue-noissue branch from ce1cdec to f16e3f8 Compare July 29, 2026 18:27
@edsiper
edsiper merged commit fb394a0 into master Jul 30, 2026
59 of 61 checks passed
@edsiper
edsiper deleted the otlp-json-anyvalue-noissue branch July 30, 2026 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant