Conversation
#2671) OCR v1.7.9 emits category and severity fields per finding, but the parser does not validate them at generation time, and production data confirms undocumented values occur (category 'correctness'/'other', severity 'info'). Until the upstream parser fix ships and we upgrade, the posting path must handle malformed/unknown metadata defensively rather than ignoring category/severity entirely. Adds a fail-open validation step between OCR result parsing and comment posting: - KNOWN_CATEGORIES (bug, maintainability, test, security, style) and KNOWN_SEVERITIES (high, medium, low) Sets defined in the header region. - validateFindingMetadata maps undocumented/missing values to 'unknown' with a best-effort redacted stderr + core.warning schema-drift message. Diagnostic emission is wrapped in try/catch so a redaction or I/O failure never suppresses findings (fail-open contract). - Raw drifted values are preserved in originalCategory/originalSeverity so downstream telemetry can distinguish intentional 'unknown' from schema drift. - Validation runs before deduplication so drift on a finding sharing a dedup key is still observed. - Arrays (typeof 'object') are passed through untouched. Adds 27 behavioral tests (scripts/tests/ocr-review-metadata-validation.test.js) that extract the real function from the committed workflow YAML and run it in an isolated vm, covering all six acceptance criteria.
Contributor
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 7 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
LLxprt PR Review – PR #2694Issue AlignmentThe PR implements the exact defensive layer described in #2671: it adds Side Effects
Code Quality
Tests and CoverageCoverage impact: increase
VerdictReady |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
OpenCodeReview — automatic reviews suspendedAutomatic OCR reviews are suspended for this PR after 2 of 2 automatic reviews. To get more reviews you can:
OpenCodeReview — PR #2694
|
Contributor
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-24.x-ubuntu-latest' artifact from the main CI run. |
Replace exact string equality (.toBe) on the location-omitted warning with substring checks (.toContain + negative location regex). The behavioral aspects (prefix present, metadata present, no location suffix) are still verified, but the test no longer breaks on harmless warning-wording changes.
github-actions
Bot
added
the
maintainer:e2e:ok
…t VM trust - Guard originalCategory/originalSeverity so an existing field from a prior processing stage is not overwritten (only set when absent). - Add a security note documenting that vm.runInNewContext executes only trusted, version-controlled workflow source (not user/PR input).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a local fail-open metadata validation step between OCR result parsing and comment posting, so the posting path no longer ignores category/severity fields entirely. Until the upstream parser fix (alibaba/open-code-review#477) lands and we upgrade, this is the local defensive layer for undocumented/malformed metadata.
Problem (from #2671)
OCR v1.7.9 emits category and severity fields per finding, but the parser does not validate them at generation time. Production data confirms undocumented values occur:
Currently the posting path treats every finding as an inline candidate regardless of metadata. This PR adds validation while never suppressing a finding because its metadata is malformed.
Changes
.github/workflows/ocr-review.yml (the code-review job Post OCR results step)
scripts/tests/ocr-review-metadata-validation.test.js (new, 27 tests)
Behavioral tests that extract the real validateFindingMetadata from the committed workflow YAML and run it in an isolated vm sandbox. No mock theater - the real function under test runs against real inputs. Covers all six acceptance criteria plus: valid metadata passthrough, undocumented category/severity mapping, missing fields as unknown, schema-drift warnings mirrored to process.stderr through redactSecretDiagnostics, structured downstream fields, non-object/null/array passthrough, immutability, fail-open contract (core.warning throwing does not suppress findings), and validation-before-dedup ordering (behavioral vm test plus structural wiring guard).
Acceptance criteria
Test plan
Relationship