Skip to content

chore(antithesis): banded-Frechet differential visual-equivalence oracle - #2190

Closed
blt wants to merge 1 commit into
blt/chore_antithesis_dsd_is_malformedfrom
blt/frechet_diffeq_oracle
Closed

chore(antithesis): banded-Frechet differential visual-equivalence oracle#2190
blt wants to merge 1 commit into
blt/chore_antithesis_dsd_is_malformedfrom
blt/frechet_diffeq_oracle

Conversation

@blt

@blt blt commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Summary

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

How did you test this PR?

References

@blt
chore(antithesis): banded-Frechet differential visual-equivalence oracle
0fd3977 
The count oracle was retired as unfixable. Its replacement compares the
two lanes per-context aggregation CURVES against a human-visual "same
line, small time-slide, nothing else" test, so an operator reading two
dashboards sees what the assertion sees.

This commit builds the one oracle end to end: value-bearing per-
(context, wire bucket-start) capture with jitter-safe keying and
conflict-not-overwrite; a per-kind Dallachiesa hard-interval ground
distance with bands derived from the ddsketch public API and sketches
compared by sup-over-quantiles, not the dilution-prone 1-Wasserstein; a
banded discrete-Frechet sup aligner with the J1 matched-jump envelope
guard; and a decision layer whose clipped signed CUSUM subsumes the
debounce and catches sustained sub-band bias, with per-quantile and
count channels and per-lane absolute-law detectors, all unioned into
one defect_count behind a single always-assertion. The v2 and sketch
paths no longer route through stele, so both lanes decode the same way.
The old contexts assertion bins are gone: this is the only differential
check. Constants are provisional pending calibration.
@dd-octo-sts dd-octo-sts Bot added the area/test All things testing: unit/integration, correctness, SMP regression, etc. label Jul 22, 2026
@blt blt mentioned this pull request Jul 22, 2026
Closed
4 tasks
@blt Graphite App

blt commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@blt blt mentioned this pull request Jul 22, 2026
Closed
4 tasks
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown

Pipelines

✨ Unblock PR with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

DataDog/saluki | check-fmt   View in Datadog   GitLab

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0fd3977 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 22, 2026

Copy link
Copy Markdown

Binary Size Analysis (Agent Data Plane)

Baseline: e4cba81 · Comparison: 0fd3977 · diff
Analysis Configuration: stripped binaries · Pass/Fail Threshold: +5%
Sizes: 40.86 MiB (baseline) vs 40.86 MiB (comparison)
Size Change: +0 B (+0.00%)

✅ Binary size difference within threshold

Changes by Module
Module File Size Symbols
anon.44a0faa3240fa409baa379409d152aec.1.llvm.10687876723069053719 +130 B 1
anon.44a0faa3240fa409baa379409d152aec.1.llvm.18163430659783575214 -130 B 1
anon.44a0faa3240fa409baa379409d152aec.4.llvm.10687876723069053719 +114 B 1
anon.44a0faa3240fa409baa379409d152aec.4.llvm.18163430659783575214 -114 B 1
anon.44a0faa3240fa409baa379409d152aec.3.llvm.10687876723069053719 +109 B 1
anon.44a0faa3240fa409baa379409d152aec.3.llvm.18163430659783575214 -109 B 1
anon.44a0faa3240fa409baa379409d152aec.0.llvm.10687876723069053719 +97 B 1
anon.44a0faa3240fa409baa379409d152aec.0.llvm.18163430659783575214 -97 B 1
anon.44a0faa3240fa409baa379409d152aec.2.llvm.10687876723069053719 +95 B 1
anon.44a0faa3240fa409baa379409d152aec.2.llvm.18163430659783575214 -95 B 1
Detailed Symbol Changes 
    FILE SIZE        VM SIZE    
 --------------  -------------- 
  [NEW]    +130  [NEW]     +40    anon.44a0faa3240fa409baa379409d152aec.1.llvm.10687876723069053719
  [NEW]    +114  [NEW]     +24    anon.44a0faa3240fa409baa379409d152aec.4.llvm.10687876723069053719
  [NEW]    +109  [NEW]     +19    anon.44a0faa3240fa409baa379409d152aec.3.llvm.10687876723069053719
  [NEW]     +97  [NEW]      +7    anon.44a0faa3240fa409baa379409d152aec.0.llvm.10687876723069053719
  [NEW]     +95  [NEW]      +5    anon.44a0faa3240fa409baa379409d152aec.2.llvm.10687876723069053719
  [DEL]     -95  [DEL]      -5    anon.44a0faa3240fa409baa379409d152aec.2.llvm.18163430659783575214
  [DEL]     -97  [DEL]      -7    anon.44a0faa3240fa409baa379409d152aec.0.llvm.18163430659783575214
  [DEL]    -109  [DEL]     -19    anon.44a0faa3240fa409baa379409d152aec.3.llvm.18163430659783575214
  [DEL]    -114  [DEL]     -24    anon.44a0faa3240fa409baa379409d152aec.4.llvm.18163430659783575214
  [DEL]    -130  [DEL]     -40    anon.44a0faa3240fa409baa379409d152aec.1.llvm.18163430659783575214
  [ = ]       0  [ = ]       0    TOTAL

@pr-commenter

pr-commenter Bot commented Jul 22, 2026

Copy link
Copy Markdown

Regression Detector (Agent Data Plane)

Run ID: 1c9f6bf8-894e-4040-b5f4-67be58893acf
Baseline: e4cba812 · Comparison: 0fd39774 · diff

Optimization Goals: ✅ No significant changes detected

Fine details of change detection per experiment (3)

Experiments configured erratic: true are tagged (ignored) and skipped when determining which experiments regressed or improved. Experiments which are detected as erratic at runtime are tagged (erratic) to flag that the run's sample dispersion was high, but their regression / improvement signal still counts.

experiment goal Δ mean % links
quality_gates_rss_dsd_medium memory ⚪ +0.61 metrics profiles logs
quality_gates_rss_dsd_low memory ⚪ +0.37 metrics profiles logs
quality_gates_rss_idle memory ⚪ -0.22 metrics profiles logs
Bounds Checks: ✅ Passed (3)
experiment check replicates observed links
quality_gates_rss_dsd_low memory_usage 10/10 ✅ 45.3 MiB ≤ 50 MiB metrics profiles logs
quality_gates_rss_dsd_medium memory_usage 10/10 ✅ 67.4 MiB ≤ 75 MiB metrics profiles logs
quality_gates_rss_idle memory_usage 10/10 ✅ 31 MiB ≤ 40 MiB metrics profiles logs
Explanation

A change is flagged as a regression when |Δ mean %| > 5.00% in the regressing direction for its optimization goal AND SMP marks the experiment as a regression (is_regression: true). Improvements use the matching criteria for the improving direction. Experiments configured erratic: true (tagged (ignored)) are skipped outright; experiments detected as erratic at runtime (tagged (erratic)) still count, since that flag describes sample dispersion rather than directional certainty. The Δ mean % cell is colored accordingly: 🟢 = improvement, 🔴 = regression, ⚪ = neutral. Reduction in CPU or memory is an improvement; reduction in ingress throughput is a regression.

@blt blt closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/test All things testing: unit/integration, correctness, SMP regression, etc.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@blt