[RankFO Integration] Port RankF_O ordinal heuristics + OBO baseline into iDFlakies minimize pipeline#72
Open
Md-Arif-Hasan wants to merge 6 commits into
Open
Conversation
…fied scoring algorithm
…plementation Implements DetectionResultsLoader.load() which reads test-run result JSON files from .dtfixingtools/test-runs/results/ in chronological order and converts them to List<TestOrderRecord> for consumption by RankFOScorer. Key adaptations from the brief: actual iDFlakies data lives in test-runs/results/ (timestamp-prefixed UUID files) rather than detection-results/round-*.json; the JSON field names testOrder and results.result match exactly. Adds Level 3 smoke test to RankFOScorerTest that validates parsing against real .dtfixingtools output from http-request/lib/. All 18 ranking tests pass (0 failures, 0 skipped).
…Reorderer - Add RankFOCandidateReorderer: loads detection results, scores candidates via RankFOScorer, and stable-sorts the prefix so the real polluter lands in the first half of the delta-debugger's split. - Modify TestMinimizer constructor to conditionally reorder when -Ddt.rankfo.enable=true; heuristic selectable via -Ddt.rankfo.heuristic (default: DISTANCE_TO_VICTIM). Falls through silently on missing data. - Add RankFOCandidateReordererTest covering reorder, BSS, unscored tests, empty orderings, and bad heuristic fallthrough (6 tests, all green).
…ics and score cache - Rename HeuristicType enum values to match paper notation exactly: PLUS_ONE, METHODS, DISTANCE, COMBINED_PLUS_ONE_DISTANCE, COMBINED_METHODS_DISTANCE - Rename heuristic classes accordingly: PlusOneHeuristic, MethodsHeuristic, DistanceHeuristic - Implement Combined heuristics: base heuristic for primary score, Distance tiebreaker from last valid ordering (per paper Figure 4) - Add RankFOScoreCache: filesystem-backed cache keyed by victim/heuristic/OdType with mtime-based invalidation against detection results directory - Update RankFOCandidateReorderer default heuristic to DISTANCE - All 39 unit tests pass
Adds -Ddt.minimizer.obo=true flag to TestMinimizer. When enabled, replaces delta-debugging with a linear scan: try each candidate individually before the victim, stop at the first that alone reproduces the expected result. Composes with RankFO ranking: when both flags are active, RankFO reorders candidates by polluter score and OBO confirms the top-ranked one first. This reproduces the paper's rank-then-linear-confirm strategy. Usage: mvn idflakies:minimize -Ddt.minimizer.obo=true # OBO baseline mvn idflakies:minimize -Ddt.rankfo.enable=true -Ddt.minimizer.obo=true # RankFO + OBO All 40 unit tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of this PR
This PR integrates the RankF_O ordinal ranking approach (Phase 2) into the iDFlakies minimization pipeline (Phase 3) and adds an OBO (One-by-One) linear-scan baseline for comparison.
What Changed
New package:
idflakies-core/.../minimizer/ranking/RankFOScorer: Ports all five RankF_O heuristics from Python to Java:DetectionResultsLoader: Loads detection results from.dtfixingtools/test-runs/results/JSON files.RankFOScoreCache: Provides disk-based score caching with modification-time (mtime) invalidation.RankFOCandidateReorderer: Public entry point used byTestMinimizerto reorder candidate tests based on RankFO scores.Modified:
TestMinimizer.javaTwo optional features were added. Existing minimization logic remains unchanged.
-Ddt.rankfo.enable=true-Ddt.minimizer.obo=trueValidation
Validated on kevinsawicki/http-request (M19 from the RankF_O paper):
customConnectionFactoryas the polluter.Test Coverage
Usage