Aplomb Benchmark is a catalogue of 9 job-shop scheduling instances (DRC-FJSP-SDST — flexible job-shop, dual resource-constrained by machine and qualified operator, sequence-dependent setup times) where every published instance carries a proven optimum, not a best-known record. It targets researchers and practitioners in flexible job-shop scheduling, dual-resource-constrained scheduling, or setup-time-dependent scheduling who need an absolute gap measurement rather than a moving comparison point. Released under CC BY 4.0, in the spirit of the Taillard and Brandimarte reference sets.
v1.0 catalogue — 9 instances with a proven optimum, CC BY 4.0. This dataset is part of Aplomb, a scheduling test bench for machining / precision-turning shops. A companion working paper describing the method is being prepared for submission (see §11). See Version notes for the dataset's release history. (Version française : README.fr.md.)
How to cite. If you use this dataset, please cite it as below (full details, including the companion working paper once it is deposited, in §11):
@misc{aplomb_benchmark_2026,
author = {{Aplomb project}},
title = {{Aplomb Benchmark: DRC-FJSP-SDST}},
year = {2026},
version = {1.0},
publisher = {GitHub},
url = {https://github.com/charlesnauroy-wq/aplomb-benchmark},
note = {Dataset, CC BY 4.0}
}Why use this benchmark
- Proven optimum, not best-known. Every published instance's ground truth is established independently of any candidate scheduler — by offline exact resolution or restricted reverse construction — and is checked, not just declared, to be independent of the evaluated engine (a code-enforced separation between the ground-truth pipeline and the scheduler under test). This supports an absolute gap measurement, rather than a comparison to a record that can still move.
- Every guarantee maps to a named, automated test. The 18 guarantees stated in this README (instance validity, reproducibility, ground-truth typing, engine independence, etc.) are each backed by a specific test in the source repository — verifiable, not just asserted.
- Bit-for-bit reproducible, openly licensed. Regenerating the catalogue from its seed and versioned parameters reproduces every instance's deterministic core exactly; the dataset is released under CC BY 4.0, permitting reuse, republication, and citation.
If you use this benchmark in your research — or are considering it — please open an issue on this repository to let us know; it helps us understand who relies on it and what to prioritize next.
This repository is a catalogue of job-shop scheduling instances, in the spirit of the reference sets published by Taillard or Brandimarte for classical job-shop scheduling. Every instance published here has an optimal solution that is known and proven independently — no bracket, no approximation: it lets you measure a scheduler's absolute gap to the optimum, not to a "best known so far" record that keeps moving. See §2 for exactly what this scope covers.
Every instance in this catalogue belongs to the same problem class, denoted DRC-FJSP-SDST:
- FJSP — Flexible Job-Shop Scheduling Problem. A job-shop that is flexible: an operation is not assigned to a single machine, it can be executed by any machine in an interchangeable family (an assignment choice, on top of the sequencing choice).
- DRC — Dual-Resource Constrained. An operation does not just require an eligible machine, it requires simultaneously and independently an operator holding the named competency it demands. The two conditions (machine family, competency) are independent of each other: a free machine with no qualified operator cannot start the operation, and vice versa.
- SDST — Sequence-Dependent Setup Times. The setup time between two consecutive operations on the same resource depends on the ordered pair (previous configuration attribute → next attribute), read from a setup matrix — not a constant, not symmetric.
An instance that only carries part of this class (for example a single possible machine per operation, or a setup that is zero regardless of the attribute pair) does not count as a DRC-FJSP-SDST instance under this catalogue: that is an acceptance criterion checked on every instance actually produced, not just declared (see the guarantees table, §9).
This repository carries 9 instances, every one of them with a ground truth that is a proven optimum (ground_truth_kind == "optimum_prouve") — no bracket, no approximation ships here. This is a scoping decision made deliberately, not a limitation we're hiding:
- This catalogue's differentiator is proof, not volume. The one promise it can make without a caveat is "proven-optimum instances → absolute gap measurement." Aplomb's source project also produces bracket-only instances (a wider, unproven range) as part of a larger internal grid — but mixing proven and merely-bracketed ground truth under the same public claim would dilute and contradict that promise, so this release does not ship them. Quality over quantity: even the Taillard sets were built as a curated set, not a large one.
- The 9 instances' tier and difficulty breakdown, and the two strategies behind them, are given in the table in §6 (strategies explained in full in §5).
published-manifest.jsonlists exactly these 9 fingerprints. In Aplomb's source project it is derived from a largergroundtruth-manifest.jsonby a pure filter onground_truth_kind— never a second, hand-edited list. In this repository the two files carry identical rows, since every ground truth shipped here already passes that filter.- A wider grid, including bracket-only instances at other size tiers, exists in Aplomb's private source project and is deferred to a future release, pending a scheduling engine able to supply a tighter upper bound than the current engine-independent heuristic.
.
├── README.md — this file (English)
├── README.fr.md — French version
├── LICENSE — dataset license (CC BY 4.0)
├── CITATION.cff — machine-readable citation metadata
├── VERSION-NOTES.md — release notes
├── manifest.json — catalogue index: one row per instance (9)
├── groundtruth-manifest.json — ground-truth index (9): type, origin, fingerprint
├── published-manifest.json — published scope (9): proven optimum only, same rows as groundtruth-manifest.json in this release
├── groundtruth-policy.json — frozen ground-truth policy (thresholds, budget)
├── parameters/ — one frozen parameter file per grid version used by these instances
│ └── benchmark-v1-*.json
├── instances/ — one instance file per fingerprint (9)
│ └── <fingerprint>.json
└── groundtruth/ — one ground-truth sidecar per fingerprint (9)
└── <fingerprint>.json
Files under instances/ and groundtruth/ are indexed by fingerprint (a hash of the instance's business content): the file name is never chosen, it is derived from content, which makes it impossible for an instance and its ground truth to silently drift apart.
Every file under instances/ and groundtruth/ shares the same envelope:
{
"schema_version": "v1",
"body": { "...": "content specific to this file" },
"provenance": {
"seed": 303,
"parameters": { "...": "what produced this content, deterministically" },
"runtime": {
"timestamp": "2026-07-21T11:25:17.674059+00:00",
"duration_ms": 79,
"machine": "...",
"tool_versions": {}
}
}
}body and provenance.seed / provenance.parameters form the deterministic core of the artifact: at equal seed and parameters, two generation runs produce this block byte-for-byte identically. provenance.runtime groups everything that legitimately varies from one run to the next (timestamp, machine, duration) — a reader comparing two runs should ignore this block entirely, never field by field.
The body carries three sections:
resources— machines (each with itsfamily), operators (each with an availabilitycalendar), the competency catalogue (skill_catalog), and the operator → held-competencies relation (skills);orders— orders, each with a due date, priority, quantity, and a routing: an ordered list of operations, each carrying the eligible machine family (machine_family), the required competency (required_skill), the setup configuration attribute (setup_attribute), and the unit duration;setup_matrix— the setup matrix: for every ordered pair of configuration attributes, the setup time (from,to,time).
provenance.parameters carries, in addition to the seed, the full set of generation parameters (number of machines, operators, competency density, setup intensity, etc.) and the generator version that produced the instance.
An array, one row per instance shipped in this repository (9), carrying the same fields as groundtruth-manifest.json — see §4.5 for the exact field list. manifest.json, groundtruth-manifest.json, and published-manifest.json are published as three synchronized copies of the same 9 rows in this release.
Kept separate from the instance (never folded into its provenance), indexed by the same fingerprint. Its body carries a typed sum, never a bare number:
"kind": "optimum_prouve"(proven optimum) —{ "value": <int>, "origin": ..., "independent_of_engine": true };"kind": "encadrement"(bracket) —{ "lower_bound": ..., "upper_bound": ..., "gap_numerator": ..., "gap_denominator": ..., "origin": ..., "independent_of_engine": true }, the relative gap being an exact rational (two integers), never a float.
origin takes one of three values — see §5.
An array, one row per instance shipped in this repository (9): fingerprint, seed, parameters_version, tier, difficulty, ground_truth_kind, origin, independent_of_engine, groundtruth_path.
Same row shape as §4.5. In Aplomb's source project, this file is derived from a larger groundtruth-manifest.json by a pure filter on ground_truth_kind == "optimum_prouve" — never hand-edited. In this repository the two files carry the same 9 rows, because every ground truth shipped here already passes that filter (§2).
Two frozen values, never hard-coded in decision logic: the offline exact-resolution budget (exact_solve_max_deterministic_time, in the solver's own deterministic units, never wall-clock seconds) and the maximum accepted gap for a bracket (bracket_gap_threshold_numerator / _denominator, an exact rational). Any instance whose bracket would exceed this threshold fails catalogue construction rather than being silently committed.
The frozen generation parameter set under its version identifier (number of machines, operators, orders, routing length, competency density, target load, duration spread, setup intensity). Once frozen, this file is authoritative — never the code that produced it.
Every instance receives its ground truth from one of three strategies, tried in this priority order:
- G-1 — Offline exact resolution. A CP-SAT exact solver models the full instance (dual resource, sequence-dependent setups, calendars) and searches for a proven optimum under a declared deterministic budget. This is the backbone: when it concludes, the ground truth is a proven optimum, without caveat.
- G-2 — Declared bracket (graceful degradation). A third strategy exists in Aplomb's general methodology, for instances where G-1 does not conclude within its budget — but no G-2 instance ships in this repository (§2); its gap threshold is frozen in
groundtruth-policy.json(§4.7). - G-3 — Restricted reverse construction. On a restricted class of shops built to carry an immediate optimality proof (independent queues, a setup that is a distance on a line), the ground truth is a proven optimum by construction, never a bracket. This catalogue uses it as an accelerator: a handful of extra instances at the hardest tier where G-1 does not conclude, so the catalogue also carries proven optima at large size.
What this catalogue publishes. Only the ground truths established by G-1 and G-3 — the two strategies that produce a proven optimum. No G-2 (bracket) instance ships in this repository.
Independence from the evaluated engine. All three strategies above, like the catalogue-construction pipeline, depend only on the problem domain and third-party tooling (the CP-SAT solver, the analytical bound) — never on a candidate scheduling engine. That is what makes the ground truth usable to judge an engine: it cannot have inherited the engine's own formulation. Every committed ground truth carries independent_of_engine: true, and Aplomb's source repository enforces this with a static guard (code analysis, not just data) verifying that no module producing ground truth imports the scheduling engine or the solution validator — a judge/defendant wall that closes on both sides. This same wall is what §10 relies on for the robustness signal.
The 9 published instances break down as follows:
| Tier | Difficulty | Ground truth | Instances |
|---|---|---|---|
| small | loose | proven optimum (G-1) | 3 |
| small | tight | proven optimum (G-1) | 3 |
| large, restricted reverse construction | — | proven optimum (G-3) | 3 |
Difficulty isn't just a version name: it's measured on the instances actually produced (more intense setups and rarer competencies under "tight") — see the guarantees table, §9. Aplomb's source project also generates instances at other size tiers and difficulty settings as part of a wider internal grid; those are not part of this release (§2).
Regenerating this catalogue from its seeds is done from Aplomb's source repository (not published here — see §12). From that repository, regeneration from an empty directory reproduces the exact same set of fingerprints and the same deterministic core for every instance, byte-for-byte, including published-manifest.json (since it is derived by a pure filter over groundtruth-manifest.json). This published repository distributes the resulting artifacts; it does not include the generator code itself.
See LICENSE. License: CC BY 4.0 (Creative Commons Attribution 4.0 International) — the convention for public scheduling benchmarks of this kind (Taillard, Brandimarte): an open license permitting reuse, republication, and commercial use of the dataset, on the sole condition of crediting the Aplomb project. See §11 for how to cite it formally.
Every guarantee stated in this README is backed by a named, automated test in Aplomb's source repository (private for now — see §12). This published repository does not include that test suite; it distributes the artifacts the suite protects.
| Guarantee |
|---|
| Every instance simultaneously carries the dual resource (machine AND competency) and a genuinely sequence-dependent setup — no trivial case |
| Instance size grows strictly with the tier (small < medium < large), at both difficulty settings |
| At equal tier, the "tight" setting is measurably harder than "loose" (more intense setups, rarer competencies) |
| The grid spans at least 3 tiers × 2 settings |
| Generator parameters never depend on a command line or environment variable — only on versioned files |
| Regenerating the catalogue from its seeds reproduces the same set of fingerprints |
| The deterministic core of every regenerated instance is byte-identical run to run |
| Replaying regeneration on an already-built catalogue has no effect |
| Every catalogue instance carries a ground truth — no orphan instance |
| Each instance's ground truth is a faithful typed sum: proven optimum or bracket, never conflated |
| Every committed bracket (G-2) stays under the policy's declared gap threshold |
| The manifest declares seed, parameter version, ground-truth type, and origin for every instance |
| G-3 (reverse-construction) instances are declared as their own type, never conflated with G-1/G-2 |
| Every committed ground truth carries the flag of independence from the evaluated engine |
| No module producing ground truth imports the scheduling engine or the validator (judge/defendant wall) |
| The published scope is exactly the proven-optimum instances — 9, no more, no less |
| The published scope is derived from the ground-truth manifest by a filter, never a hand-kept list |
| Every perturbation used by the Jalon R robustness signal (§10) reuses this same judge/defendant wall and the same offline exact-resolution strategy — never a shortcut of its own |
Everything above measures a fixed instance's absolute gap to a proven optimum. In July 2026, two commercial scheduling-solver vendors, contacted independently, converged on a different concern for their own segment: not "how far from a proven optimum," but, in one of their own words, "we place particular importance on robustness and computation time." A proven-optimum substrate can answer a question a moving "best known so far" cannot answer at all: how badly does the worst case degrade, and how fast does a scheduler approach the true target rather than a record that moves with it. That is the bet behind Jalon R (an internal milestone name meaning robustness milestone) — a bet the project treats explicitly as a bet, not an established fact, with a stated condition under which it would be considered disproven.
Jalon R measures how much a given scheduler's result degrades when a known shop-floor shock hits an instance whose optimum was already proven — it does not build a scheduler engineered to withstand a shock it hasn't seen yet. That second direction (robust or stochastic scheduling, built-in re-optimization) is a different, heavier undertaking, and is explicitly out of scope here. Blurring the two would smuggle a second engine — a re-scheduler — into the measurement itself, one the independent validator would never see: exactly what the judge/defendant wall (§5) exists to prevent.
Concretely, the pipeline runs four steps: it perturbs a base instance (already at a proven optimum) with a realistic shop-floor shock; it re-solves the perturbed instance to a fresh proven optimum, using the same offline exact-resolution strategy as the base catalogue (§5) — never recycling the old, now-obsolete one; it runs the scheduler under test on the perturbed instance; it validates, then measures, that scheduler's gap to this new, true optimum.
Each shock carries a canonical scenario identifier (S1–S4), the same one used throughout Aplomb's internal architecture and story specs — cited here so an S1–S4 reference encountered elsewhere (a companion paper, an issue tracker) maps back unambiguously to this table.
| ID | Shock | What happens to the synthetic shop | In a shop foreman's words |
|---|---|---|---|
| S1 | Setter/operator unavailable | A named competency becomes unavailable on an operator — some operations lose their only qualified person | "My best setter is out." |
| S2 | Machine breakdown | A machine drops out of available capacity (at least one remains in the same family) | "CNC-3 went down this morning." |
| S3 | Urgent order | A priority order is inserted with a tight due date | "Client X jumps the queue." |
| S4 | Reduced capacity | An availability window shortens (fewer hours, fewer people) | "Today we're running two setters instead of three." |
Each transformation is a pure, seed-deterministic function from instance to instance: it never mutates the base instance, and it never imports the scheduling engine or the validator — a guarantee checked by the same static code guard as the rest of this catalogue (§5, last row of §9). A perturbation can legitimately make an instance infeasible (e.g. removing the only operator qualified for an operation still in the routing); when it does, re-solving proves that infeasibility and the pipeline declares it explicitly, rather than inventing a measurement where none is possible.
A versioned reference family illustrates what "perturb" actually moves. Its base instance has a proven optimum of 26 (shop time units). Each of its four perturbed variants is independently re-solved to its own fresh proven optimum:
| Shock applied | Perturbed instance's proven optimum | Change vs. base (26) |
|---|---|---|
| — (base instance) | 26 | — |
| S1 — Setter/operator unavailable | 28 | +7.7% |
| S2 — Machine breakdown | 46 | +76.9% |
| S3 — Urgent order | 35 | +34.6% |
| S4 — Reduced capacity | 26 | 0% |
The last row is worth a sentence rather than silence: a shock that changes the instance without moving its optimum is not a hidden defect — the generator's guarantee is only that it produces a valid perturbed instance, not that every shock necessarily bites the critical path. Whether a given shock actually moved the optimum is exactly what the re-solving-and-measuring step (§10.2) exists to reveal, case by case.
The seven acceptance criteria that define this measurement are all built, tested, and merged:
- a shock generator producing, for all four scenarios above, a valid perturbed instance outside the private engine/validator's own code;
- re-solving the perturbed instance to a fresh proven optimum (or proving its infeasibility, never leaving it unresolved);
- a worst-case gap computed as an aggregate over a family of perturbed instances;
- that family's gap variance, computed in exact arithmetic (a fraction, never a float that would contaminate the measurement);
- a time-to-quality metric anchored on the solver's own deterministic compute budget (iterations, conflicts) — never on wall-clock time, so it stays comparable machine to machine; when a third-party engine doesn't expose this trajectory, the result is declared unavailable rather than approximated;
- an end-to-end pipeline (perturb → re-solve → run the candidate → validate → measure) whose output is accepted by the same independent validator that judges this catalogue's main engine — the judge/defendant wall (§5) holds here too;
- a versioned, reproducible reference family (a base instance × its four perturbations, at a fixed seed) — the one whose figures appear in §10.4.
What this milestone does not yet provide is a comparative robustness result. The only scheduler run through this pipeline so far is the same exact solver that establishes ground truth in the first place — asking it to re-solve a perturbed instance is asking it to re-prove an optimum it just proved: the measured gap is trivially zero, and says nothing about a real scheduler's robustness. A number that means something — a non-trivial worst-case gap, a variance that tells a gracefully-degrading scheduler apart from one that collapses on certain shocks — requires plugging in a second, distinct scheduler. That is the next step, not something already in hand. As with the rest of this catalogue, only one objective (makespan) is wired into this measurement so far, even though the underlying design treats the objective as swappable. None of this milestone's artifacts are part of a published dataset release yet.
The dataset (available now — see §2 for its scope):
Aplomb project (2026). Aplomb Benchmark: DRC-FJSP-SDST (v1.0) [Data set].
https://github.com/charlesnauroy-wq/aplomb-benchmark. License: CC BY 4.0.
@misc{aplomb_benchmark_2026,
author = {{Aplomb project}},
title = {{Aplomb Benchmark: DRC-FJSP-SDST}},
year = {2026},
version = {1.0},
publisher = {GitHub},
url = {https://github.com/charlesnauroy-wq/aplomb-benchmark},
note = {Dataset, CC BY 4.0}
}See CITATION.cff for the same record in machine-readable form.
The companion working paper — method, motivation, and a full, unvarnished discussion of the catalogue's current limits — is being prepared for submission to HAL (open-access repository) as a working paper ("pré-publication, document de travail"), expected 2026-08-04. Until the deposit exists, the entry below is a placeholder: it will be replaced by HAL's own auto-generated BibTeX record (with the permanent hal-XXXXXXX identifier and URL) as soon as the paper is live.
@unpublished{aplomb_hal_2026,
author = {{Aplomb project}},
title = {{Synthetic Job Shops with Proven Optima: Measuring Rather Than
Estimating the Optimality Gap in Dual-Resource-Constrained
Flexible Job-Shop Scheduling with Sequence-Dependent Setup
Times}},
note = {Working paper, HAL (hal.science). HAL identifier and URL to
be added once submitted (expected 2026-08-04) — replace this
placeholder with HAL's auto-generated BibTeX entry.},
year = {2026}
}Working-paper link: to be added here once the deposit is live.
CC BY 4.0 permits reuse, republication, and commercial use of the dataset, on the sole condition of crediting the Aplomb project. For questions about the benchmark's design, to reproduce the generation, or to check a claim, see the contact information in CITATION.cff.
Aplomb is a scheduling test bench for machining / precision-turning shops, built around the DRC-FJSP-SDST problem class described above. Its defining idea: generate synthetic shop instances whose optimal solution is already known, run a scheduler against them, and measure the actual gap to the optimum — rather than to a "best known so far" record that keeps moving.
This dataset, and the Jalon R robustness signal described in §10, are artifacts of that project. The generator, the exact solver used for ground truth, the scheduling engine being evaluated, and the solution validator all live in Aplomb's source repository, which stays private. If you are a solver vendor or a research lab and want to go further — to reproduce the generation, to check a claim, or to discuss the benchmark's design, including a possible second scheduler for §10.5 — see CITATION.cff for contact.
It contains neither a scheduling engine nor the result of one: ground truth (§5) and the Jalon R robustness signal (§10) are both established by third-party tooling or offline exact resolution, never by the system this benchmark is meant to evaluate. And, as stated plainly in §10.5, it does not yet contain a comparative robustness result between distinct schedulers — only the instrument built to produce one.