Finalize process session ownership

Cargo.toml

@@ -23,7 +23,6 @@ clap = { version = "4.6.4", features = ["derive", "string"] }
 ctrlc = "3.5.2"
 indicatif = { version = "0.18.6", default-features = false }
 rand = "=0.10.2"
-rand_core = "=0.10.1"
 rand_distr = "=0.6.0"
 serde = { version = "1.0.228", features = ["derive"] }
 serde_json = "1.0.149"
@@ -37,11 +36,12 @@ toml = { version = "1.1.4", default-features = false, features = [
 ] }
 
 [target.'cfg(unix)'.dependencies]
-libc = "=0.2.189"
+rustix = { version = "1", features = ["event", "fs", "pipe", "process"] }
 
 [target.'cfg(windows)'.dependencies]
 windows-sys = { version = "=0.61.2", features = [
     "Win32_Foundation",
+    "Win32_Globalization",
     "Win32_Security",
     "Win32_Storage_FileSystem",
     "Win32_System_JobObjects",

README.md

@@ -9,7 +9,7 @@ NB: `foil` is currently experimental, the API may change without warning.
 ## Usage
 
 ```sh
-foil --baseline main --candidate HEAD --repetitions 30 --interval 0.5 0.8 0.98 --output-dir benchmark/ -- cargo bench
+foil --baseline main --candidate HEAD --repetitions 30 --interval 0.5 0.8 0.9 --output-dir benchmark/ -- cargo bench
 ```
 
 Run `foil --help` for the full set of options.
@@ -27,30 +27,34 @@ baseline = "main"
 candidate = "HEAD"
 repetitions = 30
 block-size = 4
-interval = [0.5, 0.8, 0.98]
+interval = [0.5, 0.8, 0.9]
 output-dir = "benchmark/"
 ```
 
 With that file, the run above is `foil -- cargo bench`. Arguments override the file, which overrides the built-in defaults. `foil --help` always shows only the built-in CLI defaults. The command may also live in the file as a `command` list; one passed after `--` overrides it.
 
 Run order uses small-block randomization. The default `block-size = 4` gives each full block two baseline-first and two candidate-first pairs; `block-size = 1` is the minimum.
 
-Lifecycle commands surround the suite, each benchmark, or every measured run. Top-level `startup` and `teardown` run once in the original checkout around the whole suite. The same keys in a benchmark run once in each revision worktree around that benchmark. `startup-each-run` and `teardown-each-run` run outside every timed interval; suite and benchmark commands compose, with teardown unwinding in reverse order.
+Lifecycle hooks are configured only in TOML. `suite-startup` runs once in the original checkout before revision worktrees are created; `suite-teardown` runs there after every worktree has been removed. `worktree-startup` and `worktree-teardown` run once in each newly created baseline or candidate worktree. Top-level `startup-each-run` and `teardown-each-run` surround every measured command, while benchmark-local lifecycle hooks apply only to that benchmark.
 
 ```toml
-startup = ["docker", "compose", "up", "-d"]
-startup-each-run = ["reset-database"]
-teardown-each-run = ["collect-logs"]
-teardown = ["docker", "compose", "down"]
+suite-startup = ["docker", "compose", "up", "-d"]
+suite-teardown = ["docker", "compose", "down"]
+worktree-startup = ["git", "submodule", "update", "--init"]
+worktree-teardown = ["git", "clean", "-fdx"]
+startup-each-run = ["reset-global-state"]
+teardown-each-run = ["collect-global-state"]
 
 [benchmarks.parse]
 startup = ["cargo", "build", "--release"]
+startup-each-run = ["reset-database"]
+teardown-each-run = ["collect-logs"]
 command = ["./target/release/parse", "corpus/"]
 ```
 
-Benchmark lifecycle commands share the benchmark's `working-directory` and `env`. Successful lifecycle output is suppressed; failures report both nonempty streams under explicit labels. `foil` discards stdout and stderr from measured commands. If output is part of the workload, redirect it explicitly in the benchmark command. Teardown is still attempted after startup, benchmark, timeout, or interruption failures; the original error remains primary and additional cleanup errors are also reported.
+Benchmark lifecycle commands share the benchmark's `working-directory` and `env`. Their stdout and stderr are discarded, like measured commands'; redirect explicitly if the output matters. The first Ctrl-C interrupts active startup or benchmark work, then teardown unwinds on a protected cleanup wait. A second Ctrl-C exits immediately. Teardown is also attempted after startup, benchmark, or timeout failures; the original error remains primary and additional cleanup errors are reported alongside it. On macOS, containment uses a process group, which a descendant can deliberately escape with `setsid` or `setpgid`.
 
-A `[benchmarks]` table is where a command belongs in TOML. Each entry names a benchmark for `--benchmark` to select and typically sets its own `command`; it may override ordinary options, and anything it leaves unset, including `command`, is inherited from the top level. Lifecycle commands are not inherited: suite and benchmark lifecycles remain distinct and compose. Its `env` table is merged with the top-level one, variable by variable, with the benchmark's values winning on conflicts:
+A `[benchmarks]` table is where a command belongs in TOML. Each entry names a benchmark for `--benchmark` to select and typically sets its own `command`; it may override ordinary options, and anything it leaves unset, including `command`, is inherited from the top level. Benchmark lifecycle commands are local to that benchmark. Its `env` table is merged with the top-level one, variable by variable, with the benchmark's values winning on conflicts:
 
 ```toml
 repetitions = 10
@@ -68,7 +72,7 @@ command = ["cargo", "run", "--release", "--", "render"]
 RAYON_NUM_THREADS = "1"
 ```
 
-`foil --benchmark render` runs with 50 repetitions in `benchmarks/render`; `foil --benchmark parse` runs with the top-level 10. An explicit argument still overrides a benchmark's setting, except for `command`, `working-directory`, and `env`. Those define what a benchmark is, so one argument cannot sensibly stand in for all of the selected benchmarks, and passing one alongside a benchmark is an error. Lifecycle arguments apply to the suite.
+`foil --benchmark render` runs with 50 repetitions in `benchmarks/render`; `foil --benchmark parse` runs with the top-level 10. An explicit argument still overrides a benchmark's setting, except for `command`, `working-directory`, and `env`. Those define what a benchmark is, so one argument cannot sensibly stand in for all of the selected benchmarks, and passing one alongside a benchmark is an error. Lifecycle hooks are TOML-only.
 `working-directory` must be a relative path within the worktree; absolute paths and `..` are rejected.
 
 With no `--benchmark`, every benchmark in the table runs in declaration order, each in its own `--output-dir` subdirectory named after it. Pass `--benchmark render parse` to run only some of them in the order given. A configuration with no `[benchmarks]` table always runs a single, unnamed command, exactly as with no configuration file at all.
@@ -89,12 +93,7 @@ Each run writes to its output directory:
 
 Library callers can pass `measurements.csv` to `analyze_measurements`; the same seed, draw count, shrinkage, and intervals reproduce the CLI posterior exactly.
 
-Running more than one benchmark also prints a one-line-per-benchmark summary and writes it to `report_short.txt` in `--output-dir`:
-
-```
-parse: 1.2s -> 554.0ms [-52.41%, -51.31%]
-render: 3.1s -> 3.0s [-4.02%, +1.15%]
-```
+Named benchmark reports are prefixed with the benchmark name when printed.
 
 ## License
 

foil.toml

@@ -2,8 +2,12 @@ repetitions = 30
 block-size = 4
 output-dir = "benchmark/results"
 timeout = 1800
+worktree-startup = ["cargo", "fetch", "--locked"]
 
 [benchmarks.release-compile]
-startup = ["cargo", "fetch", "--locked"]
 startup-each-run = ["cargo", "clean", "--release"]
 command = ["cargo", "build", "--release", "--locked", "--offline"]
+
+[benchmarks.release-test]
+startup = ["cargo", "test", "--release", "--locked", "--offline", "--no-run"]
+command = ["cargo", "test", "--release", "--locked", "--offline"]

src/analysis.rs

@@ -1,10 +1,10 @@
-use crate::run::RunOutput;
+use crate::run::Measurement;
 use crate::{
     Interval, Pair, Posterior, Repetition, Repetitions, RunOrder, Shrinkage, Summary, Time,
 };
 use anyhow::{Context, Result, ensure};
+use rand::SeedableRng;
 use rand::rngs::Xoshiro256PlusPlus;
-use rand_core::SeedableRng;
 use std::{
     fs::File,
     io::{BufRead, BufReader},
@@ -27,22 +27,19 @@ pub fn analyze_measurements(
     shrinkage: Shrinkage,
     intervals: &[Interval],
 ) -> Result<Analysis> {
-    analyze(&read_measurements(path)?, seed, draws, shrinkage, intervals)
-}
-
-pub(crate) fn analyze(
-    repetitions: &Repetitions,
-    seed: u64,
-    draws: NonZeroUsize,
-    shrinkage: Shrinkage,
-    intervals: &[Interval],
-) -> Result<Analysis> {
-    analyze_checked(repetitions, seed, draws, shrinkage, intervals, || Ok(()))
+    analyze_checked(
+        &read_measurements(path)?,
+        seed,
+        draws,
+        shrinkage,
+        intervals,
+        || Ok(()),
+    )
 }
 
 /// Analyzes validated repetitions, checking for cancellation between draws.
 pub(crate) fn analyze_checked(
-    repetitions: &Repetitions,
+    repetitions: &Repetitions<Measurement>,
     seed: u64,
     draws: NonZeroUsize,
     shrinkage: Shrinkage,
@@ -61,7 +58,7 @@ pub(crate) fn analyze_checked(
     Ok(Analysis { posterior, summary })
 }
 
-fn read_measurements(path: &Path) -> Result<Repetitions> {
+fn read_measurements(path: &Path) -> Result<Repetitions<Measurement>> {
     let mut lines = BufReader::new(
         File::open(path).with_context(|| format!("Failed to read {}.", path.display()))?,
     )
@@ -96,16 +93,17 @@ fn read_measurements(path: &Path) -> Result<Repetitions> {
                 "candidate_first" => RunOrder::CandidateFirst,
                 order => anyhow::bail!("Unknown run order `{order}`."),
             };
-            let elapsed = |field: &str| -> Result<RunOutput> {
+            let elapsed = |field: &str| -> Result<Measurement> {
                 let seconds: f64 = field.parse()?;
                 ensure!(
                     seconds.is_finite() && seconds >= 0.0,
                     "Measurement must be finite and nonnegative."
                 );
-                Ok(RunOutput::measurement(
-                    Duration::try_from_secs_f64(seconds)
+                Ok(Measurement {
+                    elapsed: Duration::try_from_secs_f64(seconds)
                         .context("Measurement is too large to represent.")?,
-                ))
+                    peak_memory: None,
+                })
             };
             Ok(Repetition {
                 outputs: Pair {
@@ -118,88 +116,3 @@ fn read_measurements(path: &Path) -> Result<Repetitions> {
         .collect::<Result<Vec<_>>>()?
         .try_into()
 }
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-    use crate::MeasurementsCsv;
-    use std::fs;
-    use tempfile::tempdir;
-
-    fn repetitions() -> Result<Repetitions> {
-        (0..10)
-            .map(|index| Repetition {
-                outputs: Pair {
-                    baseline: RunOutput::measurement(Duration::from_secs_f64(1.0 + index as f64)),
-                    candidate: RunOutput::measurement(Duration::from_secs_f64(1.1 + index as f64)),
-                },
-                order: if index % 2 == 0 {
-                    RunOrder::BaselineFirst
-                } else {
-                    RunOrder::CandidateFirst
-                },
-            })
-            .collect::<Vec<_>>()
-            .try_into()
-    }
-
-    #[test]
-    fn csv_analysis_matches_in_memory_analysis_at_the_same_seed() -> Result<()> {
-        let repetitions = repetitions()?;
-        let directory = tempdir()?;
-        let path = directory.path().join("measurements.csv");
-        let mut csv = MeasurementsCsv::create(&path)?;
-        for repetition in repetitions.iter() {
-            csv.append(repetition)?;
-        }
-        drop(csv);
-        let intervals = [
-            Interval::new(0.5)?,
-            Interval::new(0.8)?,
-            Interval::new(0.98)?,
-        ];
-        let draws = NonZeroUsize::new(1_000).unwrap();
-
-        let from_memory = analyze(&repetitions, 0, draws, Shrinkage::NONE, &intervals)?;
-        let from_csv = analyze_measurements(&path, 0, draws, Shrinkage::NONE, &intervals)?;
-
-        assert_eq!(from_csv.posterior.draws(), from_memory.posterior.draws());
-        assert_eq!(from_csv.summary, from_memory.summary);
-        Ok(())
-    }
-
-    #[test]
-    fn an_unrepresentable_duration_is_an_error() -> Result<()> {
-        let directory = tempdir()?;
-        let path = directory.path().join("measurements.csv");
-        let mut csv = String::from("repetition,order,baseline_seconds,candidate_seconds\n");
-        for repetition in 1..=10 {
-            let order = if repetition % 2 == 0 {
-                "baseline_first"
-            } else {
-                "candidate_first"
-            };
-            csv.push_str(&format!("{repetition},{order},1e300,1\n"));
-        }
-        fs::write(&path, csv)?;
-
-        assert!(read_measurements(&path).is_err());
-        Ok(())
-    }
-
-    #[test]
-    fn an_empty_interval_set_is_rejected() -> Result<()> {
-        let error = analyze(
-            &repetitions()?,
-            0,
-            NonZeroUsize::new(1_000).unwrap(),
-            Shrinkage::NONE,
-            &[],
-        )
-        .err()
-        .context("An empty interval set should fail.")?;
-
-        assert!(error.to_string().contains("interval"), "{error}");
-        Ok(())
-    }
-}