diff --git a/CHANGELOG.md b/CHANGELOG.md index c490555c..cf5d35bf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ stricter subset of Keep a Changelog). ### Changed +- Auditing CRS type: does not require a target harness and produces bug-candidates - `oss-crs export` and `oss-crs import` commands — imports/exports docker images/CRS source code to transfer to another host. - Run-phase modules now default to `target_dependent: true`, so their images are built once per target during `build-target`. Set `target_dependent: false` for modules that can be built once during `prepare`. - `--offline` flag for all subcommands: disables git fetch diff --git a/docs/config/crs.md b/docs/config/crs.md index 5c965ceb..3ec9bbfc 100644 --- a/docs/config/crs.md +++ b/docs/config/crs.md @@ -231,8 +231,8 @@ The `supported_target` section defines what types of targets the CRS can work wi |-------|------|----------|-------------| | `mode` | `Set[TargetMode]` | Yes | Supported target modes (see [TargetMode](#targetmode)) | | `language` | `Set[TargetLanguage]` | Yes | Supported programming languages (see [TargetLanguage](#targetlanguage)) | -| `sanitizer` | `Set[TargetSanitizer]` | Yes | Supported sanitizers (see [TargetSanitizer](#targetsanitizer)) | -| `architecture` | `Set[TargetArch]` | Yes | Supported CPU architectures (see [TargetArch](#targetarch)) | +| `sanitizer` | `Set[TargetSanitizer]` | No | Supported sanitizers (see [TargetSanitizer](#targetsanitizer)). Defaults to all sanitizers when omitted. | +| `architecture` | `Set[TargetArch]` | No | Supported CPU architectures (see [TargetArch](#targetarch)). Defaults to all architectures when omitted. | | `fuzzing_engine` | `Set[FuzzingEngine]` | No | Supported fuzzing engines (see [FuzzingEngine](#fuzzingengine)). Defaults to all engines when omitted. | ### Example @@ -271,6 +271,7 @@ Defines the type of CRS: | `bug-fixing-ensemble` | Ensemble CRS that aggregates bug-fixing outputs | | `bug-finding-triage` | Post-processor CRS that triages bug-finding results (e.g. deduplication, validation). Reads from the main exchange dir and writes to the processed exchange dir. | | `seed-filter` | Post-processor CRS that filters or prioritizes seeds/inputs for downstream CRS. Reads from the main exchange dir and writes to the processed exchange dir. | +| `auditing` | Source-auditing CRS that analyzes `OSS_CRS_TARGET_SOURCE` and submits `bug-candidate` artifacts. It may run with or without a target harness. | ### TargetMode diff --git a/docs/config/target-project.md b/docs/config/target-project.md index a10d8403..d7f5b3ad 100644 --- a/docs/config/target-project.md +++ b/docs/config/target-project.md @@ -33,6 +33,8 @@ Full spec (for OSS-Fuzz compatibility/reference): [OSS-Fuzz project.yaml referen ## Usage +### Harnessed Run + ```bash # Build target uv run oss-crs build-target \ @@ -46,29 +48,34 @@ uv run oss-crs run \ --target-harness ``` -| Argument | Required | Description | -|-----------------------|----------|--------------------------------------------------------------------------------| -| `--fuzz-proj-path` (`--target-path`, `--target-proj-path`, deprecated aliases) | Yes | Path to the OSS-Fuzz target project directory (`Dockerfile`, `build.sh`; `project.yaml` optional). | -| `--target-source-path` | No | Optional local source override path. If set, source is synchronized with `rsync -a --delete` into the effective Dockerfile `WORKDIR`. | -| `--target-harness` | Yes (run)| Fuzz target harness binary name. | +### Source-Only Run -Existing [OSS-Fuzz projects](https://github.com/google/oss-fuzz/tree/master/projects) can be used directly as `--fuzz-proj-path` without modification. +```bash +uv run oss-crs run \ + --compose-file ./crs-compose.yaml \ + --target-source-path /path/to/source +``` -### Source Path Semantics +Source-only runs omit `--fuzz-proj-path` entirely. The source path is +directly bind-mounted to `OSS_CRS_TARGET_SOURCE`. There is no build step, +no `OSS_CRS_FUZZ_PROJ` mount, and `SANITIZER`, `ARCHITECTURE`, +`FUZZING_LANGUAGE`, etc. not injected into source-only containers. -- `OSS_CRS_PROJ_PATH` points to the copied target project directory. -- `OSS_CRS_REPO_PATH` points to the effective final Dockerfile `WORKDIR` inside - the target image. -- `WORKDIR` resolution follows Dockerfile semantics, with fallback chain: - final `WORKDIR` -> `$SRC` -> `/src` (when `SRC` is not provided). -- `libCRS download-source repo` prefers the live runtime source workspace - rooted at `$SRC`/`/src`. When `OSS_CRS_REPO_PATH` is inside that workspace, - the downloaded tree preserves the workspace layout rather than flattening a - nested `WORKDIR`. -- When `--target-source-path` is set, the override source is synchronized into - `OSS_CRS_REPO_PATH` via `rsync -a --delete`. +## Arguments + +| Argument | Required | Description | +|----------|----------|-------------| +| `--fuzz-proj-path` (`--target-path`, `--target-proj-path`, deprecated aliases) | Yes for harnessed/harness-gen runs; omitted for source-only runs | Path to the OSS-Fuzz target project directory (`Dockerfile`, `build.sh`; `project.yaml` optional). | +| `--target-source-path` | Required for source-only runs; optional local source override otherwise | Path to the source tree. For source-only runs, this is directly bind-mounted. For harnessed runs, source is synchronized with `rsync -a --delete` into the effective Dockerfile `WORKDIR`. | +| `--target-harness` | Yes (run) | Fuzz target harness binary name. | -### `--target-source-path` Sync Flow +At least one of `--fuzz-proj-path` or `--target-source-path` is required. + +Existing [OSS-Fuzz projects](https://github.com/google/oss-fuzz/tree/master/projects) can be used directly as `--fuzz-proj-path` without modification. + +## Source Path Semantics + +### Harnessed Runs with Source Override `--target-source-path` is not bind-mounted directly to `OSS_CRS_REPO_PATH`. Instead, during image build: @@ -78,3 +85,21 @@ Instead, during image build: 3. `rsync -a --delete /OSS_CRS_REPO_OVERRIDE/ ./` runs from the effective `WORKDIR`. 4. `OSS_CRS_REPO_PATH` points to that effective `WORKDIR` path. + +### Source-Only Runs + +For source-only runs, `--target-source-path` is directly bind-mounted to +`OSS_CRS_TARGET_SOURCE`. There is no image build, no `OSS_CRS_FUZZ_PROJ` +mount, and environment variables like `SANITIZER`, `ARCHITECTURE`, `FUZZING_LANGUAGE`, +etc. are not injected. + +### Common Semantics + +- `OSS_CRS_REPO_PATH` points to the effective final Dockerfile `WORKDIR` + inside the target image. +- `WORKDIR` resolution follows Dockerfile semantics, with fallback chain: + final `WORKDIR` -> `$SRC` -> `/src` (when `SRC` is not provided). +- `libCRS download-source repo` prefers the live runtime source workspace + rooted at `$SRC`/`/src`. When `OSS_CRS_REPO_PATH` is inside that workspace, + the downloaded tree preserves the workspace layout rather than flattening a + nested `WORKDIR`. diff --git a/docs/crs-development-guide.md b/docs/crs-development-guide.md index 188e45ae..6ed4a02c 100644 --- a/docs/crs-development-guide.md +++ b/docs/crs-development-guide.md @@ -66,7 +66,7 @@ The `crs.yaml` file is the central configuration for your CRS. It tells OSS-CRS ```yaml name: my-crs type: - - bug-finding # bug-finding, bug-fixing, or both + - bug-finding # bug-finding, bug-fixing, etc. version: "1.0.0" docker_registry: "ghcr.io/my-org/my-crs" @@ -131,7 +131,7 @@ required_envs: | Field | Description | |---|---| | `name` | Unique name for your CRS | -| `type` | Set of CRS types: `bug-finding`, `bug-fixing` | +| `type` | Set of CRS capabilities: `bug-finding`, `bug-fixing`, etc. | | `version` | Version string (used as a Docker image tag) | | `docker_registry` | Docker registry URL for your CRS images | | `prepare_phase.hcl` | Path to the HCL file for `docker buildx bake` | @@ -346,7 +346,7 @@ Your containers receive these environment variables automatically: | `OSS_CRS_NAME` | CRS name (from `crs-compose.yaml`) | `my-crs` | | `OSS_CRS_SERVICE_NAME` | Full service name | `my-crs_fuzzer` | | `OSS_CRS_TARGET` | Target project name | `libxml2` | -| `OSS_CRS_TARGET_HARNESS` | Target harness binary name | `xml` | +| `OSS_CRS_TARGET_HARNESS` | Target harness binary name. Unset for no-harness source-level runs. | `xml` | | `OSS_CRS_CPUSET` | Allocated CPU cores | `4-7` | | `OSS_CRS_MEMORY_LIMIT` | Memory limit | `16G` | | `OSS_CRS_BUILD_OUT_DIR` | Build output directory (read-only at run time) | `/OSS_CRS_BUILD_OUT_DIR` | @@ -799,6 +799,12 @@ Your CRS should submit findings through libCRS: - **`register-submit-dir`** — Best for high-volume output. Forks a daemon that watches the directory, deduplicates files by hash, and submits in batches. Use this for seeds and PoVs. - **`submit`** — Best for one-off submissions. Submits a single file immediately. +### Source-Level Bug Finding Without A Harness + +Source-only runs are invoked without `--fuzz-proj-path` and instead use `--target-source-path` to point at the source tree. They skip OSS-Fuzz target image builds and require every run module to set `target_dependent: false`; run `oss-crs prepare` first to build those target-independent images. They do not mount `OSS_CRS_BUILD_OUT_DIR` or `OSS_CRS_FUZZ_PROJ`, but still receive `OSS_CRS_SUBMIT_DIR`, `OSS_CRS_FETCH_DIR`, `OSS_CRS_SHARED_DIR`, `OSS_CRS_LOG_DIR`, and `OSS_CRS_TARGET_SOURCE`. They do not receive `OSS_CRS_TARGET_HARNESS`, `SANITIZER`, `ARCHITECTURE`, `FUZZING_LANGUAGE`, `FUZZING_ENGINE`, `HELPER`, or `RUN_FUZZER_MODE`, since none of these are consumed by source-level analysis. + +Source-only runs require all CRSs to be of type `auditing`. This ensures that only CRSs designed to analyze source code without a compiled target can run in source-only mode. An `auditing` CRS is a regular producer that reads `OSS_CRS_TARGET_SOURCE` and submits `bug-candidate` artifacts. The type is a capability label rather than a source-only restriction: auditors may run alone without `--target-harness` or alongside harness-based CRSs. + --- ## Fetching Data @@ -914,6 +920,7 @@ Before publishing your CRS, verify: - [ ] Run-phase Dockerfiles install libCRS (`COPY --from=libcrs . /opt/libCRS && RUN /opt/libCRS/install.sh`) - [ ] Containers download build outputs at startup via `libCRS download-build-output` - [ ] Artifact directories are registered with `libCRS register-submit-dir` +- [ ] Auditing CRSs submit findings as `bug-candidate` artifacts - [ ] `supported_target` accurately reflects your CRS capabilities - [ ] `required_llms` lists all models used (if any) - [ ] `required_inputs` lists inputs the CRS depends on (if any) diff --git a/docs/registry.md b/docs/registry.md index 73eadc4c..9d48eb7c 100644 --- a/docs/registry.md +++ b/docs/registry.md @@ -18,7 +18,7 @@ source: | Field | Description | |---|---| | `name` | Unique identifier for the CRS | -| `type` | List of CRS capabilities — `bug-finding`, `bug-fixing`, `bug-finding-triage`, `seed-filter`, `bug-fixing-ensemble`, or a combination | +| `type` | List of CRS capabilities — `bug-finding`, `bug-fixing`, `auditing`, `bug-finding-triage`, `seed-filter`, `bug-fixing-ensemble`, or a combination | | `source.url` | Git repository URL containing the CRS implementation | | `source.ref` | Git branch or tag to use | diff --git a/oss_crs/src/cli/archive.py b/oss_crs/src/cli/archive.py index be17f511..6fb112fd 100644 --- a/oss_crs/src/cli/archive.py +++ b/oss_crs/src/cli/archive.py @@ -10,13 +10,17 @@ from ..target import Target -def handle_archive(args, crs_compose, target: Target) -> bool: +def handle_archive( + args, crs_compose, target: Target, *, unharnessed: bool = False +) -> bool: """Handle the archive command.""" - ctx = resolve_run_context(args, crs_compose, target) + ctx = resolve_run_context(args, crs_compose, target, unharnessed=unharnessed) if ctx is None: return False - sanitizer, run_id = ctx - harness = target.target_harness + sanitizer, run_id, scope_known = ctx + if not scope_known: + print("No run artifact scope found for the selected run.", file=sys.stderr) + return False work_dir = crs_compose.work_dir out_path = Path(args.out) @@ -25,8 +29,9 @@ def handle_archive(args, crs_compose, target: Target) -> bool: triage_crs = [crs for crs in crs_compose.crs_list if crs.config.is_triage] non_triage_crs = [crs for crs in crs_compose.crs_list if not crs.config.is_triage] - # Collect (arcname, src_path) pairs for each artifact subdir - artifact_subdirs = list(SUBMITTED_ARTIFACT_DIR_NAMES) + # Collect (arcname, src_path) pairs for each artifact subdir. + # Include "harnesses" locally (harness-gen output is not exchanged). + artifact_subdirs = list(SUBMITTED_ARTIFACT_DIR_NAMES) + ["harnesses"] # In a triage run, POVs come from the triage CRS instead (see below). non_pov_subdirs = [s for s in artifact_subdirs if s not in ("povs", "reports")] @@ -81,27 +86,26 @@ def _add_dir(collected: list, src_dir: Path, arcname_prefix: str) -> None: if args.include_all: # Also include exchange dir, logs, and shared dirs - if harness: - exchange_dir = work_dir.get_exchange_dir( - target, run_id, sanitizer, create=False + exchange_dir = work_dir.get_exchange_dir( + target, run_id, sanitizer, create=False + ) + _add_dir(collected, exchange_dir, "exchange") + + run_logs_dir = work_dir.get_run_logs_dir( + target, run_id, sanitizer, create=False + ) + _add_dir(collected, run_logs_dir, "logs") + + for crs in crs_compose.crs_list: + shared_dir = work_dir.get_shared_dir( + crs.name, target, run_id, sanitizer, create=False ) - _add_dir(collected, exchange_dir, "exchange") + _add_dir(collected, shared_dir, f"shared/{crs.name}") - run_logs_dir = work_dir.get_run_logs_dir( - target, run_id, sanitizer, create=False + log_dir = work_dir.get_log_dir( + crs.name, target, run_id, sanitizer, create=False ) - _add_dir(collected, run_logs_dir, "logs") - - for crs in crs_compose.crs_list: - shared_dir = work_dir.get_shared_dir( - crs.name, target, run_id, sanitizer, create=False - ) - _add_dir(collected, shared_dir, f"shared/{crs.name}") - - log_dir = work_dir.get_log_dir( - crs.name, target, run_id, sanitizer, create=False - ) - _add_dir(collected, log_dir, f"logs/crs/{crs.name}") + _add_dir(collected, log_dir, f"logs/crs/{crs.name}") if not collected: print("No artifacts found for the selected run.", file=sys.stderr) diff --git a/oss_crs/src/cli/artifacts.py b/oss_crs/src/cli/artifacts.py index 63b49fc5..9565dd10 100644 --- a/oss_crs/src/cli/artifacts.py +++ b/oss_crs/src/cli/artifacts.py @@ -12,12 +12,18 @@ RunLogs, RunMeta, ) +from ..constants import UNHARNESSED from ..target import Target from ..utils import normalize_run_id, select def collect_run_ids_for_target( - crs_compose, target: Target, harness: str | None, sanitizer: str + crs_compose, + target: Target, + harness: str | None, + sanitizer: str, + *, + unharnessed: bool = False, ) -> list[str]: """Collect all run-ids for a target from SUBMIT_DIR (run artifacts).""" seen = set() @@ -32,8 +38,8 @@ def collect_run_ids_for_target( submit_path = crs_compose.work_dir.get_submit_dir( crs.name, target, entry.run_id, sanitizer, create=False ) - # For harness=None, check parent dir (without harness component) - if not harness: + if not harness and not unharnessed: + # No requested harness: discover runs under any scope. submit_path = submit_path.parent if submit_path.exists(): seen.add(entry.run_id) @@ -60,10 +66,17 @@ def format_run_id(run_id: str) -> str: def select_run_id_interactively( - crs_compose, target: Target, harness: str | None, sanitizer: str + crs_compose, + target: Target, + harness: str | None, + sanitizer: str, + *, + unharnessed: bool = False, ) -> str | None: """Prompt user to select a run-id from available runs.""" - all_run_ids = collect_run_ids_for_target(crs_compose, target, harness, sanitizer) + all_run_ids = collect_run_ids_for_target( + crs_compose, target, harness, sanitizer, unharnessed=unharnessed + ) if not all_run_ids: print("No runs found for this target.", file=sys.stderr) return None @@ -72,7 +85,13 @@ def select_run_id_interactively( return select("Select run-id:", choices) -def resolve_run_context(args, crs_compose, target: Target) -> tuple[str, str] | None: +def resolve_run_context( + args, + crs_compose, + target: Target, + *, + unharnessed: bool = False, +) -> tuple[str, str, bool] | None: """Resolve (sanitizer, run_id) from args, returning None on failure. Shared by the artifacts and archive commands. Handles sanitizer resolution, @@ -101,36 +120,90 @@ def resolve_run_context(args, crs_compose, target: Target) -> tuple[str, str] | return None elif getattr(args, "latest", False): all_run_ids = collect_run_ids_for_target( - crs_compose, target, harness, sanitizer + crs_compose, target, harness, sanitizer, unharnessed=unharnessed ) if not all_run_ids: print("No runs found for this target.", file=sys.stderr) return None run_id = all_run_ids[0] else: - run_id = select_run_id_interactively(crs_compose, target, harness, sanitizer) + run_id = select_run_id_interactively( + crs_compose, target, harness, sanitizer, unharnessed=unharnessed + ) if run_id is None: return None - return sanitizer, run_id + scope_known = target.target_harness is not None or unharnessed + if not scope_known: + scope_known = _apply_existing_run_scope(crs_compose, target, run_id, sanitizer) + if scope_known is None: + return None + + return sanitizer, run_id, scope_known + + +def _apply_existing_run_scope( + crs_compose, target: Target, run_id: str, sanitizer: str +) -> bool | None: + """Apply the sole existing run scope to target. + + Returns False when no scope exists yet and None when multiple scopes violate + the one-scope-per-run invariant. + """ + scopes: set[str] = set() + for crs in crs_compose.crs_list: + submit_parent = crs_compose.work_dir.get_submit_dir( + crs.name, target, run_id, sanitizer, create=False + ).parent + if submit_parent.is_dir(): + scopes.update( + path.name for path in submit_parent.iterdir() if path.is_dir() + ) + + if not scopes: + return False + if len(scopes) > 1: + print( + f"Error: Run '{run_id}' has multiple harness scopes: " + + ", ".join(sorted(scopes)), + file=sys.stderr, + ) + return None + + scope = next(iter(scopes)) + target.target_harness = None if scope == UNHARNESSED else scope + return True -def handle_artifacts(args, crs_compose, target: Target) -> bool: +def handle_artifacts( + args, + crs_compose, + target: Target, + *, + source_only: bool = False, + unharnessed: bool = False, +) -> bool: """Handle the artifacts command.""" - ctx = resolve_run_context(args, crs_compose, target) + if source_only and args.build_id: + print("Error: --build-id is unavailable for source-only runs.", file=sys.stderr) + return False + + ctx = resolve_run_context(args, crs_compose, target, unharnessed=unharnessed) if ctx is None: return False - sanitizer, run_id = ctx - harness = target.target_harness + sanitizer, run_id, scope_known = ctx work_dir = crs_compose.work_dir - # build_id for BUILD_OUT_DIR - use provided, read from run, or find latest + # build_id for BUILD_OUT_DIR - use provided, read from run, or find latest. + # Source-only runs have no build outputs, so skip build_id resolution. if args.build_id: resolved_build_id = work_dir.resolve_build_id(args.build_id, sanitizer) if resolved_build_id is None: print(f"Build '{args.build_id}' not found.", file=sys.stderr) return False build_id = resolved_build_id + elif source_only: + build_id = None else: # Try to get build_id from the run directory first build_id = work_dir.read_build_id_for_run(run_id, sanitizer) @@ -140,13 +213,20 @@ def handle_artifacts(args, crs_compose, target: Target) -> bool: # Build structured result output = ArtifactsOutput(build_id=build_id, run_id=run_id, sanitizer=sanitizer) + if not scope_known: + for crs in crs_compose.crs_list: + if build_id: + build_path = work_dir.get_build_output_dir( + crs.name, target, build_id, sanitizer, create=False + ) + output.crs[crs.name] = CRSArtifacts(build=str(build_path)) + print(output.to_json()) + return True + output.meta = RunMeta.from_work_dir(work_dir, run_id, sanitizer) - if harness: - output.exchange_dir = ExchangeDir.from_work_dir( - work_dir, target, run_id, sanitizer - ) - output.run_logs = RunLogs.from_work_dir(work_dir, target, run_id, sanitizer) + output.exchange_dir = ExchangeDir.from_work_dir(work_dir, target, run_id, sanitizer) + output.run_logs = RunLogs.from_work_dir(work_dir, target, run_id, sanitizer) exchange_base = output.exchange_dir.base if output.exchange_dir else None for crs in crs_compose.crs_list: diff --git a/oss_crs/src/cli/crs_compose.py b/oss_crs/src/cli/crs_compose.py index 7a43c234..c1b4f039 100644 --- a/oss_crs/src/cli/crs_compose.py +++ b/oss_crs/src/cli/crs_compose.py @@ -48,14 +48,14 @@ def add_common_arguments(parser): ) -def add_target_arguments(parser): +def add_target_arguments(parser, *, require_fuzz_proj: bool = True): parser.add_argument( "--fuzz-proj-path", "--target-path", "--target-proj-path", dest="target_proj_path", type=Path, - required=True, + required=require_fuzz_proj, help=( "Path to target project directory " "(contains Dockerfile/build.sh; project.yaml optional). " @@ -124,14 +124,14 @@ def collect_artifact_inputs_from_args(args, specs) -> dict[str, ArtifactInput]: return artifact_inputs -def add_target_resolution_arguments(parser): +def add_target_resolution_arguments(parser, *, require_fuzz_proj: bool = True): parser.add_argument( "--fuzz-proj-path", "--target-path", "--target-proj-path", dest="target_proj_path", type=Path, - required=True, + required=require_fuzz_proj, help=( "Path to target project directory " "(contains Dockerfile/build.sh; project.yaml optional). " @@ -217,12 +217,15 @@ def add_run_command(subparsers): "run", help="Run CRSs against a target using CRS Compose file" ) add_common_arguments(run) - add_target_arguments(run) + add_target_arguments(run, require_fuzz_proj=False) run.add_argument( "--target-harness", type=str, default=None, - help="Specify the target harness to use for the run (omit for harness-gen CRSs)", + help=( + "Specify the target harness to use for the run. " + "Omit for harness generation or source-level analysis." + ), ) run.add_argument( "--timeout", @@ -272,7 +275,10 @@ def add_run_command(subparsers): "--early-exit", action="store_true", default=False, - help="Stop run when first artifact is discovered (POV for bug-finding, patch for bug-fixing CRSs)", + help=( + "Stop run when the first artifact is discovered " + "(POV, patch, or bug candidate)" + ), ) run.add_argument( "--incremental-build", @@ -296,13 +302,13 @@ def add_artifacts_command(subparsers): "artifacts", help="Show directories for run artifacts (JSON output)" ) add_common_arguments(artifacts) - add_target_resolution_arguments(artifacts) + add_target_resolution_arguments(artifacts, require_fuzz_proj=False) artifacts.add_argument( "--target-harness", type=str, required=False, default=None, - help="Specify the target harness (required for submit/fetch/shared dirs)", + help=("Specify the target harness."), ) artifacts.add_argument( "--build-id", @@ -341,13 +347,13 @@ def add_archive_command(subparsers): help="Package submitted artifacts from a run into a tarball", ) add_common_arguments(archive) - add_target_resolution_arguments(archive) + add_target_resolution_arguments(archive, require_fuzz_proj=False) archive.add_argument( "--target-harness", type=str, - required=True, + required=False, default=None, - help="Target harness name", + help="Target harness name (omit for source-only runs)", ) archive.add_argument( "--sanitizer", @@ -620,13 +626,39 @@ def add_gen_compose_command(subparsers): ) -def init_target_from_args(args) -> Target: +def _resolve_source_only(args, crs_compose) -> bool: + """Source-only iff no --target-harness, no --fuzz-proj-path, and the + composition contains no harness-generation CRSs.""" + if args.target_harness is not None or args.target_proj_path is not None: + return False + return not any(crs.config.is_harness_gen for crs in crs_compose.crs_list) + + +def init_target_from_args( + args, *, source_only: bool = False, require_source_dir: bool = False +) -> Target: target_harness = args.target_harness if hasattr(args, "target_harness") else None + target_proj_path = getattr(args, "target_proj_path", None) + target_repo_path = getattr(args, "target_repo_path", None) + if source_only: + if target_repo_path is None: + raise ValueError( + "--target-source-path is required when --target-harness is omitted" + ) + if require_source_dir and not target_repo_path.is_dir(): + raise ValueError( + f"--target-source-path must be an existing directory: {target_repo_path}" + ) + if target_proj_path is None: + target_proj_path = target_repo_path + elif target_proj_path is None: + raise ValueError("--fuzz-proj-path or --target-source-path is required") return Target( args.work_dir, - args.target_proj_path, - args.target_repo_path, + target_proj_path, + target_repo_path, target_harness, + source_only=source_only, ) @@ -899,7 +931,11 @@ def cli() -> bool | int: if not crs_compose.prepare(publish=args.publish, no_pull=args.no_pull): return False elif args.command == "build-target": - target = init_target_from_args(args) + try: + target = init_target_from_args(args) + except ValueError as exc: + print(f"Error: {exc}", file=sys.stderr) + return False build_artifact_inputs = collect_artifact_inputs_from_args( args, [spec for spec in RUN_ARTIFACT_INPUT_SPECS if spec.name == "bug-candidate"], @@ -923,7 +959,17 @@ def cli() -> bool | int: ): return False elif args.command == "run": - target = init_target_from_args(args) + source_only = _resolve_source_only(args, crs_compose) + try: + target = init_target_from_args( + args, source_only=source_only, require_source_dir=source_only + ) + except ValueError as exc: + print(f"Error: {exc}", file=sys.stderr) + return False + if source_only and args.web_ui: + print("Error: --web-ui requires --target-harness", file=sys.stderr) + return False if args.timeout is not None: crs_compose.set_deadline(time.monotonic() + args.timeout) artifact_inputs = collect_artifact_inputs_from_args( @@ -967,11 +1013,29 @@ def cli() -> bool | int: if run_rc != 0: return run_rc elif args.command == "artifacts": - target = init_target_from_args(args) - return handle_artifacts(args, crs_compose, target) + source_only = _resolve_source_only(args, crs_compose) + try: + target = init_target_from_args(args, source_only=source_only) + except ValueError as exc: + print(f"Error: {exc}", file=sys.stderr) + return False + return handle_artifacts( + args, + crs_compose, + target, + source_only=source_only, + unharnessed=args.target_harness is None, + ) elif args.command == "archive": - target = init_target_from_args(args) - return handle_archive(args, crs_compose, target) + source_only = _resolve_source_only(args, crs_compose) + try: + target = init_target_from_args(args, source_only=source_only) + except ValueError as exc: + print(f"Error: {exc}", file=sys.stderr) + return False + return handle_archive( + args, crs_compose, target, unharnessed=args.target_harness is None + ) elif args.command == "export": return handle_export(args, crs_compose) elif args.command == "check": diff --git a/oss_crs/src/config/crs.py b/oss_crs/src/config/crs.py index 6fbdd21c..70033e1e 100644 --- a/oss_crs/src/config/crs.py +++ b/oss_crs/src/config/crs.py @@ -155,8 +155,10 @@ class SupportedTarget(BaseModel): mode: Set[TargetMode] language: Set[TargetLanguage] - sanitizer: Set[TargetSanitizer] - architecture: Set[TargetArch] + sanitizer: Set[TargetSanitizer] = Field( + default_factory=lambda: set(TargetSanitizer) + ) + architecture: Set[TargetArch] = Field(default_factory=lambda: set(TargetArch)) fuzzing_engine: Set[FuzzingEngine] = Field( default_factory=lambda: set(FuzzingEngine) ) @@ -169,6 +171,7 @@ class CRSType(Enum): BUG_FINDING_TRIAGE = "bug-finding-triage" SEED_FILTER = "seed-filter" HARNESS_GEN = "harness-gen" + AUDITING = "auditing" VALID_REQUIRED_INPUT_NAMES: set[str] = { @@ -220,6 +223,10 @@ def is_seed_filter(self) -> bool: def is_harness_gen(self) -> bool: return CRSType.HARNESS_GEN in self.type + @property + def is_auditing(self) -> bool: + return CRSType.AUDITING in self.type + @field_validator("version") @classmethod def validate_version(cls, v: str) -> str: diff --git a/oss_crs/src/crs_compose.py b/oss_crs/src/crs_compose.py index c8e11aea..28bacc9b 100644 --- a/oss_crs/src/crs_compose.py +++ b/oss_crs/src/crs_compose.py @@ -19,6 +19,7 @@ ) from .llm import LLM from .crs import CRS +from .config.crs import CRSType from .ui import MultiTaskProgress, TaskResult, EarlyExitConfig from .target import Target, file_lock from .templates import renderer @@ -220,6 +221,41 @@ def __init__( ] self.deadline: Optional[float] = None + def _validate_source_only_run(self) -> TaskResult: + non_auditing = [crs.name for crs in self.crs_list if not crs.config.is_auditing] + if non_auditing: + return TaskResult( + success=False, + error=( + "Source-only runs (without --target-harness) require all " + "CRSs to be of type 'auditing'. Incompatible CRSs: " + + ", ".join(sorted(non_auditing)) + ), + ) + incompatible = [ + crs.name + for crs in self.crs_list + if any( + module.target_dependent + for module in crs.config.crs_run_phase.modules.values() + ) + ] + if incompatible: + return TaskResult( + success=False, + error=( + "Runs without --target-harness require target-independent CRS " + "modules. Set target_dependent: false for all run modules and run " + "`oss-crs prepare` first. Incompatible CRSs: " + + ", ".join(sorted(incompatible)) + ), + ) + return TaskResult(success=True) + + def is_source_only_run(self, target: Target) -> bool: + """Return whether a no-harness run should skip target builds.""" + return target.source_only + def _resolve_target_build_options( self, target: Target, @@ -1096,6 +1132,25 @@ def run( incremental_build: bool = False, web_ui: bool = False, ) -> int: + source_only = self.is_source_only_run(target) + if source_only: + if not target._has_repo: + print("Error: --target-source-path is required for source-only runs") + return 1 + if build_id is not None: + print("Error: --build-id requires --target-harness") + return 1 + if incremental_build: + print("Error: --incremental-build requires --target-harness") + return 1 + if web_ui: + print("Error: --web-ui requires --target-harness") + return 1 + source_only_check = self._validate_source_only_run() + if not source_only_check.success: + print(f"Error: {source_only_check.error}") + return 1 + resolved_options = self._resolve_target_build_options( target, sanitizer=sanitizer, @@ -1110,9 +1165,13 @@ def run( # Auto-detect cgroup-parent availability cgroup_parent, _ = check_cgroup_parent_available() - # Determine build_id: use provided, find latest, or generate new + # Determine build_id: use provided, find latest, or generate new. + # Source-only runs do not have target build outputs, but a build_id still + # gives artifact paths a stable run/build namespace. if build_id: build_id = normalize_run_id(build_id) + elif source_only: + build_id = f"source-only-{run_id}" else: # Look for latest existing build for this target/sanitizer build_id = self.get_latest_build_id(target, sanitizer) @@ -1140,10 +1199,14 @@ def run( forwarded_artifact_names=self._forwarded_artifact_names(forward_sources), ): return 1 - target.init_repo() + if not target.init_repo(): + print(f"Error: Failed to initialize target source: {target.repo_path}") + return 1 # Check if we need to build - if build_id: + if source_only: + need_build = False + elif build_id: need_build = not self.__check_target_built(target, build_id, sanitizer) else: need_build = True # No builds exist yet @@ -1181,8 +1244,12 @@ def run( print(f"Error: {snapshot_error}") return 1 - # Write build_id to run directory for later retrieval (e.g., by artifacts command) - self.work_dir.write_build_id_for_run(run_id, sanitizer, build_id) + # Source-only runs use a synthetic ID internally for APIs that still + # require one, but they have no build output to associate with the run. + if source_only: + self.work_dir.get_build_id_file(run_id, sanitizer).unlink(missing_ok=True) + else: + self.work_dir.write_build_id_for_run(run_id, sanitizer, build_id) result = self.__run( target, @@ -1196,6 +1263,7 @@ def run( early_exit=early_exit, incremental_build=incremental_build, web_ui=web_ui, + source_only=source_only, ) return result @@ -1715,6 +1783,7 @@ def __run( early_exit: bool = False, incremental_build: bool = False, web_ui: bool = False, + source_only: bool = False, ) -> int: if self.crs_compose_env.run_env == RunEnv.LOCAL: return self.__run_local( @@ -1729,11 +1798,27 @@ def __run( early_exit=early_exit, incremental_build=incremental_build, web_ui=web_ui, + source_only=source_only, ) else: print(f"TODO: Support run env {self.crs_compose_env.run_env}") return 1 + # Note: campaigns running multiple types of CRS will exit on the first artifact + # (e.g. bug-finding + bug-fixing will exit when a single PoV is found) + def _early_exit_artifact_subdirs(self) -> set[str]: + artifact_subdirs: set[str] = set() + for crs in self.crs_list: + if crs.config.is_triage or crs.config.is_seed_filter: + continue + if crs.config.is_bug_fixing: + artifact_subdirs.add("patches") + if crs.config.is_auditing: + artifact_subdirs.add("bug-candidates") + if CRSType.BUG_FINDING in crs.config.type: + artifact_subdirs.add("povs") + return artifact_subdirs + def __run_local( self, target: Target, @@ -1747,6 +1832,7 @@ def __run_local( early_exit: bool = False, incremental_build: bool = False, web_ui: bool = False, + source_only: bool = False, ) -> int: # Create cgroups if cgroup_parent mode is enabled worker_cgroup_path: Optional[Path] = None @@ -1766,9 +1852,6 @@ def __run_local( # Build early exit configuration if enabled early_exit_config: Optional[EarlyExitConfig] = None if early_exit: - # Bug-fixing CRSs watch for patches, bug-finding watch for POVs - has_bug_fixing = any(crs.config.is_bug_fixing for crs in self.crs_list) - artifact_subdir = "patches" if has_bug_fixing else "povs" # Collect SUBMIT_DIR paths for all CRSs watch_dirs: list[Path] = [ self.work_dir.get_submit_dir( @@ -1779,14 +1862,14 @@ def __run_local( and not crs.config.is_seed_filter # post-processors run until timeout ] # Also watch exchange dir when multiple CRSs (shared artifact location) - if len(self.crs_list) > 1: + if watch_dirs and len(self.crs_list) > 1: exchange_dir = self.work_dir.get_exchange_dir( target, run_id, sanitizer, create=False ) watch_dirs.append(exchange_dir) early_exit_config = EarlyExitConfig( watch_dirs=watch_dirs, - artifact_subdir=artifact_subdir, + artifact_subdirs=self._early_exit_artifact_subdirs(), ) with MultiTaskProgress( @@ -1838,6 +1921,7 @@ def __run_local( cgroup_parents=cgroup_parents, incremental_build=incremental_build, web_ui=web_ui, + source_only=source_only, ), ), ( @@ -2209,6 +2293,7 @@ def __prepare_local_running_env( cgroup_parents: Optional[dict[str, str]] = None, incremental_build: bool = False, web_ui: bool = False, + source_only: bool = False, ) -> TaskResult: docker_compose_path = tmp_docker_compose.docker_compose assert docker_compose_path is not None @@ -2259,6 +2344,7 @@ def prepare_docker_compose(progress: MultiTaskProgress) -> TaskResult: incremental_build=incremental_build, sidecar_env=sidecar_env, web_ui=web_ui, + source_only=source_only, ) for warning in warnings: progress.add_note(warning) diff --git a/oss_crs/src/env_policy.py b/oss_crs/src/env_policy.py index dcc7cc10..492ac5d1 100644 --- a/oss_crs/src/env_policy.py +++ b/oss_crs/src/env_policy.py @@ -172,19 +172,26 @@ def build_run_service_env( crs_additional_env: Mapping[str, str] | None, scope: str, harness: str | None = None, + source_only: bool = False, include_fetch_dir: bool = False, llm_api_url: str | None = None, llm_api_key: str | None = None, ) -> EnvPlan: - base_env = { - "HELPER": "True", - "RUN_FUZZER_MODE": "interactive", - **{ - k: target_env[v] for k, v in OSS_FUZZ_TARGET_ENV.items() if k != "SANITIZER" - }, - "SANITIZER": sanitizer, # override: run phase uses resolved sanitizer - "PROJECT_NAME": target_env["name"], - } + base_env = {} + if not source_only: + base_env["HELPER"] = "True" + base_env["RUN_FUZZER_MODE"] = "interactive" + # SANITIZER is excluded: its value always comes from the resolved + # sanitizer argument below, never from target_env. + base_env.update( + { + k: target_env[v] + for k, v in OSS_FUZZ_TARGET_ENV.items() + if k != "SANITIZER" + } + ) + base_env["SANITIZER"] = sanitizer + base_env["PROJECT_NAME"] = target_env["name"] # Preserve existing behavior: module env first, CRS env last. system_env = { "OSS_CRS_RUN_ENV_TYPE": run_env_type, @@ -195,17 +202,20 @@ def build_run_service_env( "OSS_CRS_RUN_ID": run_id, "OSS_CRS_CPUSET": cpuset, "OSS_CRS_MEMORY_LIMIT": memory_limit, - "OSS_CRS_PROJ_PATH": "/OSS_CRS_PROJ_PATH", - "OSS_CRS_REPO_PATH": target_env["repo_path"], - "OSS_CRS_BUILD_OUT_DIR": "/OSS_CRS_BUILD_OUT_DIR", - "OSS_CRS_REBUILD_OUT_DIR": "/OSS_CRS_REBUILD_OUT_DIR", "OSS_CRS_SUBMIT_DIR": "/OSS_CRS_SUBMIT_DIR", "OSS_CRS_SHARED_DIR": "/OSS_CRS_SHARED_DIR", "OSS_CRS_LOG_DIR": "/OSS_CRS_LOG_DIR", - "BUILDER_MODULE": "builder-sidecar", - "OSS_CRS_FUZZ_PROJ": "/OSS_CRS_FUZZ_PROJ", "OSS_CRS_TARGET_SOURCE": "/OSS_CRS_TARGET_SOURCE", } + if source_only: + system_env["OSS_CRS_REPO_PATH"] = "/OSS_CRS_TARGET_SOURCE" + else: + system_env["OSS_CRS_PROJ_PATH"] = "/OSS_CRS_PROJ_PATH" + system_env["OSS_CRS_REPO_PATH"] = target_env["repo_path"] + system_env["OSS_CRS_BUILD_OUT_DIR"] = "/OSS_CRS_BUILD_OUT_DIR" + system_env["OSS_CRS_REBUILD_OUT_DIR"] = "/OSS_CRS_REBUILD_OUT_DIR" + system_env["BUILDER_MODULE"] = "builder-sidecar" + system_env["OSS_CRS_FUZZ_PROJ"] = "/OSS_CRS_FUZZ_PROJ" if harness: system_env["OSS_CRS_TARGET_HARNESS"] = harness if include_fetch_dir: diff --git a/oss_crs/src/target.py b/oss_crs/src/target.py index 3202a7da..b2a878a6 100644 --- a/oss_crs/src/target.py +++ b/oss_crs/src/target.py @@ -156,6 +156,7 @@ def __init__( proj_path: Path, repo_path: Optional[Path], target_harness: Optional[str] = None, + source_only: bool = False, ): self.name = extract_name_from_proj_path(str(proj_path)) self.proj_path = proj_path @@ -179,6 +180,7 @@ def __init__( self.repo_hash: Optional[str] = None self.target_harness = target_harness + self.source_only = source_only def _load_project_yaml_defaults(self) -> None: """Load optional OSS-Fuzz project.yaml defaults. @@ -236,6 +238,15 @@ def get_docker_image_name(self) -> str: repo_hash = self.get_repo_hash() return f"{self.name}:{repo_hash}" + def get_workdir_target_key(self) -> str: + """Return the target identity used in build and run workdir paths.""" + if self.source_only: + path_hash = hashlib.sha256( + str(self.repo_path.resolve()).encode() + ).hexdigest()[:12] + return f"{self.name}_{path_hash}" + return self.get_docker_image_name().replace(":", "_") + @property def base_runner_image(self) -> str: """Full base-runner image reference whose OS matches the build toolchain. diff --git a/oss_crs/src/templates/renderer.py b/oss_crs/src/templates/renderer.py index c0c63abd..8f3df963 100644 --- a/oss_crs/src/templates/renderer.py +++ b/oss_crs/src/templates/renderer.py @@ -321,6 +321,7 @@ def render_run_crs_compose_docker_compose( incremental_build: bool = False, sidecar_env: dict[str, str] | None = None, web_ui: bool = False, + source_only: bool = False, ) -> tuple[str, list[str]]: template_path = CUR_DIR / "run-crs-compose.docker-compose.yaml.j2" compose_env = crs_compose.crs_compose_env @@ -400,7 +401,8 @@ def render_run_crs_compose_docker_compose( "bug_finding_ensemble": bug_finding_ensemble, "bug_fix_ensemble": bug_fix_ensemble, "cgroup_parents": cgroup_parents, # Dict mapping CRS name to cgroup_parent path - "fuzz_proj_path": str(target.proj_path.resolve()), + "source_only": source_only, + "fuzz_proj_path": None if source_only else str(target.proj_path.resolve()), "target_source_path": str(target.repo_path.resolve()) if target._has_repo else str( @@ -469,6 +471,7 @@ def render_run_crs_compose_docker_compose( module_additional_env=module_config.additional_env, crs_additional_env=resource.additional_env, harness=target_env.get("harness"), + source_only=source_only, include_fetch_dir=bool(fetch_dir), llm_api_url=llm_url, llm_api_key=llm_key, diff --git a/oss_crs/src/templates/run-crs-compose.docker-compose.yaml.j2 b/oss_crs/src/templates/run-crs-compose.docker-compose.yaml.j2 index 210148b7..05d6617b 100644 --- a/oss_crs/src/templates/run-crs-compose.docker-compose.yaml.j2 +++ b/oss_crs/src/templates/run-crs-compose.docker-compose.yaml.j2 @@ -46,8 +46,10 @@ services: mem_limit: "{{ crs.resource.memory }}" {%- endif %} volumes: +{%- if not source_only %} - {{ work_dir.get_build_output_dir(crs.name, target, build_id, sanitizer) }}:/OSS_CRS_BUILD_OUT_DIR:ro - {{ rebuild_out_dir }}:/OSS_CRS_REBUILD_OUT_DIR:ro +{%- endif %} - {{ work_dir.get_submit_dir(crs.name, target, run_id, sanitizer) }}:/OSS_CRS_SUBMIT_DIR:rw - {{ work_dir.get_shared_dir(crs.name, target, run_id, sanitizer) }}:/OSS_CRS_SHARED_DIR:rw - {{ work_dir.get_log_dir(crs.name, target, run_id, sanitizer) }}:/OSS_CRS_LOG_DIR:rw @@ -58,7 +60,9 @@ services: {%- elif fetch_dir %} - {{ fetch_dir }}:/OSS_CRS_FETCH_DIR:ro {%- endif %} +{%- if fuzz_proj_path %} - {{ fuzz_proj_path }}:/OSS_CRS_FUZZ_PROJ:ro +{%- endif %} - {{ target_source_path }}:/OSS_CRS_TARGET_SOURCE:ro {%- if bug_fix_ensemble and not crs.config.is_bug_fixing_ensemble %} attach: false @@ -229,6 +233,7 @@ services: ########################################### # OSS-CRS-PUBLISHER (pushes metrics to standalone web-ui) ########################################### +{%- if not source_only %} {%- if web_ui %} oss-crs-publisher: build: @@ -362,6 +367,7 @@ services: {%- for crs in crs_list %} - runner-sidecar.{{ crs.name }} {%- endfor %} +{%- endif %} {%- if llm_context is defined %} secrets: {%- if llm_context.mode == "internal" %} diff --git a/oss_crs/src/ui.py b/oss_crs/src/ui.py index 63da1f40..d742f477 100644 --- a/oss_crs/src/ui.py +++ b/oss_crs/src/ui.py @@ -28,7 +28,7 @@ class EarlyExitConfig: """Configuration for early exit artifact monitoring.""" watch_dirs: list[Path] # SUBMIT_DIR paths to monitor - artifact_subdir: str # "povs" or "patches" + artifact_subdirs: set[str] poll_interval: float = 2.0 # seconds between checks @@ -140,15 +140,16 @@ def _check_early_exit(self) -> bool: if not self.early_exit_config: return False for watch_dir in self.early_exit_config.watch_dirs: - artifact_dir = watch_dir / self.early_exit_config.artifact_subdir - if artifact_dir.exists(): - files = [ - f - for f in artifact_dir.iterdir() - if f.is_file() and not f.name.startswith(".") - ] - if files: - return True + for artifact_subdir in self.early_exit_config.artifact_subdirs: + artifact_dir = watch_dir / artifact_subdir + if artifact_dir.exists(): + files = [ + f + for f in artifact_dir.iterdir() + if f.is_file() and not f.name.startswith(".") + ] + if files: + return True return False def _start_early_exit_monitor(self) -> threading.Thread: @@ -795,10 +796,12 @@ def show_run_result(self, crs_results: list[dict]) -> None: pov_dir = submit_dir / "povs" seed_dir = submit_dir / "seeds" patch_dir = submit_dir / "patches" + bug_candidate_dir = submit_dir / "bug-candidates" pov_count = _count_files(pov_dir) seed_count = _count_files(seed_dir) patch_count = _count_files(patch_dir) + bug_candidate_count = _count_files(bug_candidate_dir) output.append(f"{entry['name']}:\n", style="bold cyan") output.append(" Patches: ", style="bold") @@ -813,6 +816,10 @@ def show_run_result(self, crs_results: list[dict]) -> None: output.append(f"{seed_count}\n", style="green") if seed_count > 0: output.append(f" {seed_dir}\n", style="dim") + output.append(" Bug Candidates: ", style="bold") + output.append(f"{bug_candidate_count}\n", style="magenta") + if bug_candidate_count > 0: + output.append(f" {bug_candidate_dir}\n", style="dim") if i < len(crs_results) - 1: output.append("\n") diff --git a/oss_crs/src/workdir.py b/oss_crs/src/workdir.py index 72113979..1ba61a9d 100644 --- a/oss_crs/src/workdir.py +++ b/oss_crs/src/workdir.py @@ -51,6 +51,8 @@ class RunEntry: class WorkDir: """Centralized path management for CRS Compose work directories.""" + NO_HARNESS_SCOPE = UNHARNESSED + def __init__(self, base_path: Path): """Initialize WorkDir with a base path. @@ -63,7 +65,7 @@ def __init__(self, base_path: Path): @staticmethod def _get_target_key(target: Target) -> str: """Compute target_key from a Target.""" - return target.get_docker_image_name().replace(":", "_") + return target.get_workdir_target_key() @staticmethod def count_data_files(dir_path: Path, recursive: bool = False) -> int: @@ -82,6 +84,11 @@ def count_data_files(dir_path: Path, recursive: bool = False) -> int: entries = dir_path.rglob("*") if recursive else dir_path.iterdir() return sum(1 for f in entries if f.is_file() and not f.name.startswith(".")) + @classmethod + def _get_harness_scope(cls, target: Target) -> str: + """Return the run artifact scope for harnessed or source-level runs.""" + return target.target_harness or cls.NO_HARNESS_SCOPE + # ------------------------------------------------------------------------- # Base directory helpers # ------------------------------------------------------------------------- @@ -216,11 +223,15 @@ def get_run_logs_dir( ) -> Path: """Get run-scoped logs directory (outside EXCHANGE_DIR/SUBMIT_DIR mounts). - Structure: /runs//logs/// + Structure: /runs//logs/// """ - harness = target.target_harness or UNHARNESSED target_key = self._get_target_key(target) - path = self.get_run_dir(run_id, sanitizer) / "logs" / target_key / harness + path = ( + self.get_run_dir(run_id, sanitizer) + / "logs" + / target_key + / self._get_harness_scope(target) + ) if create: path.mkdir(parents=True, exist_ok=True) return path @@ -299,13 +310,12 @@ def get_submit_dir( ) -> Path: """Get the SUBMIT_DIR for a CRS run. - Structure: /runs//crs///SUBMIT_DIR// + Structure: /runs//crs///SUBMIT_DIR// """ - harness = target.target_harness or UNHARNESSED path = ( self.get_crs_run_dir(crs_name, target, run_id, sanitizer) / "SUBMIT_DIR" - / harness + / self._get_harness_scope(target) ) if create: path.mkdir(parents=True, exist_ok=True) @@ -321,13 +331,12 @@ def get_shared_dir( ) -> Path: """Get the SHARED_DIR for a CRS run. - Structure: /runs//crs///SHARED_DIR// + Structure: /runs//crs///SHARED_DIR// """ - harness = target.target_harness or UNHARNESSED path = ( self.get_crs_run_dir(crs_name, target, run_id, sanitizer) / "SHARED_DIR" - / harness + / self._get_harness_scope(target) ) if create: path.mkdir(parents=True, exist_ok=True) @@ -343,13 +352,12 @@ def get_log_dir( ) -> Path: """Get the LOG_DIR for a CRS run (agent/internal logs). - Structure: /runs//crs///LOG_DIR// + Structure: /runs//crs///LOG_DIR// """ - harness = target.target_harness or UNHARNESSED path = ( self.get_crs_run_dir(crs_name, target, run_id, sanitizer) / "LOG_DIR" - / harness + / self._get_harness_scope(target) ) if create: path.mkdir(parents=True, exist_ok=True) @@ -368,12 +376,14 @@ def get_exchange_dir( ) -> Path: """Get the shared EXCHANGE_DIR (shared across all CRSs). - Structure: /runs//EXCHANGE_DIR/// + Structure: /runs//EXCHANGE_DIR/// """ - harness = target.target_harness or UNHARNESSED target_key = self._get_target_key(target) path = ( - self.get_run_dir(run_id, sanitizer) / "EXCHANGE_DIR" / target_key / harness + self.get_run_dir(run_id, sanitizer) + / "EXCHANGE_DIR" + / target_key + / self._get_harness_scope(target) ) if create: path.mkdir(parents=True, exist_ok=True) @@ -393,13 +403,12 @@ def get_processed_exchange_dir( Structure: /runs//PROCESSED_EXCHANGE_DIR/// """ - harness = target.target_harness or UNHARNESSED target_key = self._get_target_key(target) path = ( self.get_run_dir(run_id, sanitizer) / "PROCESSED_EXCHANGE_DIR" / target_key - / harness + / self._get_harness_scope(target) ) if create: path.mkdir(parents=True, exist_ok=True) diff --git a/oss_crs/tests/unit/config/test_crs.py b/oss_crs/tests/unit/config/test_crs.py index b0fd5b51..499fc51f 100644 --- a/oss_crs/tests/unit/config/test_crs.py +++ b/oss_crs/tests/unit/config/test_crs.py @@ -10,6 +10,7 @@ CRSRunPhaseModule, _validate_dockerfile_value, ) +from oss_crs.src.config.target import TargetArch, TargetSanitizer class TestDockerfileValidation: @@ -66,7 +67,7 @@ class TestCRSRunPhaseModule: def test_module_requires_dockerfile(self): """A module must specify a dockerfile.""" with pytest.raises(ValidationError): - CRSRunPhaseModule() + CRSRunPhaseModule() # type: ignore[reportCallIssue] def test_module_defaults_to_target_dependent(self): """Without target_dependent the image is built during build-target.""" @@ -244,6 +245,76 @@ def test_seed_filter_is_not_triage(self): config = CRSConfig.from_dict(_minimal_crs_config(type=["seed-filter"])) assert config.is_triage is False + def test_auditing(self): + config = CRSConfig.from_dict(_minimal_crs_config(type=["auditing"])) + + assert config.is_auditing is True + assert config.is_bug_fixing is False + assert config.is_bug_fixing_ensemble is False + assert config.is_triage is False + assert config.is_seed_filter is False + + +class TestSupportedTargetDefaults: + """Tests for SupportedTarget optional fields.""" + + def test_sanitizer_defaults_to_all(self): + """Omitting sanitizer defaults to all sanitizers.""" + config = CRSConfig.from_dict( + _minimal_crs_config( + supported_target={ + "mode": ["full"], + "language": ["c"], + } + ) + ) + assert config.supported_target.sanitizer == { + TargetSanitizer.ASAN, + TargetSanitizer.MSAN, + TargetSanitizer.UBSAN, + } + + def test_architecture_defaults_to_all(self): + """Omitting architecture defaults to all architectures.""" + config = CRSConfig.from_dict( + _minimal_crs_config( + supported_target={ + "mode": ["full"], + "language": ["c"], + } + ) + ) + assert config.supported_target.architecture == { + TargetArch.X86_64, + TargetArch.I386, + } + + def test_sanitizer_explicit_overrides_default(self): + """Explicit sanitizer value overrides the default.""" + config = CRSConfig.from_dict( + _minimal_crs_config( + supported_target={ + "mode": ["full"], + "language": ["c"], + "sanitizer": ["address"], + } + ) + ) + assert config.supported_target.sanitizer == {TargetSanitizer.ASAN} + + def test_architecture_explicit_overrides_default(self): + """Explicit architecture value overrides the default.""" + config = CRSConfig.from_dict( + _minimal_crs_config( + supported_target={ + "mode": ["full"], + "language": ["c"], + "architecture": ["x86_64"], + } + ) + ) + assert config.supported_target.architecture == {TargetArch.X86_64} + class TestSidecarDeploymentConditions: """Tests for exchange/lifecycle sidecar deployment logic. diff --git a/oss_crs/tests/unit/test_cli_archive.py b/oss_crs/tests/unit/test_cli_archive.py index 6ae101f5..2a3d24ab 100644 --- a/oss_crs/tests/unit/test_cli_archive.py +++ b/oss_crs/tests/unit/test_cli_archive.py @@ -142,7 +142,7 @@ def test_latest_picks_most_recent_run(tmp_path: Path, monkeypatch) -> None: ctx = resolve_run_context(args, compose, target) assert ctx is not None - _, run_id = ctx + _, run_id, _ = ctx assert run_id == "1700000002ab" @@ -173,7 +173,7 @@ def test_run_id_takes_precedence_over_latest(tmp_path: Path, monkeypatch) -> Non ctx = resolve_run_context(args, compose, target) assert ctx is not None - _, run_id = ctx + _, run_id, _ = ctx # --latest should be ignored; the explicit run_id wins assert run_id == explicit_id assert run_id != "1700000002ab" @@ -401,3 +401,76 @@ def test_archive_deduplicates_colliding_arcnames(tmp_path: Path) -> None: # Both files present, one with a suffix assert "povs/crash" in members assert "povs/crash.1" in members + + +# --------------------------------------------------------------------------- +# archive: source-only runs (no harness) +# --------------------------------------------------------------------------- + + +def _make_source_only_target() -> SimpleNamespace: + return SimpleNamespace(target_harness=None) + + +def test_archive_source_only_collects_bug_candidates(tmp_path: Path) -> None: + run_id = "1700000001ab" + compose = _make_compose(tmp_path, [_make_crs("crs-a")]) + work_dir = compose.work_dir + + _write_file( + work_dir.get_submit_dir("crs-a", _make_source_only_target(), run_id, "address") + / "bug-candidates" + / "bug-001" + ) + + out = tmp_path / "results.tar.gz" + args = _make_args(run_id=run_id, out=str(out)) + ok = handle_archive(args, compose, _make_source_only_target(), unharnessed=True) + + assert ok is True + members = _tar_members(out) + assert "bug-candidates/bug-001" in members + + +def test_archive_source_only_all_includes_exchange_and_logs(tmp_path: Path) -> None: + run_id = "1700000001ab" + compose = _make_compose(tmp_path, [_make_crs("crs-a")]) + work_dir = compose.work_dir + target = _make_source_only_target() + + _write_file( + work_dir.get_submit_dir("crs-a", target, run_id, "address") + / "bug-candidates" + / "bug-001" + ) + _write_file(work_dir.get_exchange_dir(target, run_id, "address") / "extra.bin") + _write_file(work_dir.get_run_logs_dir(target, run_id, "address") / "compose.log") + + out = tmp_path / "results.tar.gz" + args = _make_args(run_id=run_id, out=str(out), include_all=True) + ok = handle_archive(args, compose, target, unharnessed=True) + + assert ok is True + members = _tar_members(out) + assert "bug-candidates/bug-001" in members + assert "exchange/extra.bin" in members + assert "logs/compose.log" in members + + +def test_archive_collects_generated_harness_tree(tmp_path: Path) -> None: + run_id = "1700000001ab" + compose = _make_compose(tmp_path, [_make_crs("crs-harness-gen")]) + target = _make_source_only_target() + submit_dir = compose.work_dir.get_submit_dir( + "crs-harness-gen", target, run_id, "address" + ) + _write_file(submit_dir / "harnesses" / "generated" / "fuzz-proj" / "project.yaml") + _write_file(submit_dir / "harnesses" / "generated" / "target-source" / "source.c") + + out = tmp_path / "results.tar.gz" + args = _make_args(run_id=run_id, out=str(out)) + + assert handle_archive(args, compose, target, unharnessed=True) is True + members = _tar_members(out) + assert "harnesses/generated/fuzz-proj/project.yaml" in members + assert "harnesses/generated/target-source/source.c" in members diff --git a/oss_crs/tests/unit/test_cli_artifacts_prerun.py b/oss_crs/tests/unit/test_cli_artifacts_prerun.py index 56a9688d..09ecf5d7 100644 --- a/oss_crs/tests/unit/test_cli_artifacts_prerun.py +++ b/oss_crs/tests/unit/test_cli_artifacts_prerun.py @@ -1,10 +1,18 @@ # SPDX-License-Identifier: MIT """Tests for artifacts command pre-run path resolution behavior.""" +import argparse import json +from pathlib import Path from types import SimpleNamespace -from oss_crs.src.cli.artifacts import handle_artifacts +from oss_crs.src.constants import UNHARNESSED +from oss_crs.src.cli.artifacts import ( + collect_run_ids_for_target, + handle_artifacts, + resolve_run_context, +) +from oss_crs.src.cli.crs_compose import add_artifacts_command, add_archive_command from oss_crs.src.utils import normalize_run_id @@ -17,13 +25,28 @@ def get_docker_image_name(self) -> str: class _FakeWorkDir: + NO_HARNESS_SCOPE = UNHARNESSED + def __init__(self, tmp_path, resolved_run_id: str | None): self._tmp = tmp_path self._resolved_run_id = resolved_run_id + def _harness_scope(self, target) -> str: + return target.target_harness or self.NO_HARNESS_SCOPE + def resolve_run_id(self, _raw: str, _sanitizer: str) -> str | None: return self._resolved_run_id + def iter_runs(self, sanitizer: str): + runs_dir = self._tmp / sanitizer / "runs" + if not runs_dir.exists(): + return [] + return [ + SimpleNamespace(run_id=path.name) + for path in runs_dir.iterdir() + if path.is_dir() + ] + def resolve_build_id(self, _raw: str, _sanitizer: str) -> str | None: return None @@ -44,7 +67,7 @@ def get_exchange_dir( / run_id / "EXCHANGE_DIR" / target.get_docker_image_name().replace(":", "_") - / target.target_harness + / self._harness_scope(target) ) def get_run_logs_dir( @@ -58,7 +81,7 @@ def get_run_logs_dir( / run_id / "logs" / target.get_docker_image_name().replace(":", "_") - / target.target_harness + / self._harness_scope(target) ) def get_build_output_dir( @@ -101,7 +124,7 @@ def get_submit_dir( / crs_name / target.get_docker_image_name().replace(":", "_") / "SUBMIT_DIR" - / target.target_harness + / self._harness_scope(target) ) def get_shared_dir( @@ -123,7 +146,7 @@ def get_shared_dir( / crs_name / target.get_docker_image_name().replace(":", "_") / "SHARED_DIR" - / target.target_harness + / self._harness_scope(target) ) def get_log_dir( @@ -145,7 +168,7 @@ def get_log_dir( / crs_name / target.get_docker_image_name().replace(":", "_") / "LOG_DIR" - / target.target_harness + / self._harness_scope(target) ) def get_sidecar_metrics_file( @@ -224,6 +247,110 @@ def test_artifacts_resolves_default_sanitizer_from_compose(tmp_path, capsys) -> assert '"sanitizer": "memory"' in out +def test_artifacts_reports_no_harness_scope(tmp_path, capsys) -> None: + compose = _make_compose(tmp_path, resolved_run_id=None) + args = _make_args("No Harness Run") + target = _FakeTarget(None) + + ok = handle_artifacts(args, compose, target, source_only=True, unharnessed=True) + assert ok is True + + out = capsys.readouterr().out + assert UNHARNESSED in out + assert '"pov":' in out + assert '"exchange_dir":' in out + + +def test_source_only_artifacts_ignore_recorded_build_id(tmp_path, capsys) -> None: + compose = _make_compose(tmp_path, resolved_run_id="run-1") + compose.work_dir.read_build_id_for_run = lambda *_args: "source-only-run-1" + args = _make_args("run-1") + target = _FakeTarget(None) + + assert handle_artifacts(args, compose, target, source_only=True, unharnessed=True) + output = json.loads(capsys.readouterr().out) + assert "build_id" not in output + assert "build" not in output["crs"]["crs-a"] + + +def test_no_harness_discovery_matches_arbitrary_harness_scope(tmp_path) -> None: + compose = _make_compose(tmp_path, resolved_run_id=None) + target = _FakeTarget(None) + run_id = "run-1700000000" + submit_parent = compose.work_dir.get_submit_dir( + "crs-a", target, run_id, "address", create=False + ).parent + (submit_parent / "generated-harness" / "povs").mkdir(parents=True) + + assert collect_run_ids_for_target(compose, target, None, "address") == [run_id] + + +def test_no_harness_discovery_matches_unharnessed_harness_generator(tmp_path) -> None: + compose = _make_compose(tmp_path, resolved_run_id=None) + target = _FakeTarget(None) + run_id = "run-1700000001" + harnesses_dir = ( + compose.work_dir.get_submit_dir( + "crs-a", target, run_id, "address", create=False + ) + / "harnesses" + / "generated-harness" + ) + harnesses_dir.mkdir(parents=True) + + assert collect_run_ids_for_target(compose, target, None, "address") == [run_id] + + +def test_explicit_harness_discovery_only_matches_requested_scope(tmp_path) -> None: + compose = _make_compose(tmp_path, resolved_run_id=None) + target = _FakeTarget(None) + matching_run = "run-1700000002" + other_run = "run-1700000003" + + target.target_harness = "fuzz-a" + compose.work_dir.get_submit_dir( + "crs-a", target, matching_run, "address", create=False + ).mkdir(parents=True) + target.target_harness = "fuzz-b" + compose.work_dir.get_submit_dir( + "crs-a", target, other_run, "address", create=False + ).mkdir(parents=True) + target.target_harness = None + + assert collect_run_ids_for_target(compose, target, "fuzz-a", "address") == [ + matching_run + ] + assert target.target_harness is None + + +def test_existing_no_harness_query_applies_sole_scope(tmp_path) -> None: + run_id = "run-1700000004" + compose = _make_compose(tmp_path, resolved_run_id=run_id) + target = _FakeTarget(None) + submit_parent = compose.work_dir.get_submit_dir( + "crs-a", target, run_id, "address", create=False + ).parent + (submit_parent / "fuzz-a").mkdir(parents=True) + args = _make_args(run_id) + + assert resolve_run_context(args, compose, target) == ("address", run_id, True) + assert target.target_harness == "fuzz-a" + + +def test_existing_no_harness_query_rejects_multiple_scopes(tmp_path, capsys) -> None: + run_id = "run-1700000005" + compose = _make_compose(tmp_path, resolved_run_id=run_id) + target = _FakeTarget(None) + submit_parent = compose.work_dir.get_submit_dir( + "crs-a", target, run_id, "address", create=False + ).parent + (submit_parent / "fuzz-a").mkdir(parents=True) + (submit_parent / "fuzz-b").mkdir(parents=True) + + assert resolve_run_context(_make_args(run_id), compose, target) is None + assert "multiple harness scopes" in capsys.readouterr().err + + def test_artifacts_includes_meta_stats_when_meta_json_exists(tmp_path, capsys) -> None: run_id = "existing-run-id" sanitizer = "address" @@ -320,3 +447,74 @@ def test_artifacts_omits_sidecar_metrics_when_file_absent(tmp_path, capsys) -> N out = json.loads(capsys.readouterr().out) assert "sidecar_metrics" not in out["crs"]["crs-a"] + + +# --------------------------------------------------------------------------- +# Argparse-level tests for source-only (no --fuzz-proj-path) support +# --------------------------------------------------------------------------- + + +def _make_artifacts_parser(): + parser = argparse.ArgumentParser() + sub = parser.add_subparsers(dest="command") + add_artifacts_command(sub) + return parser + + +def _make_archive_parser(): + parser = argparse.ArgumentParser() + sub = parser.add_subparsers(dest="command") + add_archive_command(sub) + return parser + + +def test_artifacts_accepts_target_source_path_without_fuzz_proj_path(): + """artifacts --target-source-path should work without --fuzz-proj-path.""" + parser = _make_artifacts_parser() + args = parser.parse_args( + [ + "artifacts", + "--compose-file", + "c.yaml", + "--target-source-path", + "/tmp/src", + ] + ) + assert args.target_repo_path == Path("/tmp/src") + assert args.target_proj_path is None + + +def test_archive_accepts_target_source_path_without_fuzz_proj_path(): + """archive --target-source-path should work without --fuzz-proj-path.""" + parser = _make_archive_parser() + args = parser.parse_args( + [ + "archive", + "--compose-file", + "c.yaml", + "--target-source-path", + "/tmp/src", + "--out", + "results.tar.gz", + ] + ) + assert args.target_repo_path == Path("/tmp/src") + assert args.target_proj_path is None + assert args.out == "results.tar.gz" + + +def test_archive_target_harness_is_optional(): + """archive --target-harness should be optional (source-only support).""" + parser = _make_archive_parser() + args = parser.parse_args( + [ + "archive", + "--compose-file", + "c.yaml", + "--target-source-path", + "/tmp/src", + "--out", + "results.tar.gz", + ] + ) + assert args.target_harness is None diff --git a/oss_crs/tests/unit/test_early_exit.py b/oss_crs/tests/unit/test_early_exit.py index 8b3bdce8..53e53806 100644 --- a/oss_crs/tests/unit/test_early_exit.py +++ b/oss_crs/tests/unit/test_early_exit.py @@ -3,8 +3,11 @@ import time from pathlib import Path +from types import SimpleNamespace +from oss_crs.src.config.crs import CRSType +from oss_crs.src.crs_compose import CRSCompose from oss_crs.src.ui import EarlyExitConfig, MultiTaskProgress @@ -66,17 +69,17 @@ def test_basic_instantiation(self, tmp_path: Path): """EarlyExitConfig can be instantiated with required fields.""" config = EarlyExitConfig( watch_dirs=[tmp_path / "submit1", tmp_path / "submit2"], - artifact_subdir="povs", + artifact_subdirs={"povs"}, ) assert len(config.watch_dirs) == 2 - assert config.artifact_subdir == "povs" + assert config.artifact_subdirs == {"povs"} assert config.poll_interval == 2.0 # default def test_custom_poll_interval(self, tmp_path: Path): """Poll interval can be customized.""" config = EarlyExitConfig( watch_dirs=[tmp_path], - artifact_subdir="patches", + artifact_subdirs={"patches"}, poll_interval=0.5, ) assert config.poll_interval == 0.5 @@ -94,7 +97,7 @@ def test_returns_false_when_dir_not_exists(self, tmp_path: Path): """Returns False when watch directory doesn't exist.""" config = EarlyExitConfig( watch_dirs=[tmp_path / "nonexistent"], - artifact_subdir="povs", + artifact_subdirs={"povs"}, ) progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) assert progress._check_early_exit() is False @@ -107,7 +110,7 @@ def test_returns_false_when_artifact_dir_empty(self, tmp_path: Path): config = EarlyExitConfig( watch_dirs=[submit_dir], - artifact_subdir="povs", + artifact_subdirs={"povs"}, ) progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) assert progress._check_early_exit() is False @@ -122,7 +125,7 @@ def test_returns_true_when_artifact_exists(self, tmp_path: Path): config = EarlyExitConfig( watch_dirs=[submit_dir], - artifact_subdir="povs", + artifact_subdirs={"povs"}, ) progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) assert progress._check_early_exit() is True @@ -137,7 +140,7 @@ def test_ignores_hidden_files(self, tmp_path: Path): config = EarlyExitConfig( watch_dirs=[submit_dir], - artifact_subdir="patches", + artifact_subdirs={"patches"}, ) progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) assert progress._check_early_exit() is False @@ -155,7 +158,7 @@ def test_checks_multiple_watch_dirs(self, tmp_path: Path): config = EarlyExitConfig( watch_dirs=[submit1, submit2], - artifact_subdir="povs", + artifact_subdirs={"povs"}, ) progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) assert progress._check_early_exit() is True @@ -170,11 +173,71 @@ def test_patches_subdir_for_bug_fixing(self, tmp_path: Path): config = EarlyExitConfig( watch_dirs=[submit_dir], - artifact_subdir="patches", + artifact_subdirs={"patches"}, ) progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) assert progress._check_early_exit() is True + def test_checks_multiple_artifact_subdirs(self, tmp_path: Path): + submit_dir = tmp_path / "submit" + bug_candidates_dir = submit_dir / "bug-candidates" + bug_candidates_dir.mkdir(parents=True) + (bug_candidates_dir / "report.md").write_text("bug report") + + config = EarlyExitConfig( + watch_dirs=[submit_dir], + artifact_subdirs={"povs", "patches", "bug-candidates"}, + ) + progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) + + assert progress._check_early_exit() is True + + +class TestEarlyExitArtifactTypes: + @staticmethod + def _crs(*types: CRSType) -> SimpleNamespace: + type_set = set(types) + return SimpleNamespace( + config=SimpleNamespace( + type=type_set, + is_bug_fixing=bool( + type_set & {CRSType.BUG_FIXING, CRSType.BUG_FIXING_ENSEMBLE} + ), + is_auditing=CRSType.AUDITING in type_set, + is_triage=CRSType.BUG_FINDING_TRIAGE in type_set, + is_seed_filter=CRSType.SEED_FILTER in type_set, + ) + ) + + def test_auditing_watches_bug_candidates(self): + compose = CRSCompose.__new__(CRSCompose) + compose.crs_list = [self._crs(CRSType.AUDITING)] + + assert compose._early_exit_artifact_subdirs() == {"bug-candidates"} + + def test_mixed_producers_watch_each_output_type(self): + compose = CRSCompose.__new__(CRSCompose) + compose.crs_list = [ + self._crs(CRSType.BUG_FINDING), + self._crs(CRSType.BUG_FIXING), + self._crs(CRSType.AUDITING), + ] + + assert compose._early_exit_artifact_subdirs() == { + "povs", + "patches", + "bug-candidates", + } + + def test_post_processors_do_not_add_early_exit_types(self): + compose = CRSCompose.__new__(CRSCompose) + compose.crs_list = [ + self._crs(CRSType.BUG_FINDING_TRIAGE), + self._crs(CRSType.SEED_FILTER), + ] + + assert compose._early_exit_artifact_subdirs() == set() + class TestEarlyExitMonitor: """Tests for the early exit monitoring thread.""" @@ -188,7 +251,7 @@ def test_monitor_sets_event_when_artifact_appears(self, tmp_path: Path): config = EarlyExitConfig( watch_dirs=[submit_dir], - artifact_subdir="povs", + artifact_subdirs={"povs"}, poll_interval=0.1, # Fast polling for test ) progress = MultiTaskProgress(tasks=[], title="Test", early_exit_config=config) diff --git a/oss_crs/tests/unit/test_env_policy.py b/oss_crs/tests/unit/test_env_policy.py index 7216469a..1435ff04 100644 --- a/oss_crs/tests/unit/test_env_policy.py +++ b/oss_crs/tests/unit/test_env_policy.py @@ -142,6 +142,71 @@ def test_run_env_always_includes_fuzz_proj() -> None: assert plan.effective_env["OSS_CRS_FUZZ_PROJ"] == "/OSS_CRS_FUZZ_PROJ" +def test_source_only_run_env_omits_build_and_fuzz_env() -> None: + plan = build_run_service_env( + target_env={ + "engine": "libfuzzer", + "architecture": "x86_64", + "name": "proj", + "language": "c", + "repo_path": "/repo", + }, + sanitizer="address", + run_env_type="local", + crs_name="crs-a", + module_name="auditor", + run_id="r1", + cpuset="0-1", + memory_limit="2G", + module_additional_env=None, + crs_additional_env=None, + source_only=True, + scope="test:source-only-run", + ) + + assert "OSS_CRS_BUILD_OUT_DIR" not in plan.effective_env + assert "OSS_CRS_REBUILD_OUT_DIR" not in plan.effective_env + assert "OSS_CRS_FUZZ_PROJ" not in plan.effective_env + assert "FUZZING_LANGUAGE" not in plan.effective_env + assert "BUILDER_MODULE" not in plan.effective_env + assert "OSS_CRS_PROJ_PATH" not in plan.effective_env + assert "SANITIZER" not in plan.effective_env + assert "ARCHITECTURE" not in plan.effective_env + assert "FUZZING_ENGINE" not in plan.effective_env + assert "HELPER" not in plan.effective_env + assert "RUN_FUZZER_MODE" not in plan.effective_env + assert plan.effective_env["OSS_CRS_REPO_PATH"] == "/OSS_CRS_TARGET_SOURCE" + + +def test_harnessed_run_env_includes_oss_fuzz_runtime_vars() -> None: + """Harnessed runs keep HELPER, RUN_FUZZER_MODE, and all target env vars.""" + plan = build_run_service_env( + target_env={ + "engine": "libfuzzer", + "architecture": "x86_64", + "name": "proj", + "language": "c", + "repo_path": "/repo", + }, + sanitizer="address", + run_env_type="local", + crs_name="crs-a", + module_name="patcher", + run_id="r1", + cpuset="0-1", + memory_limit="2G", + module_additional_env=None, + crs_additional_env=None, + scope="test:harnessed-run", + ) + assert plan.effective_env["HELPER"] == "True" + assert plan.effective_env["RUN_FUZZER_MODE"] == "interactive" + assert plan.effective_env["FUZZING_ENGINE"] == "libfuzzer" + assert plan.effective_env["SANITIZER"] == "address" + assert plan.effective_env["ARCHITECTURE"] == "x86_64" + assert plan.effective_env["FUZZING_LANGUAGE"] == "c" + + def test_user_cannot_override_reserved_source_env_vars() -> None: """User-provided OSS_CRS_FUZZ_PROJ and OSS_CRS_TARGET_SOURCE are superseded by system values.""" plan = build_target_builder_env( diff --git a/oss_crs/tests/unit/test_renderer_run_compose.py b/oss_crs/tests/unit/test_renderer_run_compose.py index 4af33ece..67377be3 100644 --- a/oss_crs/tests/unit/test_renderer_run_compose.py +++ b/oss_crs/tests/unit/test_renderer_run_compose.py @@ -412,6 +412,57 @@ def _make_bug_finding_crs(tmp_path: Path, name: str) -> SimpleNamespace: ) +def _make_auditing_crs(tmp_path: Path, name: str) -> SimpleNamespace: + from oss_crs.src.config.crs import CRSType + + module_config = SimpleNamespace( + dockerfile="auditor.Dockerfile", + target_dependent=False, + additional_env={}, + ) + config = SimpleNamespace( + version="1.0", + type=[CRSType.AUDITING], + is_bug_fixing=False, + is_bug_fixing_ensemble=False, + is_triage=False, + is_seed_filter=False, + is_auditing=True, + crs_run_phase=SimpleNamespace(modules={"auditor": module_config}), + target_build_phase=None, + ) + return SimpleNamespace( + name=name, + crs_path=tmp_path, + resource=SimpleNamespace( + cpuset="2-7", memory="8G", additional_env={}, llm_budget=1 + ), + config=config, + ) + + +def test_auditing_is_regular_source_to_bug_candidate_producer( + monkeypatch, tmp_path: Path +) -> None: + _patch_renderer(monkeypatch) + auditing_crs = _make_auditing_crs(tmp_path, "crs-auditor") + crs_compose = _make_crs_compose(tmp_path, [auditing_crs]) + target = _make_target(tmp_path) + + rendered, warnings = _render(crs_compose, target, tmp_path) + + assert warnings == [] + services = yaml.safe_load(rendered)["services"] + auditor = services["crs-auditor_auditor"] + assert any("/OSS_CRS_TARGET_SOURCE:ro" in mount for mount in auditor["volumes"]) + assert any("/OSS_CRS_SUBMIT_DIR:rw" in mount for mount in auditor["volumes"]) + assert any( + "/submit/crs-auditor:ro" in mount + for mount in services["oss-crs-exchange"]["volumes"] + ) + assert "oss-crs-processed-exchange" not in services + + def test_compose03_exchange_sidecar_present_for_triage_only_compose( monkeypatch, tmp_path: Path ) -> None: @@ -856,3 +907,120 @@ def test_offline_gates_litellm_local_cost_map_env( assert "$(cat /run/secrets/litellm_env_OPENAI_API_KEY)" in command, ( f"secret-derived exports missing from command; got: {command}" ) + + +def test_no_harness_run_does_not_inject_harness_env( + monkeypatch, tmp_path: Path +) -> None: + harness_values: list[str | None] = [] + + def fake_build_run_service_env(**kwargs): + harness_values.append(kwargs["harness"]) + return SimpleNamespace( + effective_env={"EXAMPLE": "1", "OSS_CRS_SUBMIT_DIR": "/OSS_CRS_SUBMIT_DIR"}, + warnings=[], + ) + + monkeypatch.setattr( + "oss_crs.src.templates.renderer.build_run_service_env", + fake_build_run_service_env, + ) + monkeypatch.setattr( + "oss_crs.src.templates.renderer.prepare_llm_context", + lambda *_args, **_kwargs: None, + ) + + module_config = SimpleNamespace( + dockerfile="finder.Dockerfile", + target_dependent=False, + additional_env={}, + ) + crs = SimpleNamespace( + name="crs-bug-finding-claude-code", + crs_path=tmp_path, + resource=SimpleNamespace( + cpuset="2-7", + memory="8G", + additional_env={}, + llm_budget=1, + ), + config=SimpleNamespace( + version="0.1", + type=["bug-finding"], + is_bug_fixing=False, + is_bug_fixing_ensemble=False, + is_triage=False, + is_seed_filter=False, + is_auditing=False, + crs_run_phase=SimpleNamespace(modules={"finder": module_config}), + ), + ) + crs_compose = SimpleNamespace( + crs_list=[crs], + work_dir=SimpleNamespace( + get_exchange_dir=lambda *_args, **_kwargs: ( + tmp_path / "exchange" / "OSS_CRS_UNHARNESSED" + ), + get_processed_exchange_dir=lambda *_args, **_kwargs: ( + tmp_path / "processed-exchange" / "OSS_CRS_UNHARNESSED" + ), + get_build_output_dir=lambda *_args, **_kwargs: tmp_path / "build", + get_submit_dir=lambda *_args, **_kwargs: ( + tmp_path / "submit" / "OSS_CRS_UNHARNESSED" + ), + get_shared_dir=lambda *_args, **_kwargs: ( + tmp_path / "shared" / "OSS_CRS_UNHARNESSED" + ), + get_log_dir=lambda *_args, **_kwargs: ( + tmp_path / "log" / "OSS_CRS_UNHARNESSED" + ), + get_rebuild_out_dir=lambda *_args, **_kwargs: ( + tmp_path / "rebuild_out" / "_no_harness" + ), + get_target_source_dir=lambda *_args, **_kwargs: tmp_path / "target-source", + get_run_dir=lambda *_args, **_kwargs: tmp_path / "run", + ), + crs_compose_env=SimpleNamespace(get_env=lambda: {"type": "local"}), + llm=SimpleNamespace(exists=lambda: False, mode="external"), + offline=False, + config=SimpleNamespace( + oss_crs_infra=SimpleNamespace(cpuset="0-1", memory="16G") + ), + ) + target = SimpleNamespace( + snapshot_image_tag="", + get_target_env=lambda: {}, + get_docker_image_name=lambda: "target:latest", + proj_path=tmp_path / "proj", + repo_path=tmp_path / "repo", + _has_repo=True, + ) + target.repo_path.mkdir(parents=True, exist_ok=True) + tmp_compose = SimpleNamespace(dir=tmp_path / "tmp-compose") + + rendered, warnings = render_run_crs_compose_docker_compose( + crs_compose=crs_compose, + tmp_docker_compose=tmp_compose, + crs_compose_name="proj", + target=target, + run_id="run-1", + build_id="build-1", + sanitizer="address", + source_only=True, + ) + + assert warnings == [] + assert harness_values == [None] + compose_data = yaml.safe_load(rendered) + finder_service = compose_data["services"]["crs-bug-finding-claude-code_finder"] + assert not any( + item.startswith("OSS_CRS_TARGET_HARNESS=") + for item in finder_service["environment"] + ) + assert any( + "OSS_CRS_UNHARNESSED:/OSS_CRS_SUBMIT_DIR:rw" in item + for item in finder_service["volumes"] + ) + assert "oss-crs-exchange" in compose_data["services"] + assert "oss-crs-builder-sidecar" not in compose_data["services"] + assert "oss-crs-runner-sidecar" not in compose_data["services"] diff --git a/oss_crs/tests/unit/test_source_only_run.py b/oss_crs/tests/unit/test_source_only_run.py new file mode 100644 index 00000000..dd4a19b0 --- /dev/null +++ b/oss_crs/tests/unit/test_source_only_run.py @@ -0,0 +1,285 @@ +from pathlib import Path +from types import SimpleNamespace + +import yaml + +from oss_crs.src.cli.crs_compose import _resolve_source_only, init_target_from_args +from oss_crs.src.crs_compose import CRSCompose +from oss_crs.src.config.crs import CRSType +from oss_crs.src.templates.renderer import render_run_crs_compose_docker_compose +from oss_crs.src.workdir import WorkDir + + +def test_source_only_target_uses_source_path_as_project_path(tmp_path: Path) -> None: + source_dir = tmp_path / "source" + source_dir.mkdir() + args = SimpleNamespace( + work_dir=tmp_path / "work", + target_proj_path=None, + target_repo_path=source_dir, + target_harness=None, + ) + + target = init_target_from_args(args, source_only=True) + + assert target.proj_path == source_dir + assert target.repo_path == source_dir + assert target.target_harness is None + + +def test_source_only_target_accepts_plain_source_directory(tmp_path: Path) -> None: + source_dir = tmp_path / "source" + source_dir.mkdir() + (source_dir / "main.rs").write_text("fn main() {}") + args = SimpleNamespace( + work_dir=tmp_path / "work", + target_proj_path=None, + target_repo_path=source_dir, + target_harness=None, + ) + + target = init_target_from_args(args, source_only=True) + first_workdir_key = WorkDir._get_target_key(target) + (source_dir / "main.rs").write_text('fn main() { println!("changed"); }') + changed_target = init_target_from_args(args, source_only=True) + + assert WorkDir._get_target_key(changed_target) == first_workdir_key + + +def test_source_only_workdir_key_distinguishes_source_paths(tmp_path: Path) -> None: + keys = [] + for name in ("source-a", "source-b"): + source_dir = tmp_path / name + source_dir.mkdir() + (source_dir / "main.rs").write_text("fn main() {}") + args = SimpleNamespace( + work_dir=tmp_path / "work", + target_proj_path=None, + target_repo_path=source_dir, + target_harness=None, + ) + target = init_target_from_args(args, source_only=True) + keys.append(WorkDir._get_target_key(target)) + + assert keys[0] != keys[1] + + +def test_source_only_target_requires_source_path(tmp_path: Path) -> None: + args = SimpleNamespace( + work_dir=tmp_path / "work", + target_proj_path=None, + target_repo_path=None, + target_harness=None, + ) + + try: + init_target_from_args(args, source_only=True) + except ValueError as exc: + assert ( + "--target-source-path is required when --target-harness is omitted" + in str(exc) + ) + else: + raise AssertionError("expected ValueError") + + +def test_source_only_target_requires_existing_source_directory(tmp_path: Path) -> None: + for source_path in (tmp_path / "missing", tmp_path / "source-file"): + if source_path.name == "source-file": + source_path.write_text("not a directory") + args = SimpleNamespace( + work_dir=tmp_path / "work", + target_proj_path=None, + target_repo_path=source_path, + target_harness=None, + ) + + try: + init_target_from_args(args, source_only=True, require_source_dir=True) + except ValueError as exc: + assert "must be an existing directory" in str(exc) + else: + raise AssertionError("expected ValueError") + + +def test_target_without_harness_uses_normal_target_configuration( + tmp_path: Path, +) -> None: + project_dir = tmp_path / "project" + project_dir.mkdir() + args = SimpleNamespace( + work_dir=tmp_path / "work", + target_proj_path=project_dir, + target_repo_path=None, + ) + + target = init_target_from_args(args) + + assert target.proj_path == project_dir + assert target.target_harness is None + + +def test_source_only_render_omits_build_and_fuzz_mounts(monkeypatch, tmp_path: Path): + def fake_build_run_service_env(**kwargs): + return SimpleNamespace( + effective_env={ + "OSS_CRS_SUBMIT_DIR": "/OSS_CRS_SUBMIT_DIR", + "OSS_CRS_TARGET_SOURCE": "/OSS_CRS_TARGET_SOURCE", + }, + warnings=[], + ) + + monkeypatch.setattr( + "oss_crs.src.templates.renderer.build_run_service_env", + fake_build_run_service_env, + ) + monkeypatch.setattr( + "oss_crs.src.templates.renderer.prepare_llm_context", + lambda *_args, **_kwargs: None, + ) + + module_config = SimpleNamespace( + dockerfile="finder.Dockerfile", + target_dependent=False, + additional_env={}, + ) + crs = SimpleNamespace( + name="source-finder", + crs_path=tmp_path, + resource=SimpleNamespace( + cpuset="2-7", + memory="8G", + additional_env={}, + llm_budget=1, + ), + config=SimpleNamespace( + version="1.0", + type={CRSType.BUG_FINDING}, + is_bug_fixing=False, + is_bug_fixing_ensemble=False, + is_triage=False, + is_seed_filter=False, + is_auditing=False, + crs_run_phase=SimpleNamespace(modules={"finder": module_config}), + ), + ) + crs_compose = SimpleNamespace( + crs_list=[crs], + work_dir=SimpleNamespace( + get_exchange_dir=lambda *_args, **_kwargs: tmp_path / "exchange", + get_processed_exchange_dir=lambda *_args, **_kwargs: ( + tmp_path / "processed-exchange" + ), + get_build_output_dir=lambda *_args, **_kwargs: tmp_path / "build", + get_submit_dir=lambda *_args, **_kwargs: tmp_path / "submit", + get_shared_dir=lambda *_args, **_kwargs: tmp_path / "shared", + get_log_dir=lambda *_args, **_kwargs: tmp_path / "log", + get_rebuild_out_dir=lambda *_args, **_kwargs: tmp_path / "rebuild_out", + get_target_source_dir=lambda *_args, **_kwargs: tmp_path / "target-source", + get_run_dir=lambda *_args, **_kwargs: tmp_path / "run", + ), + crs_compose_env=SimpleNamespace(get_env=lambda: {"type": "local"}), + llm=SimpleNamespace(exists=lambda: False, mode="external"), + offline=False, + config=SimpleNamespace( + oss_crs_infra=SimpleNamespace(cpuset="0-1", memory="16G") + ), + ) + target = SimpleNamespace( + snapshot_image_tag="", + get_target_env=lambda: {}, + get_docker_image_name=lambda: "should-not-be-used:latest", + proj_path=tmp_path / "source", + repo_path=tmp_path / "source", + _has_repo=True, + ) + target.repo_path.mkdir() + + rendered, warnings = render_run_crs_compose_docker_compose( + crs_compose=crs_compose, + tmp_docker_compose=SimpleNamespace(dir=tmp_path / "tmp-compose"), + crs_compose_name="proj", + target=target, + run_id="run-1", + build_id="source-only-run-1", + sanitizer="address", + source_only=True, + ) + + assert warnings == [] + compose_data = yaml.safe_load(rendered) + service = compose_data["services"]["source-finder_finder"] + assert service["image"] == "oss-crs-runner:source-finder-finder" + assert "build" not in service + assert not any("/OSS_CRS_BUILD_OUT_DIR" in item for item in service["volumes"]) + assert not any("/OSS_CRS_FUZZ_PROJ" in item for item in service["volumes"]) + assert any("/OSS_CRS_TARGET_SOURCE:ro" in item for item in service["volumes"]) + assert "oss-crs-builder-sidecar" not in compose_data["services"] + assert "oss-crs-runner-sidecar" not in compose_data["services"] + + +def test_source_only_rejects_target_dependent_run_module() -> None: + crs = SimpleNamespace( + name="target-dependent-crs", + config=SimpleNamespace( + is_auditing=True, + crs_run_phase=SimpleNamespace( + modules={ + "finder": SimpleNamespace(target_dependent=True), + } + ), + ), + ) + compose = CRSCompose.__new__(CRSCompose) + compose.crs_list = [crs] + + result = compose._validate_source_only_run() + + assert result.success is False + assert "target-dependent-crs" in result.error + + +def test_source_only_rejects_non_auditing_crs() -> None: + crs = SimpleNamespace( + name="bug-finding-crs", + config=SimpleNamespace(is_auditing=False), + ) + compose = CRSCompose.__new__(CRSCompose) + compose.crs_list = [crs] + + result = compose._validate_source_only_run() + + assert result.success is False + assert "bug-finding-crs" in result.error + assert "auditing" in result.error + + +def _source_mode_args(**overrides) -> SimpleNamespace: + defaults: dict = {"target_harness": None, "target_proj_path": None} + defaults.update(overrides) + return SimpleNamespace(**defaults) + + +def _compose_with(crs_types: list[str]) -> SimpleNamespace: + return SimpleNamespace( + crs_list=[ + SimpleNamespace(config=SimpleNamespace(is_harness_gen=t == "harness-gen")) + for t in crs_types + ] + ) + + +def test_resolve_source_only_classification_matrix() -> None: + auditing = _compose_with(["auditing"]) + harness_gen = _compose_with(["harness-gen"]) + + assert _resolve_source_only(_source_mode_args(), auditing) is True + assert _resolve_source_only(_source_mode_args(), harness_gen) is False + assert ( + _resolve_source_only(_source_mode_args(target_harness="fuzz_target"), auditing) + is False + ) + assert ( + _resolve_source_only(_source_mode_args(target_proj_path="proj"), auditing) + is False + ) diff --git a/oss_crs/tests/unit/test_target_defaults.py b/oss_crs/tests/unit/test_target_defaults.py index bde492ac..0ead68ed 100644 --- a/oss_crs/tests/unit/test_target_defaults.py +++ b/oss_crs/tests/unit/test_target_defaults.py @@ -1,5 +1,6 @@ # SPDX-License-Identifier: MIT from pathlib import Path +from unittest.mock import Mock from oss_crs.src.target import Target @@ -142,3 +143,20 @@ def test_user_provided_missing_repo_path_fails_init(tmp_path: Path) -> None: missing_repo = tmp_path / "missing-repo" target = Target(tmp_path / "work", proj, missing_repo) assert target.init_repo() is False + + +def test_build_docker_image_does_not_depend_on_harness( + tmp_path: Path, monkeypatch +) -> None: + proj = tmp_path / "proj" + proj.mkdir(parents=True) + (proj / "Dockerfile").write_text("FROM scratch\n") + target = Target(tmp_path / "work", proj, None, target_harness=None) + built = Mock(return_value=Mock(success=True)) + monkeypatch.setattr( + "oss_crs.src.target.subprocess.run", lambda *_a, **_k: Mock(returncode=1) + ) + monkeypatch.setattr("oss_crs.src.target.MultiTaskProgress.run_added_tasks", built) + + assert target.build_docker_image() == target.get_docker_image_name() + built.assert_called_once() diff --git a/oss_crs/tests/unit/test_workdir_no_harness.py b/oss_crs/tests/unit/test_workdir_no_harness.py new file mode 100644 index 00000000..4df83235 --- /dev/null +++ b/oss_crs/tests/unit/test_workdir_no_harness.py @@ -0,0 +1,30 @@ +from pathlib import Path + +from oss_crs.src.workdir import WorkDir + + +class _Target: + target_harness = None + + def get_docker_image_name(self) -> str: + return "mock-target:abc123" + + def get_workdir_target_key(self) -> str: + return self.get_docker_image_name().replace(":", "_") + + +def test_run_artifact_dirs_use_no_harness_scope(tmp_path: Path) -> None: + work_dir = WorkDir(tmp_path) + target = _Target() + + submit_dir = work_dir.get_submit_dir("crs-a", target, "run-1", "address") + shared_dir = work_dir.get_shared_dir("crs-a", target, "run-1", "address") + log_dir = work_dir.get_log_dir("crs-a", target, "run-1", "address") + exchange_dir = work_dir.get_exchange_dir(target, "run-1", "address") + run_logs_dir = work_dir.get_run_logs_dir(target, "run-1", "address") + + assert submit_dir.name == WorkDir.NO_HARNESS_SCOPE + assert shared_dir.name == WorkDir.NO_HARNESS_SCOPE + assert log_dir.name == WorkDir.NO_HARNESS_SCOPE + assert exchange_dir.name == WorkDir.NO_HARNESS_SCOPE + assert run_logs_dir.name == WorkDir.NO_HARNESS_SCOPE diff --git a/oss_crs/tests/unit/test_workdir_target_source.py b/oss_crs/tests/unit/test_workdir_target_source.py index a5a35bee..ce361b7f 100644 --- a/oss_crs/tests/unit/test_workdir_target_source.py +++ b/oss_crs/tests/unit/test_workdir_target_source.py @@ -14,6 +14,9 @@ def __init__(self, name: str = "test-proj", image_hash: str = "abc123"): def get_docker_image_name(self) -> str: return f"{self.name}:{self._image_hash}" + def get_workdir_target_key(self) -> str: + return self.get_docker_image_name().replace(":", "_") + class TestGetTargetSourceDir: """Tests for WorkDir.get_target_source_dir()."""