diff --git a/README.md b/README.md index e224c38..f1cc69b 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # 51Code Agent Skills -This repository is the source package for 15 skills maintained by 51Code and +This repository is the source package for 16 skills maintained by 51Code and the bootstrap for our reviewed machine-global skill baseline. The repository intentionally has no catalog, registry, generated lock, sync @@ -9,14 +9,15 @@ package directly. ## Global baseline -The global baseline supports Codex and Claude Code and contains these 38 skills: +The global baseline supports Codex and Claude Code and contains these 39 skills: - 51Code-owned: `code-review`, `gemini-files-api`, `harness-engineering`, `hint-overlay-visual-verification`, `ios-xcodegen`, `lifecycle-and-side-effects-correctness`, `local-model-serving`, `mechanism-audit`, - `meeting-transcription`, `silent-pushes-setup`, `spec-creation-updating`, - `swift-testing`, `swiftui-view-refactor`, `xcode-build`, and `xcode-cloud` + `meeting-transcription`, `public-source-release-audit`, + `silent-pushes-setup`, `spec-creation-updating`, `swift-testing`, + `swiftui-view-refactor`, `xcode-build`, and `xcode-cloud` - Third-party: `swift-concurrency` and the 22 `asc-*` App Store Connect CLI skills from `rorkai/app-store-connect-cli-skills` @@ -37,7 +38,7 @@ idempotent. The script uses `skills@1.5.14`, explicit Git tags or commits, and explicit skill names. In this CLI, `#ref` selects a Git branch or tag; `@name` selects a skill and must not be used as a version pin. Sources pinned to a raw commit are checked out and verified before being passed to the manager as a -local source. Before reporting success, the script verifies all 38 entrypoints +local source. Before reporting success, the script verifies all 39 entrypoints in the shared and Claude Code manager roots and bootstraps the copied `gemini-files-api` dependencies in both roots. The Claude Code root honors `CLAUDE_CONFIG_DIR` when it is set. The script also converts the pinned diff --git a/bootstrap.sh b/bootstrap.sh index 72780a0..8de31f2 100755 --- a/bootstrap.sh +++ b/bootstrap.sh @@ -39,6 +39,7 @@ owned_skills=( local-model-serving mechanism-audit meeting-transcription + public-source-release-audit silent-pushes-setup spec-creation-updating swift-testing diff --git a/public-source-release-audit/SKILL.md b/public-source-release-audit/SKILL.md new file mode 100644 index 0000000..e0b68f2 --- /dev/null +++ b/public-source-release-audit/SKILL.md @@ -0,0 +1,105 @@ +--- +name: public-source-release-audit +description: Check a Git repository's current candidate for high-confidence credential, private-key, and machine-home leaks, then perform a bounded public-release review. Use before publishing source or opening a public PR; do not use it to claim complete history or workflow-execution safety. +--- + +# Public-Source Release Audit + +Use this skill to answer a narrow question honestly: does the current source +candidate contain a supported high-confidence leak, and have the separate +human and GitHub-native release checks been confirmed? + +## Automated Current-Source Check + +Run the repository's own verifier when it exists. Then run: + +```bash +ruby /scripts/check_current_source.rb +``` + +The checker reads exact Git-index blobs so an unstaged replacement cannot hide +the candidate. It also scans tracked and non-ignored untracked worktree files +for early feedback. It reports only paths and finding categories, never the +matched content. + +The supported lexical forms are deliberately small: + +- PEM, OpenSSH, PGP, and PuTTY private-key headers; +- GitHub `ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_`, and `github_pat_` token + prefixes; +- AWS `AKIA`/`ASIA` access-key IDs and canonical named 40-character secret + access-key values; +- OpenAI `sk-` and `sk-proj-` key prefixes; +- explicit `Authorization: Bearer` header or map values; and +- literal, commonly source-escaped, or `file://`-wrapped rooted POSIX families + `Users/`, `var/root`, `private/var/root`, `home/`, and `root`, plus + drive-letter `Users\\` paths, including UTF-8 profile names and bounded + environment and command-option values. + +The home-path rule covers literal paths and common source escaping for POSIX +slashes and Windows backslashes. It does not decode URL-encoded content. The +credential map rules inspect direct source text; they do not decode nested +escaped documents. + +Stage all intended release changes before relying on the candidate result. +Ignored files and submodule contents are outside this check. The checker does +not inspect Git history. Git LFS pointers fail the check because their external +objects require separate review. Missing index blobs, including unavailable +partial-clone objects, fail without network fetching. A symlink's own target +text is scanned, while a symlinked parent component fails without reading +outside the repository. Git's optional locks are disabled so read-only index +operations do not freshen split-index metadata, and inherited Git trace-file +destinations are cleared. BOM-marked UTF-16 source also requires separate +review; the checker does not transcode or unpack content. + +## Semantic Review + +Automation cannot decide whether prose, filenames, images, or other assets +contain customer information, internal links, personal data, or company-only +context. Inspect the complete tree before first publication and the exact diff +for later releases. Mark this check unconfirmed unless a reviewer actually +performed it. + +## GitHub-Native Controls + +For an existing public GitHub repository, use read-only GitHub commands to +confirm the live state rather than recreating GitHub policy logic locally: + +```bash +gh repo view OWNER/REPO --json visibility,defaultBranchRef +gh api repos/OWNER/REPO --jq '{visibility,security_and_analysis}' +gh ruleset check --default --repo OWNER/REPO +gh ruleset list --repo OWNER/REPO --parents --limit 100 +gh ruleset view RULESET-ID --repo OWNER/REPO +default_branch="$(gh repo view OWNER/REPO --json defaultBranchRef \ + --jq '.defaultBranchRef.name')" +encoded_branch="$(ruby -rerb -e \ + 'print ERB::Util.url_encode(ARGV.fetch(0))' "$default_branch")" +gh api "repos/OWNER/REPO/branches/${encoded_branch}/protection" +gh api 'repos/OWNER/REPO/actions/runners?per_page=100' \ + --jq '{total_count,runners:[.runners[] | {name,status,busy}]}' +``` + +Inspect applicable rulesets and classic branch protection when necessary to +confirm the required hosted check, update strictness, and bypass actors. A 404 +from the classic endpoint means no classic rule is configured; it does not +invalidate an applicable ruleset. If authorization cannot expose a setting, +report it as unconfirmed. Do not weaken or mutate settings unless the user +separately authorizes that action. + +## Result + +Report these fields separately: + +- repository verifier: `passed`, `failed`, or `not available`; +- automated current-source check: `passed` or `failed`; +- semantic context review: `confirmed` or `unconfirmed`; +- GitHub-native controls: `confirmed`, `unconfirmed`, or `not applicable`; +- history review: normally `not performed`; +- bounded verdict: `ready` only when every check required for this release is + passed or confirmed. + +Do not summarize the bounded verdict as “the repository is safe.” Full-history +investigation is a separate, exceptional workflow using established tooling. +Publishing, rewriting history, revoking credentials, and changing repository +settings remain separate actions requiring their own authorization. diff --git a/public-source-release-audit/scripts/check_current_source.rb b/public-source-release-audit/scripts/check_current_source.rb new file mode 100755 index 0000000..19fb698 --- /dev/null +++ b/public-source-release-audit/scripts/check_current_source.rb @@ -0,0 +1,337 @@ +#!/usr/bin/env ruby +# frozen_string_literal: true + +require "open3" +require "set" + +RULES = [ + [ + "machine-local home path", + %r{ + (?:\A|[\s"'`>:(,]|(?"'\s,;:)\]\}]|\z)) + | + root(?:(?:/|\\/)|(?![A-Za-z0-9._-])) + | + (?:private(?:/|\\/))?var(?:/|\\/)root(?:(?:/|\\/)|(?![A-Za-z0-9._-])) + ) + | + [A-Za-z]:[\\/]{1,2}(?i:Users)[\\/]{1,2}[A-Za-z0-9._\x80-\xFF-]+(?:[\\/]{1,2}|(?=[<>"'\s,;:)\]\}]|\z)) + ) + }xn + ], + ["AWS access key", /\b(?:AKIA|ASIA)[0-9A-Z]{16}\b/], + [ + "AWS secret access key", + /(?["']?)aws_secret_access_key\k(?![A-Za-z0-9_])\s*(?:=|:)\s*["']?[A-Za-z0-9\/=+]{40}(?=["'\s\r\n,}\]]|\z)/i + ], + [ + "bearer credential", + /(?["']?)Authorization\k(?![A-Za-z0-9_.-])\s*(?::|=>)\s*["']?Bearer\s+[A-Za-z0-9._~+\/=:-]{16,}/i + ], + [ + "GitHub token", + /\b(?:gh[pousr]_[A-Za-z0-9]{20,}|github_pat_[A-Za-z0-9_]{20,})\b/ + ], + ["OpenAI API key", /\bsk-(?:proj-)?[A-Za-z0-9_-]{20,}\b/], + [ + "private key", + /-----BEGIN (?:[A-Z0-9][A-Z0-9 ]* )?PRIVATE KEY(?: BLOCK)?-----/ + ], + ["private key", /\APuTTY-User-Key-File-[23]:/] +].freeze + +INDEX_BLOB_MODES = %w[100644 100755 120000].freeze +LFS_POINTER = %r{\Aversion https://git-lfs\.github\.com/spec/v1(?:\r?\n|\z)}.freeze +UTF16_BOMS = ["\xFF\xFE".b, "\xFE\xFF".b].freeze +GIT_ENVIRONMENT = { + "GIT_ALTERNATE_OBJECT_DIRECTORIES" => nil, + "GIT_COMMON_DIR" => nil, + "GIT_DIR" => nil, + "GIT_INDEX_FILE" => nil, + "GIT_NO_LAZY_FETCH" => "1", + "GIT_NO_REPLACE_OBJECTS" => "1", + "GIT_OBJECT_DIRECTORY" => nil, + "GIT_OPTIONAL_LOCKS" => "0", + "GIT_TRACE" => nil, + "GIT_TRACE2" => nil, + "GIT_TRACE2_EVENT" => nil, + "GIT_TRACE2_PERF" => nil, + "GIT_TRACE_CURL" => nil, + "GIT_TRACE_PACK_ACCESS" => nil, + "GIT_TRACE_PACKET" => nil, + "GIT_TRACE_PERFORMANCE" => nil, + "GIT_TRACE_SETUP" => nil, + "GIT_TRACE_SHALLOW" => nil, + "GIT_WORK_TREE" => nil +}.freeze + +def usage(message = nil) + warn message if message + warn "Usage: check_current_source.rb [repository]" + exit 64 +end + +def git_command(repo, *arguments) + [ + "git", + "-C", + repo, + "-c", + "core.fsmonitor=false", + *arguments + ] +end + +def git_capture(repo, *arguments) + Open3.capture3(GIT_ENVIRONMENT, *git_command(repo, *arguments)) +end + +def read_blob_results(repo, object_ids) + return [{}, true] if object_ids.empty? + + results = {} + protocol_failed = false + Open3.popen3( + GIT_ENVIRONMENT, + *git_command(repo, "cat-file", "--batch") + ) do |input, output, error, wait_thread| + input.binmode + output.binmode + error_reader = Thread.new { error.read } + + begin + object_ids.each do |object_id| + input.write(object_id) + input.write("\n") + input.flush + + header = output.gets + unless header + protocol_failed = true + break + end + + fields = header.delete_suffix("\n".b).split(" ".b) + if fields.length == 2 && fields[0] == object_id && fields[1] == "missing".b + next + end + + returned_id, type, size_text = fields + size = begin + Integer(size_text, 10) + rescue ArgumentError, TypeError + nil + end + unless fields.length == 3 && returned_id == object_id && size && size >= 0 + protocol_failed = true + break + end + + content = output.read(size) + terminator = output.read(1) + unless content&.bytesize == size && terminator == "\n".b + protocol_failed = true + break + end + next unless type == "blob".b + + results[object_id] = { + labels: matching_labels(content), + lfs_pointer: LFS_POINTER.match?(content.b), + utf16_bom: utf16_bom?(content) + } + end + rescue IOError, SystemCallError + protocol_failed = true + ensure + input.close unless input.closed? + output.close if protocol_failed && !output.closed? + end + + error_reader.value + protocol_failed = true unless wait_thread.value.success? + end + + [results, !protocol_failed] +end + +def matching_labels(content) + source = content.b + RULES.filter_map { |label, pattern| label if pattern.match?(source) } +end + +def utf16_bom?(content) + source = content.b + UTF16_BOMS.any? { |bom| source.start_with?(bom) } +end + +def record_findings(labels, relative_path, findings, sensitive_paths = nil) + labels.each do |label| + findings.add([relative_path, label]) + sensitive_paths&.add(relative_path) + end +end + +def scan_source(content, relative_path, findings, sensitive_paths = nil) + record_findings( + matching_labels(content), + relative_path, + findings, + sensitive_paths + ) +end + +def display_path(relative_path, sensitive_paths) + sensitive_paths.include?(relative_path) ? "" : relative_path.dump +end + +def symlinked_parent?(repo, relative_path) + current = repo.b + relative_path.b.split("/".b)[0...-1].any? do |component| + current = File.join(current, component) + File.symlink?(current) + end +end + +usage("Too many arguments") if ARGV.length > 1 +repo = File.expand_path(ARGV.first || ".") +usage("Repository must be a directory") unless File.directory?(repo) + +toplevel_output, _toplevel_error, toplevel_status = git_capture( + repo, + "rev-parse", + "--show-toplevel" +) +usage("Repository must be a Git worktree") unless toplevel_status.success? +repo = toplevel_output.b.sub(/\r?\n\z/, "".b) +usage("Unable to resolve Git worktree root") unless File.directory?(repo) + +index_output, _index_error, index_status = git_capture( + repo, + "ls-files", + "--stage", + "-z" +) +usage("Unable to enumerate Git index") unless index_status.success? + +errors = Set.new +findings = Set.new +sensitive_paths = Set.new +index_entries = index_output.b.split("\0".b).reject(&:empty?).filter_map do |record| + metadata, relative_path = record.split("\t".b, 2) + mode, object_id, stage = metadata&.split(" ".b, 3) + unless mode && object_id && stage && relative_path + errors.add([relative_path || "", "unreadable Git index entry"]) + next + end + + if stage != "0" + errors.add([relative_path, "unresolved Git index entry"]) + next + end + + { mode: mode, object_id: object_id, path: relative_path } +end + +blob_paths = Hash.new { |paths, object_id| paths[object_id] = [] } +submodule_paths = Set.new +index_entries.each do |entry| + relative_path = entry.fetch(:path) + scan_source(relative_path, relative_path, findings, sensitive_paths) + if entry.fetch(:mode) == "160000" + submodule_paths.add(relative_path) + next + end + + unless INDEX_BLOB_MODES.include?(entry.fetch(:mode)) + errors.add([relative_path, "unsupported Git index mode"]) + next + end + + blob_paths[entry.fetch(:object_id)] << relative_path +end + +blob_results, blob_reader_success = read_blob_results(repo, blob_paths.keys) +unless blob_reader_success + errors.add(["", "Git index blob reader failed"]) +end +blob_paths.each do |object_id, relative_paths| + result = blob_results[object_id] + relative_paths.each do |relative_path| + unless result + errors.add([relative_path, "unable to read Git index blob"]) + next + end + if result.fetch(:lfs_pointer) + errors.add([relative_path, "Git LFS object requires separate review"]) + end + if result.fetch(:utf16_bom) + errors.add([relative_path, "BOM-marked UTF-16 source requires separate review"]) + end + record_findings(result.fetch(:labels), relative_path, findings) + end +end + +worktree_output, _worktree_error, worktree_status = git_capture( + repo, + "ls-files", + "-co", + "--exclude-standard", + "-z" +) +usage("Unable to enumerate repository worktree source") unless worktree_status.success? + +worktree_paths = worktree_output.b.split("\0".b).reject(&:empty?).uniq +worktree_paths.each do |relative_path| + scan_source(relative_path, relative_path, findings, sensitive_paths) + next if submodule_paths.include?(relative_path) + + begin + if symlinked_parent?(repo, relative_path) + errors.add([relative_path, "symlinked parent component"]) + next + end + absolute_path = File.join(repo.b, relative_path) + stat = File.lstat(absolute_path) + content = if stat.symlink? + File.readlink(absolute_path) + elsif stat.file? + File.binread(absolute_path) + else + errors.add([relative_path, "unsupported worktree source type"]) + next + end + if content + if LFS_POINTER.match?(content.b) + errors.add([relative_path, "Git LFS object requires separate review"]) + end + if utf16_bom?(content) + errors.add([relative_path, "BOM-marked UTF-16 source requires separate review"]) + end + scan_source(content, relative_path, findings) + end + rescue Errno::ENOENT + next + rescue SystemCallError + errors.add([relative_path, "unable to read worktree source"]) + end +end + +unless errors.empty? && findings.empty? + warn "current public-source check failed:" + errors.to_a.sort_by { |relative_path, label| [relative_path.b, label] }.each do |relative_path, label| + warn "- #{display_path(relative_path, sensitive_paths)}: #{label}" + end + findings.to_a.sort_by { |relative_path, label| [relative_path.b, label] }.each do |relative_path, label| + warn "- #{display_path(relative_path, sensitive_paths)}: #{label}" + end + warn "Matched content is intentionally omitted." unless findings.empty? + exit 1 +end + +puts "current public-source check passed" diff --git a/public-source-release-audit/tests/test_check_current_source.rb b/public-source-release-audit/tests/test_check_current_source.rb new file mode 100644 index 0000000..242e512 --- /dev/null +++ b/public-source-release-audit/tests/test_check_current_source.rb @@ -0,0 +1,619 @@ +# frozen_string_literal: true + +require "fileutils" +require "minitest/autorun" +require "open3" +require "tmpdir" + +class CheckCurrentSourceTest < Minitest::Test + CHECKER = File.expand_path("../scripts/check_current_source.rb", __dir__) + + def with_repo + Dir.mktmpdir("public-source-check-") do |repo| + system("git", "-C", repo, "init", "--quiet", exception: true) + yield repo + end + end + + def write(repo, relative_path, content) + path = File.join(repo, relative_path) + FileUtils.mkdir_p(File.dirname(path)) + File.binwrite(path, content) + end + + def stage(repo, relative_path, content) + write(repo, relative_path, content) + system("git", "-C", repo, "add", "--", relative_path, exception: true) + end + + def run_checker(repo, environment = {}) + Open3.capture3(environment, "ruby", CHECKER, repo) + end + + def github_token + "gh" + "p_" + ("A" * 24) + end + + def lfs_pointer + [ + "version https://git-lfs.github.com/spec/v1", + "oid sha256:#{'a' * 64}", + "size 123" + ].join("\n") + "\n" + end + + def test_safe_candidate_passes + with_repo do |repo| + stage(repo, "README.md", "Public documentation without credentials.\n") + + stdout, stderr, status = run_checker(repo) + + assert status.success?, stderr + assert_includes stdout, "current public-source check passed" + end + end + + def test_staged_credential_cannot_be_hidden_by_clean_worktree_replacement + with_repo do |repo| + token = github_token + stage(repo, "masked.txt", token) + write(repo, "masked.txt", "clean replacement\n") + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "masked.txt" + assert_includes stderr, "GitHub token" + refute_includes stderr, token + end + end + + def test_subdirectory_argument_still_scans_the_complete_worktree + with_repo do |repo| + token = github_token + stage(repo, "root-secret.txt", token) + child = File.join(repo, "child") + FileUtils.mkdir_p(child) + + _stdout, stderr, status = run_checker(child) + + refute status.success? + assert_includes stderr, "root-secret.txt" + assert_includes stderr, "GitHub token" + refute_includes stderr, token + end + end + + def test_unstaged_credential_is_checked_before_it_can_be_added + with_repo do |repo| + token = github_token + stage(repo, "pending.txt", "safe candidate\n") + write(repo, "pending.txt", token) + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "pending.txt" + assert_includes stderr, "GitHub token" + refute_includes stderr, token + end + end + + def test_git_environment_cannot_redirect_the_candidate_index + with_repo do |repo| + token = github_token + stage(repo, "candidate.txt", token) + alternate_index = File.join(repo, "alternate-index") + environment = { "GIT_INDEX_FILE" => alternate_index } + _output, error, status = Open3.capture3( + environment, + "git", + "-C", + repo, + "read-tree", + "--empty" + ) + assert status.success?, error + + _stdout, stderr, checker_status = run_checker(repo, environment) + + refute checker_status.success? + assert_includes stderr, "candidate.txt" + assert_includes stderr, "GitHub token" + refute_includes stderr, token + end + end + + def test_sensitive_filename_is_detected_without_echoing_it + with_repo do |repo| + token = github_token + stage(repo, token, "safe content\n") + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "" + assert_includes stderr, "GitHub token" + refute_includes stderr, token + end + end + + def test_supported_high_confidence_formats_are_detected_and_redacted + with_repo do |repo| + escaped_slash = "\\" + "/" + fixtures = { + "aws.txt" => "AK" + "IA" + ("A" * 16), + "aws-secret.txt" => "AWS_SECRET_ACCESS_KEY=" + ("S" * 40), + "aws-secret-json.txt" => "{\"AWS_SECRET_ACCESS_KEY\":\"" + ("J" * 40) + "\"}", + "bearer.txt" => "Authorization: " + "Bearer " + ("b" * 24), + "bearer-json.txt" => "{\"Authorization\":\"" + "Bearer " + ("e" * 24) + "\"}", + "fine-grained.txt" => "github_" + "pat_" + ("C" * 24), + "openai.txt" => "s" + "k-proj-" + ("D" * 24), + "private-key.txt" => "-----BEGIN " + "OPENSSH PRIVATE KEY-----", + "pgp-private-key.txt" => "-----BEGIN " + "PGP PRIVATE KEY BLOCK-----", + "putty-private-key.txt" => "PuTTY" + "-User-Key-File-3: ssh-ed25519", + "markup-home.txt" => "" + "/" + "Users/example/private.txt", + "posix-home.txt" => "/" + "Users/example/private.txt", + "escaped-posix-home.txt" => [ + escaped_slash, + "home", + escaped_slash, + "example", + escaped_slash, + "private.txt" + ].join, + "root-home.txt" => "/" + "root/.ssh/id_ed25519", + "macos-root-home.txt" => "/" + "var/root/.ssh/id_ed25519", + "macos-private-root-home.txt" => "/" + "private/var/root/.ssh/id_ed25519", + "unicode-posix-home.txt" => "/" + "home/山田/private.txt", + "terminal-unicode-posix-home.txt" => "/" + "home/山田\n", + "file-uri-home.txt" => "file://" + "/" + "home/example/private.txt", + "windows-home.txt" => "C:" + "\\Users\\example\\private.txt", + "lowercase-windows-home.txt" => "c:" + "\\users\\example\\private.txt", + "unicode-windows-home.txt" => "C:" + "\\Users\\山田\\private.txt", + "terminal-markup-windows-home.txt" => "" + "C:" + "\\Users\\example" + "", + "escaped-windows-home.txt" => [ + "C:", + "\\" * 2, + "Users", + "\\" * 2, + "example", + "\\" * 2, + "private.txt" + ].join + } + fixtures.each { |path, content| stage(repo, path, content) } + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + fixtures.each do |path, content| + assert_includes stderr, path + refute_includes stderr, content + end + [ + "AWS access key", + "AWS secret access key", + "bearer credential", + "GitHub token", + "OpenAI API key", + "private key", + "machine-local home path" + ].each { |label| assert_includes stderr, label } + end + end + + def test_symlink_target_is_scanned_without_following_it + with_repo do |repo| + target = "/" + "home/example/private.txt" + File.symlink(target, File.join(repo, "local-link")) + system("git", "-C", repo, "add", "local-link", exception: true) + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "local-link" + assert_includes stderr, "machine-local home path" + refute_includes stderr, target + end + end + + def test_symlinked_parent_is_rejected_without_reading_outside_repo + with_repo do |repo| + stage(repo, "dir/file.txt", "safe candidate\n") + FileUtils.mv(File.join(repo, "dir"), File.join(repo, "original-dir")) + + Dir.mktmpdir("public-source-check-outside-") do |outside| + token = github_token + write(outside, "file.txt", token) + File.symlink(outside, File.join(repo, "dir")) + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "dir/file.txt" + assert_includes stderr, "symlinked parent component" + refute_includes stderr, "GitHub token" + refute_includes stderr, token + end + end + end + + def test_ignored_local_files_are_outside_the_release_candidate + with_repo do |repo| + stage(repo, ".gitignore", "ignored.txt\n") + write(repo, "ignored.txt", github_token) + + stdout, stderr, status = run_checker(repo) + + assert status.success?, stderr + assert_includes stdout, "current public-source check passed" + end + end + + def test_initialized_submodule_directory_is_outside_the_check + with_repo do |repo| + object_id, error, status = Open3.capture3( + "git", + "-C", + repo, + "hash-object", + "-w", + "--stdin", + stdin_data: "placeholder" + ) + assert status.success?, error + system( + "git", + "-C", + repo, + "update-index", + "--add", + "--cacheinfo", + "160000,#{object_id.strip},module", + exception: true + ) + FileUtils.mkdir_p(File.join(repo, "module")) + + stdout, stderr, checker_status = run_checker(repo) + + assert checker_status.success?, stderr + assert_includes stdout, "current public-source check passed" + end + end + + def test_unrelated_home_directory_and_web_route_do_not_match_user_homes + with_repo do |repo| + windows_profiles = "C:" + "\\Users\\" + stage(repo, "paths.txt", [ + "/tmp/home/.local/bin/tool", + "https://example.invalid/home/user/profile", + "https://example.invalid/login?next=/home/user/profile", + "Windows stores profiles under #{windows_profiles} by default." + ].join("\n")) + + stdout, stderr, status = run_checker(repo) + + assert status.success?, stderr + assert_includes stdout, "current public-source check passed" + end + end + + def test_similar_bearer_map_key_is_not_authorization + with_repo do |repo| + value = "Bearer " + ("a" * 24) + stage(repo, "map.json", "{\"NotAuthorization\":\"#{value}\"}\n") + + stdout, stderr, status = run_checker(repo) + + assert status.success?, stderr + assert_includes stdout, "current public-source check passed" + end + end + + def test_home_assignment_is_still_detected + with_repo do |repo| + home = "/" + "home/example/private.txt" + stage(repo, "settings.env", "HOME=#{home}\n") + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "settings.env" + assert_includes stderr, "machine-local home path" + refute_includes stderr, home + end + end + + def test_command_option_home_assignment_is_detected + with_repo do |repo| + home = "/" + "home/example/private.txt" + stage(repo, "command.txt", "tool --cache=#{home}\n") + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "command.txt" + assert_includes stderr, "machine-local home path" + refute_includes stderr, home + end + end + + def test_duplicate_index_blobs_report_each_candidate_path + with_repo do |repo| + token = github_token + stage(repo, "first.txt", token) + stage(repo, "second.txt", token) + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "first.txt" + assert_includes stderr, "second.txt" + assert_includes stderr, "GitHub token" + refute_includes stderr, token + end + end + + def test_git_commands_enforce_read_only_environment + with_repo do |repo| + stage(repo, "README.md", "Public documentation without credentials.\n") + real_git = ENV.fetch("PATH").split(File::PATH_SEPARATOR).filter_map do |dir| + candidate = File.join(dir, "git") + candidate if File.executable?(candidate) + end.first + refute_nil real_git + + Dir.mktmpdir("public-source-check-bin-") do |bin_dir| + trace_path = File.join(bin_dir, "git-trace.log") + fake_git = File.join(bin_dir, "git") + File.write(fake_git, <<~RUBY) + #!/usr/bin/env ruby + abort "lazy fetching was not disabled" unless ENV["GIT_NO_LAZY_FETCH"] == "1" + abort "optional Git locks were not disabled" unless ENV["GIT_OPTIONAL_LOCKS"] == "0" + abort "Git tracing was not disabled" if ENV.key?("GIT_TRACE") + exec #{real_git.dump}, *ARGV + RUBY + FileUtils.chmod(0o755, fake_git) + + stdout, stderr, status = run_checker( + repo, + { + "GIT_NO_LAZY_FETCH" => "0", + "GIT_OPTIONAL_LOCKS" => "1", + "GIT_TRACE" => trace_path, + "PATH" => [bin_dir, ENV.fetch("PATH")].join(File::PATH_SEPARATOR) + } + ) + + assert status.success?, stderr + assert_includes stdout, "current public-source check passed" + refute File.exist?(trace_path), "checker wrote inherited Git trace output" + end + end + end + + def test_inaccessible_worktree_source_fails_closed + skip "permission test requires a non-root user" if Process.uid.zero? + + with_repo do |repo| + token = github_token + stage(repo, "private/file.txt", "safe candidate\n") + write(repo, "private/file.txt", token) + directory = File.join(repo, "private") + FileUtils.chmod(0o000, directory) + + begin + _stdout, stderr, status = run_checker(repo) + ensure + FileUtils.chmod(0o755, directory) + end + + refute status.success? + assert_includes stderr, "private/file.txt" + assert_includes stderr, "unable to read worktree source" + refute_includes stderr, token + end + end + + def test_repository_fsmonitor_hook_is_disabled + with_repo do |repo| + stage(repo, "README.md", "Public documentation without credentials.\n") + hook = File.join(repo, "fsmonitor-hook") + marker = File.join(repo, "fsmonitor-ran") + File.write(hook, <<~'SH') + #!/bin/sh + printf 'ran\n' > "$FSMONITOR_MARKER" + SH + FileUtils.chmod(0o755, hook) + system( + "git", + "-C", + repo, + "config", + "core.fsmonitor", + hook, + exception: true + ) + + _output, error, status = Open3.capture3( + { "FSMONITOR_MARKER" => marker }, + "git", + "-C", + repo, + "ls-files" + ) + assert status.success?, error + assert File.exist?(marker), "fsmonitor fixture did not execute" + FileUtils.rm(marker) + + stdout, stderr, checker_status = run_checker( + repo, + { "FSMONITOR_MARKER" => marker } + ) + + assert checker_status.success?, stderr + assert_includes stdout, "current public-source check passed" + refute File.exist?(marker), "checker executed the repository fsmonitor hook" + end + end + + def test_unique_blobs_use_one_cat_file_process + with_repo do |repo| + stage(repo, "first.txt", "first safe blob\n") + stage(repo, "second.txt", "second safe blob\n") + real_git = ENV.fetch("PATH").split(File::PATH_SEPARATOR).filter_map do |dir| + candidate = File.join(dir, "git") + candidate if File.executable?(candidate) + end.first + refute_nil real_git + + Dir.mktmpdir("public-source-check-bin-") do |bin_dir| + marker = File.join(bin_dir, "cat-file-processes") + fake_git = File.join(bin_dir, "git") + File.write(fake_git, <<~RUBY) + #!/usr/bin/env ruby + if ARGV.include?("cat-file") + File.open(ENV.fetch("CAT_FILE_MARKER"), "a") { |file| file.puts("started") } + end + exec #{real_git.dump}, *ARGV + RUBY + FileUtils.chmod(0o755, fake_git) + + stdout, stderr, status = run_checker( + repo, + { + "CAT_FILE_MARKER" => marker, + "PATH" => [bin_dir, ENV.fetch("PATH")].join(File::PATH_SEPARATOR) + } + ) + + assert status.success?, stderr + assert_includes stdout, "current public-source check passed" + assert_equal ["started\n"], File.readlines(marker) + end + end + end + + def test_non_utf8_git_paths_are_scanned_without_crashing + with_repo do |repo| + token = github_token + Dir.mktmpdir("public-source-check-bin-") do |bin_dir| + fake_git = File.join(bin_dir, "git") + File.write(fake_git, <<~'RUBY') + #!/usr/bin/env ruby + STDOUT.binmode + path = "raw-\xFF.txt".b + if ARGV.include?("rev-parse") + repo_index = ARGV.index("-C") + 1 + STDOUT.write(ARGV.fetch(repo_index) + "\n") + elsif ARGV.include?("--stage") + STDOUT.write("100644 #{'a' * 40} 0\t".b + path + "\0".b) + elsif ARGV.include?("cat-file") + STDOUT.sync = true + content = "gh" + "p_" + ("A" * 24) + while (object_id = STDIN.gets&.chomp) + STDOUT.write("#{object_id} blob #{content.bytesize}\n") + STDOUT.write(content) + STDOUT.write("\n") + end + elsif ARGV.include?("-co") + STDOUT.write(path + "\0".b) + else + abort "unexpected git invocation: #{ARGV.join(' ')}" + end + RUBY + FileUtils.chmod(0o755, fake_git) + + _stdout, stderr, status = run_checker( + repo, + { "PATH" => [bin_dir, ENV.fetch("PATH")].join(File::PATH_SEPARATOR) } + ) + + refute status.success? + assert_includes stderr, "GitHub token" + assert_includes stderr, "\\xFF" + refute_includes stderr, "invalid byte sequence" + refute_includes stderr, token + end + end + end + + def test_git_lfs_pointer_requires_separate_review + with_repo do |repo| + stage(repo, "large.dat", lfs_pointer) + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "large.dat" + assert_includes stderr, "Git LFS object requires separate review" + refute_includes stderr, "sha256:" + end + end + + def test_unstaged_git_lfs_pointer_requires_separate_review + with_repo do |repo| + stage(repo, "large.dat", "safe candidate\n") + write(repo, "large.dat", lfs_pointer) + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "large.dat" + assert_includes stderr, "Git LFS object requires separate review" + refute_includes stderr, "sha256:" + end + end + + def test_bom_marked_utf16_source_requires_separate_review + with_repo do |repo| + token = github_token + content = "\xFF\xFE".b + token.encode("UTF-16LE").b + stage(repo, "script.ps1", content) + + _stdout, stderr, status = run_checker(repo) + + refute status.success? + assert_includes stderr, "script.ps1" + assert_includes stderr, "BOM-marked UTF-16 source requires separate review" + refute_includes stderr, token + end + end + + def test_unresolved_index_entries_fail_closed + with_repo do |repo| + object_ids = %w[base ours theirs].map do |content| + output, error, status = Open3.capture3( + "git", + "-C", + repo, + "hash-object", + "-w", + "--stdin", + stdin_data: content + ) + assert status.success?, error + output.strip + end + index_info = object_ids.each_with_index.map do |object_id, index| + "100644 #{object_id} #{index + 1}\tconflict.txt\n" + end.join + _output, error, status = Open3.capture3( + "git", + "-C", + repo, + "update-index", + "--index-info", + stdin_data: index_info + ) + assert status.success?, error + + _stdout, stderr, checker_status = run_checker(repo) + + refute checker_status.success? + assert_includes stderr, "conflict.txt" + assert_includes stderr, "unresolved Git index entry" + end + end +end diff --git a/scripts/verify.sh b/scripts/verify.sh index f4c15f2..7ac71bb 100755 --- a/scripts/verify.sh +++ b/scripts/verify.sh @@ -60,13 +60,13 @@ if (JSON.stringify(documentedOwned) !== JSON.stringify([...owned].sort())) { fail("README.md owned skill list does not match bootstrap.sh"); } -if (owned.length !== 15 || standalone.length !== 1 || asc.length !== 22) { - fail(`Expected 15 owned, 1 standalone, and 22 ASC skills; found ${owned.length}, ${standalone.length}, and ${asc.length}`); +if (owned.length !== 16 || standalone.length !== 1 || asc.length !== 22) { + fail(`Expected 16 owned, 1 standalone, and 22 ASC skills; found ${owned.length}, ${standalone.length}, and ${asc.length}`); } const managed = [...owned, ...standalone, ...asc]; -if (managed.length !== 38 || new Set(managed).size !== managed.length) { - fail("The 38-skill global baseline contains a missing or duplicate name"); +if (managed.length !== 39 || new Set(managed).size !== managed.length) { + fail("The 39-skill global baseline contains a missing or duplicate name"); } console.log("validated direct package contract"); @@ -98,33 +98,8 @@ end puts "validated skill front matter YAML" RUBY -ruby <<'RUBY' -patterns = { - "machine-local home path" => %r{/(?:Users|home)/[A-Za-z0-9._-]+(?:/|\b)}, - "AWS access key" => /AKIA[0-9A-Z]{16}/, - "bearer credential" => /Authorization:\s*Bearer\s+[A-Za-z0-9._~-]{16,}/i, - "GitHub token" => /gh[pousr]_[A-Za-z0-9]{20,}/, - "API secret" => /sk-(?:proj-)?[A-Za-z0-9_-]{20,}/, - "private key" => /-----BEGIN (?:RSA )?PRIVATE KEY-----/ -} - -files = IO.popen(["git", "ls-files", "-co", "--exclude-standard", "-z"], &:read).split("\0") -failures = files.sort.filter_map do |file| - next unless File.file?(file) && !File.symlink?(file) - - content = File.binread(file) - next if content.include?("\0") - - text = content.force_encoding(Encoding::UTF_8) - next unless text.valid_encoding? - - labels = patterns.filter_map { |label, pattern| label if text.match?(pattern) } - "#{file}: #{labels.join(", ")}" unless labels.empty? -end - -abort "public-safety scan failed:\n#{failures.join("\n")}" unless failures.empty? -puts "public-safety scan passed" -RUBY +ruby public-source-release-audit/tests/test_check_current_source.rb +ruby public-source-release-audit/scripts/check_current_source.rb . while IFS= read -r -d '' script; do [[ "$script" == *.sh && -f "$script" && ! -L "$script" ]] || continue