chore: prepare repo for public release - #73
Merged
Conversation
…valents Problem. Benchmark docs, ADRs, tests, and configs quoted a real issuer by name along with real audited balance-sheet figures taken from a customer document. Configs and doc links also embedded an absolute path from a development machine, which leaked a local directory layout and username and broke every one of those links for anyone else. What changed. The issuer is now Contoso, the standard fictional company. All sixteen real figures were identified by diffing against the source document rather than by eye, and replaced with perturbed values verified not to collide with the original. Two of them also appeared without thousands separators in test fixtures; those are covered too. The audit firm's PCAOB identifier and the incorporation date are likewise replaced. Absolute paths become repo-relative, and markdown links become root-relative so they resolve from any directory depth. Sample document filenames are now neutral. Contoso is deliberately a single word. The entity-audit tests in test_llm_dataset_generator.py assert on how a multi-word entity's first token is handled, so a two-word replacement would have quietly changed what those tests prove. What did NOT change. No behavior. The corpus codenames avatar, pj-doc, and f1-doc are kept: they identify nothing once the company name and figures are gone, and renaming them would churn config filenames, scripts, and collection names for no privacy gain. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem. The repo had no contributor-facing docs. The PR template cited docs/contributing.md §3 and §3.1, a file that never existed. The README advertised configs/chunker.yml and configs/chunker-ragflow.yml, neither of which exists, linked to the gitignored data/ directory, and carried a phasing table claiming v0.1 was current while the project sits at v0.5.3 — the same README both used the Xrag client in its quickstart and said that client "ships in v0.2". What changed. CONTRIBUTING.md is numbered so the PR template's §3 and §3.1 references resolve, and it records the setup, checks, commit conventions, and eval discipline the project already follows. SECURITY.md routes vulnerability reports to private advisories and documents scope and credential handling. CODE_OF_CONDUCT.md is Contributor Covenant 2.1. CHANGELOG.md is reconstructed from the existing tags. The README's dead config links now point at configs/chunkers/, the data/ link is prose, and the stale phasing table is replaced by a status section describing what ships today versus what is planned. What did NOT change. No behavior, no public API. Code of Conduct enforcement routes through GitHub private advisories rather than an email address, since publishing a personal address is the maintainer's call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tomation Problem. A fresh clone could not run the test suite. make unit-test ran pytest against a bare uv sync, but tests/unit/test_vector_store.py imports chromadb, so collection failed before a single test ran. CI passed only because its unit-test job adds the extras by hand. This was the first thing a new contributor would hit. What changed. A DEV_EXTRAS variable now feeds both make sync and make unit-test, matching the CI job and commented as such so the two stay in step. Dropped a ruff per-file-ignore for news.py, a file not in the repo. Added issue templates that route security reports away from public issues, and a dependabot config that groups routine bumps; LangChain majors are ignored there because they are pinned deliberately and exercised by the contract matrix in ci.yml. What did NOT change. CI job definitions, and no runtime behavior. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Problem. `pre-commit run --all-files` had never been run to completion, so two hooks failed on code already in main. end-of-file-fixer flagged seven files — three survey docs missing a trailing newline, and CLAUDE.md plus two parser scripts carrying extra trailing blank lines. gitleaks reported three findings, all false positives, which meant the secret-scanning hook failed for every contributor who installed it. What changed. Applied the end-of-file fixes. Added .gitleaksignore with the three reviewed false positives: `dedup_family=True`, a retrieval config flag that matches the generic-api-key shape, and two `Authorization: Bearer rk_live_...` lines in an API design document whose trailing ellipsis is literal. Each entry records why the match is safe, and the file warns that fingerprints are line-sensitive and must be re-verified rather than blindly refreshed. `pre-commit run --all-files` now passes clean, so the hook suite is trustworthy before the repo goes public. What did NOT change. No behavior. gitleaks still reports the same two placeholder matches when scanning full history at their pre-rename path; they are the same reviewed false positives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The pinned v8.21.2 builds a binary that recent macOS dyld refuses to load, aborting with "missing LC_UUID load command". pre-commit surfaces that as exit -6, which reads like a hook failure rather than the crash it is — so the secret scanner appeared to be running and failing when it was never running at all. Bumped to v8.30.1, which loads and passes, and left a comment recording the failure mode so the next person to see exit -6 knows to bump rather than to hunt for a leak. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
xrag.__all__is untouched.Confidential data
The real figures were identified by diffing against the source document rather than by eye, which found 16 — not the 6 visible on inspection. Two also appeared without thousands separators in test fixtures.
Contoso. Deliberately a single word: the entity-audit tests intest_llm_dataset_generator.pyassert on how a multi-word entity's first token is handled, so a two-word name would have quietly changed what those tests prove.Corpus codenames (
avatar,pj-doc,f1-doc) are kept — they identify nothing once the company name and figures are gone, and renaming would churn config filenames, scripts, and collection names for no privacy gain.Contributor-facing fixes
make sync/make unit-testnow install the extras the tests import. Previouslytest_vector_store.pyfailed collection onchromadbbefore a single test ran; CI passed only because its job added the extras by hand.data/, and a phasing table claiming v0.1 while the project sits at v0.5.3 — the same README used theXragclient in its quickstart and said that client "ships in v0.2".docs/contributing.md§3/§3.1, a file that never existed.CONTRIBUTING.mdis numbered so those references now resolve.Pre-commit
pre-commit run --all-fileshad never completed. Two hooks failed on code already inmain:v8.21.2builds a binary recent macOS dyld refuses to load; pre-commit surfaced the abort as exit-6, which reads like a finding. Bumped tov8.30.1. Once actually running, it reported 3 false positives — now documented in.gitleaksignorewith the reasoning, and a warning that fingerprints are line-sensitive and must be re-verified rather than blindly refreshed.Test plan
make lintmake unit-test— 609 passedpre-commit run --all-files— 0 failuresgitleaks dir— no leaks;gitleaks gitover full history — only the 2 reviewedrk_live_...placeholders at their pre-rename pathRisks
This PR does not remove anything from git history. The source document is still a reachable blob, and the old strings remain in past commits. A
git filter-reporewrite must run before the repo goes public — I dry-ran it and diffed its output against this branch: every scrubbed file is byte-identical, so the rewrite reproduces exactly this tree.Force-pushing is also not sufficient on its own. GitHub retains unreachable objects until support garbage-collects them, so the blob stays fetchable by SHA until that ticket is closed.
AI assistance
Claude — fully tested; all gates above run and passing.