Skip to content

fix: Doc-drift review follow-ups (precheck worklist, scanner test) - #132

Open
rubambiza wants to merge 3 commits into
rossoctl:mainfrom
rubambiza:fix/doc-drift-precheck-worklist-and-scanner-test
Open

rubambiza wants to merge 3 commits into
rossoctl:mainfrom
rubambiza:fix/doc-drift-precheck-worklist-and-scanner-test

Conversation

@rubambiza

Copy link
Copy Markdown
Contributor

Summary

Two doc-drift review follow-ups from the #124 / rossoctl/repoman#12 review threads, folded into one PR under that shared purpose.

#130 — precheck worklist defaults to a persistent path. doc-drift-precheck.sh defaulted its worklist into the per-run temp dir that setup_workspace's EXIT trap deletes, so a bare run (no --worklist) printed worklist: <path> for a file already gone by the time the caller read it — fail-open in the worst direction. It now defaults to ${REPORTS_DIR:-$HOME/reports/doc-drift}/worklist.json (mirroring the scanner), mkdir -ps it, and states the contract in usage. --worklist still overrides.

#128 — behavioural test for the scanner. doc-drift-scanner.sh was the only doc-drift component with no behavioural test (just bash -n + shellcheck). Adds tests/test-doc-drift-scanner.sh driving the real scanner with gh stubbed:

  • new findings → gh issue create with [doc-drift] owner/name
  • checker failure → prior findings carried into latest.json, issue not closed, exit 0, repos_failed==1
  • findings cleared → existing issue closed
  • --dry-run → no issue create/edit/close, latest.json not written
  • --issue-limit 1 → stops after one repo
  • latest.json shape (repos_scanned/corrections/additions/delta) + one history.json row

Registered in both CI lists (shellcheck group + run loop) — the lists are explicit, so an unlisted test never runs.

Testing

Fixes #130
Fixes #128

Assisted-By: Claude Code

A bare run (no --worklist) defaulted the worklist into the per-run temp
dir that setup_workspace's EXIT trap deletes, so precheck printed a
'worklist: <path>' the caller could not read (fail-open in the worst
direction). Default to ${REPORTS_DIR:-$HOME/reports/doc-drift}/worklist.json
instead, mkdir -p it, and state the contract in usage. Adds a regression
test asserting the default path persists after the script returns.

Fixes rossoctl#130

Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
doc-drift-scanner.sh was the only doc-drift component with no behavioural
test (only bash -n + shellcheck). Add tests/test-doc-drift-scanner.sh
driving the real scanner with gh stubbed: issue create on new findings,
carry-forward + issue-not-closed on checker failure, close when findings
clear, dry-run writes no latest.json and no issue calls, --issue-limit
stops early, and the latest/history report shape. Register it in both CI
lists (shellcheck + run loop).

Fixes rossoctl#128

Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
@rubambiza rubambiza added the ready-for-ai-review Request automated AI code review from clawgenti label Nov 5, 2026

@clawgenti clawgenti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean follow-up pair: the #130 fix correctly defaults the precheck worklist to a persistent path (verified against setup_workspace's EXIT trap, with a real regression test), and the new scanner behavioural test drives the real scanner with stubbed gh across create / carry-forward / close / dry-run / issue-limit / report-shape paths — both tests pass locally and CI is green. Two nits below.

  • nit scripts/doc-drift-precheck.sh:27 — --help renders literal backslashes
  • nit tests/test-doc-drift-scanner.sh:303 — grep -c ... || echo 0 double-prints on the zero-match path

Reviewed by clawgenti using the github-pr-review skill

Comment thread scripts/doc-drift-precheck.sh Outdated
exit 0 and print "worklist: <path>" when it is not. Fetch failures are
exit 0 and print "worklist: <path>" when it is not. The printed path persists
after the run. --worklist overrides the default of
\${REPORTS_DIR:-\$HOME/reports/doc-drift}/worklist.json. Fetch failures are

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The usage heredoc is quoted (<<'EOF'), so ${...} never expands — the backslashes here are unnecessary and render literally in --help output: \${REPORTS_DIR:-\$HOME/reports/doc-drift}/worklist.json. Dropping the escapes prints the intended plain form.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cf08866 — dropped the backslash escapes; the single-quoted heredoc now renders the plain ${REPORTS_DIR:-$HOME/reports/doc-drift}/worklist.json in --help.

Comment thread tests/test-doc-drift-scanner.sh Outdated
else
fail_msg "case5: --issue-limit exit $rc5"
fi
c5creates=$(grep -c "issue create" "$C5_LOG" 2>/dev/null || echo 0)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: grep -c already prints 0 (with exit status 1) when nothing matches, so the || echo 0 fallback appends a second 0 — on the failure path c5creates becomes two lines and the [ "$c5creates" -le 1 ] check below errors with "integer expression expected" instead of reaching the intended fail_msg. Dropping || echo 0 (or using || true) keeps that path clean.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in cf08866 — switched || echo 0 to || true, so the no-match path yields a single 0 and the -le integer check stays well-formed.

- precheck --help: drop backslash escapes inside the single-quoted
  heredoc; they rendered literally instead of as $REPORTS_DIR/$HOME.
- scanner test: grep -c already prints 0 on no-match, so '|| echo 0'
  double-printed and could break the -le integer check; use '|| true'.

Assisted-By: Claude Code (Anthropic AI) <noreply@anthropic.com>

Signed-off-by: Gloire Rubambiza <gloire@ibm.com>
@rubambiza rubambiza self-assigned this Nov 5, 2026

@clawgenti clawgenti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit-fix commit cf08866 correctly addresses both findings from the earlier review — the --help heredoc now renders ${REPORTS_DIR:-$HOME/...} literally, and the case-5 count uses || true so the zero-match path stays a clean integer — with both test suites passing (verified locally against the PR head) and CI green on both platforms.

All checks pass. Ready for human review.


Reviewed by clawgenti using the github-pr-review skill

@rubambiza rubambiza added ready-for-human-review AI review passed, ready for human reviewer and removed ready-for-ai-review Request automated AI code review from clawgenti labels Nov 5, 2026

@jimcadden jimcadden left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed cf08866. No blocking bugs or security findings found. The persistent worklist default fixes the deleted-temp-file problem, and its regression passes. The new scanner tests cover issue creation, failure carry-forward, closure, dry-run, issue limiting, and report structure; both explicit CI lists include the test. Two nonblocking test improvements are included inline.

Author: rubambiza (MEMBER, maintainer).
Areas reviewed: shell, tests, CI/YAML.
Agent/IDE config (.claude/.vscode): none.
Commits: 3, all signed off.
CI status: 4/4 checks passing, including Ubuntu and macOS.
Verification: precheck, scanner, and checker test suites passed locally; shell syntax checks and git diff --check passed. Local ShellCheck was unavailable; CI runs it.

Generated by RepoMan github-pr-review@235a7f7 (https://github.com/rossoctl/agent-skills/blob/235a7f7620139ee3e4ab0872b8439591fe52e8c1/skills/github-pr-review/SKILL.md).

fail_msg "case5: --issue-limit exit $rc5"
fi
c5creates=$(grep -c "issue create" "$C5_LOG" 2>/dev/null || true)
if [ "${c5creates:-0}" -le 1 ]; then

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Assert exactly one issue create rather than <= 1. With two drifting fixtures, the current assertion also passes if --issue-limit 1 incorrectly suppresses all issue creation. Assert repos_scanned == 2 as well to guard against limiting the scan instead of issue writes.

else
fail_msg "case4: --dry-run exit $rc4"
fi
if grep -Eq "issue (create|edit|close)" "$C4_LOG"; then

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Extend the dry-run guard to reject issue comment and label create, and seed/assert unchanged history.json. The current case protects issue create/edit/close and latest.json, but could miss other remote writes or history changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human-review AI review passed, ready for human reviewer

Projects

None yet

3 participants