Skip to content

fix: the discoverability checker reported success without scanning an… #4

fix: the discoverability checker reported success without scanning an…

fix: the discoverability checker reported success without scanning an… #4

Workflow file for this run

name: CI
# ---------------------------------------------------------------------------
# GitHub Actions. Nothing to enable — Actions is on by default for this
# repo, and `ubuntu-latest` is a GitHub-hosted runner.
#
# Two things do need setting by hand:
#
# 1. The `integration` and `network-anonymous` jobs need repository
# secrets DA_CLIENT_ID and DA_CLIENT_SECRET
# (Settings -> Secrets and variables -> Actions). Without them those
# jobs skip rather than fail — see the `if:` guards below.
#
# 2. Branch protection on `main` should require these checks AND
# "Require branches to be up to date before merging". Without the
# up-to-date requirement a green PR can merge into a base that has
# moved and break main — which is exactly what happened once on the
# previous forge (a PR deleted an import as unused while another PR
# added the first use of it; no textual conflict, both green).
#
# GITHUB_TOKEN permissions are set to least privilege at the top level
# below and raised per-job only where a job genuinely needs more.
# ---------------------------------------------------------------------------
on:
push:
branches: [main]
# No base-branch filter on pull_request: stacked PRs (a PR whose base
# is another feature branch) must get CI too, not just PRs into main.
pull_request:
schedule:
# Weekly pip-audit job — Monday 06:00 UTC. Catches CVEs in the dev
# toolchain (ruff/mypy/pytest transitive deps). Runtime has 0 deps.
- cron: "0 6 * * 1"
workflow_dispatch:
# Manual trigger for the pip-audit job (or any other) without waiting
# for the weekly schedule.
# Four jobs run in parallel so a failure in one category (lint vs. unit
# tests vs. integration tests vs. smoke) is immediately visible in the
# runs UI without scrolling through a single big log.
#
# Runtime is stdlib-only (no `pip install` required to *use* da-cli),
# but the dev toolchain — ruff, mypy, pytest, pytest-cov — is needed
# for CI. We install via the `dev` extra:
#
# pip install -e '.[dev]'
#
# The test job runs the full Python 3.10–3.14 matrix da-cli claims to
# support (see pyproject.toml `requires-python` + classifiers). Runtime
# is stdlib-only, so the matrix's only job is to surface version-specific
# regressions (e.g. argparse / sqlite3 / ssl changes between minor
# releases).
# Least privilege: the default GITHUB_TOKEN for a public repo can be
# broader than any job here needs. Every job in this file only reads the
# checkout; jobs that need more raise it themselves.
permissions:
contents: read
# A new push to the same ref makes the in-flight run obsolete. Cancelling
# it keeps queue times honest on a shared runner pool. `main` is excluded
# from cancellation so a push-to-main run always completes.
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install dev deps
run: |
python -m pip install --upgrade pip
pip install -e '.[dev]'
- name: ruff check
run: ruff check .
- name: ruff format --check
run: ruff format --check .
- name: mypy
run: mypy dacli
- name: Documented flags and defaults match the parser
# The per-command guides carry a table per command. A wrong
# default is worse than vague prose: the reader acts on it.
run: python3 tools/check_doc_flags.py
- name: Documentation references resolve
# Catches prose that confidently describes a renamed command, a
# dropped flag, or a moved file. Link checkers miss these because
# they are not links.
run: python3 tools/check_doc_references.py
- name: Generated docs are current
# The CLI reference is generated from build_parser(). If someone
# adds a flag without running `make docs`, this fails — which is
# the only thing that stops a reference from silently rotting.
run: |
python3 tools/gen_cli_docs.py
git diff --exit-code docs/reference/cli.md \
|| { echo "::error::docs/reference/cli.md is stale — run 'make docs'"; exit 1; }
test:
strategy:
fail-fast: false
matrix:
# Cover every Python version da-cli claims to support
# (pyproject.toml `requires-python` + classifiers).
# Single OS — the macOS-specific paths (`_keychain_*` via
# /usr/bin/security, launchctl probe in diagnose) are unit-tested
# locally; add a `macos-latest` leg here once a macOS runner is
# registered with that label.
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dev deps
run: |
python -m pip install --upgrade pip
pip install -e '.[dev]'
- name: Test suite + coverage gate
# Covers the unit tests and tests/test_integration.py (the
# MOCKED end-to-end suite). The live tests in tests/integration/
# are collected but skipped here — they need credentials and run
# in the cassette-replay / network-anonymous jobs below.
# This job is the canonical coverage check.
run: pytest -q
- name: Upload coverage report
# Only one leg uploads the artifact to avoid duplicates.
if: always() && matrix.python-version == '3.13'
# v4: v3 was retired 2025-01-30 and now fails outright. The old
# v3 pin existed only because Gitea's artifact server implemented
# the v3 API and rejected v4 with GHESNotSupportedError.
uses: actions/upload-artifact@v4
# Never fail the build on this. The coverage GATE is enforced inside
# the test step by pytest-cov's --cov-fail-under, so by the time we
# get here the number has already passed or failed; this upload only
# exists so a human can download the report. It is also the one step
# that depends on a shared, account-wide resource — the first run of
# this repo failed here with "Artifact storage quota has been hit",
# which said nothing about the commit under test.
continue-on-error: true
with:
name: coverage-report
path: coverage.xml
retention-days: 7
if-no-files-found: warn
integration:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install dev deps
run: |
python -m pip install --upgrade pip
pip install -e '.[dev]'
- name: Integration test count
run: |
n=$(pytest tests/test_integration.py --collect-only -q --no-header 2>/dev/null \
| grep -c '::test_' || true)
echo "::notice::Integration test count: $n"
- name: Integration tests (verbose for visibility)
# Disable coverage gate here — the test job already enforces
# the 92% bar across the full suite; running just the
# integration subset would naturally have lower coverage.
run: pytest tests/test_integration.py -v --tb=short --no-cov
smoke:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Run `da --version` (stdlib-only — no deps needed)
run: python3 da --version
- name: Run `da --help`
run: python3 da --help
- name: Run `da config show` against a fresh empty config
run: |
# Use an empty XDG_CONFIG_HOME so the smoke is hermetic
tmp=$(mktemp -d)
XDG_CONFIG_HOME="$tmp/cfg" XDG_STATE_HOME="$tmp/state" \
python3 da config show
- name: Run `da diagnose` against a fresh empty config
run: |
# Should exit 2 (client_id missing → fail) but not crash
tmp=$(mktemp -d)
XDG_CONFIG_HOME="$tmp/cfg" XDG_STATE_HOME="$tmp/state" \
python3 da diagnose && exit_code=0 || exit_code=$?
if [ "$exit_code" -ne 2 ]; then
echo "::error::Expected da diagnose to exit 2 on empty config, got $exit_code"
exit 1
fi
echo "::notice::da diagnose correctly exited 2 on empty config"
- name: Perf floor — `da bench` items/sec must clear 500
run: |
# `da bench` is a synthetic sync against a fully mocked HTTP
# layer — no network. Measures CLI overhead (parse, index,
# file IO, thread-pool dispatch). A typical dev laptop
# clears 2,500+ items/sec; the floor here is intentionally
# conservative (500) to catch genuine perf regressions
# (e.g. accidentally O(n²) index lookup) without flapping
# on CI noise.
tmp=$(mktemp -d)
out=$(XDG_CONFIG_HOME="$tmp/cfg" XDG_STATE_HOME="$tmp/state" \
python3 da bench --pages 10 --per-page 24 --concurrency 4 --json)
echo "$out"
ips=$(echo "$out" | python3 -c "import json,sys; print(int(json.load(sys.stdin)['items_per_sec']))")
if [ "$ips" -lt 500 ]; then
echo "::error::Bench items_per_sec=$ips is below floor (500)"
exit 1
fi
echo "::notice::Bench items_per_sec=$ips (floor 500) OK"
artifact:
# Every other job runs da out of the checkout, where `import dacli`
# resolves to the source tree and therefore always works. That masks
# packaging faults completely: a wheel missing a subpackage, or an
# installer that flattens one, passes all of them. This job runs only
# what a user would actually receive.
runs-on: ubuntu-latest
needs: test
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Build wheel + sdist
run: |
python -m pip install --upgrade pip build
# setuptools reuses build/lib if present, which can carry a
# previously-packaged module into a wheel whose config no longer
# selects it — a stale pass exactly where this job must not give
# one. A fresh checkout has none, but make it explicit.
rm -rf build dist ./*.egg-info
python -m build
- name: Wheel contains every subpackage in the source tree
run: |
# Catches the explicit-list packaging fault directly: any
# dacli/ subpackage that exists in git but not in the wheel.
python3 - <<'EOF'
import glob, pathlib, sys, zipfile
src = {
str(p.parent.relative_to(".")).replace("/", ".")
for p in pathlib.Path("dacli").rglob("__init__.py")
}
whl = zipfile.ZipFile(glob.glob("dist/*.whl")[0])
packed = {
str(pathlib.PurePosixPath(n).parent).replace("/", ".")
for n in whl.namelist() if n.endswith("__init__.py")
}
missing = sorted(src - packed)
if missing:
print(f"::error::wheel is missing subpackage(s): {missing}")
sys.exit(1)
print(f"wheel ships all {len(src)} package(s): {sorted(src)}")
EOF
- name: Install the wheel into a clean venv and run it
run: |
python3 -m venv /tmp/venv
/tmp/venv/bin/pip install --quiet dist/*.whl
# cd out of the repo: from here, the source tree cannot shadow
# the installed package, so an import fault actually surfaces.
cd /tmp
/tmp/venv/bin/da --version
/tmp/venv/bin/da --help > /dev/null
# Import every shipped module — a missing submodule that only
# one subcommand touches would otherwise slip through.
/tmp/venv/bin/python - <<'EOF'
import importlib, pkgutil, dacli
mods = [m.name for m in pkgutil.walk_packages(dacli.__path__, "dacli.")]
for m in mods:
importlib.import_module(m)
print(f"imported {len(mods)} submodules from the installed wheel")
EOF
- name: Installed wheel — every subcommand parses
run: |
cd /tmp
# `--help` on each leaf command forces the subparser to be
# constructed, which is where a missing import shows up.
for c in auth sync search config index deviation user watch \
daily whoami refresh diagnose bench; do
/tmp/venv/bin/da "$c" --help > /dev/null \
|| { echo "::error::da $c --help failed from the wheel"; exit 1; }
done
echo "::notice::all 14 command groups parse from the installed wheel"
- name: install.sh produces a working `da`
run: |
tmp=$(mktemp -d)
DA_INSTALL_PREFIX="$tmp" ./install.sh
# By absolute path, never `da` off PATH — resolving through PATH
# can silently test a different copy than the one just written.
cd /tmp
"$tmp/bin/da" --version
"$tmp/bin/da" config --help > /dev/null
# The installer copies files itself, so verify it preserved the
# tree rather than flattening subpackages onto their parents.
diff <(cd "$OLDPWD" && find dacli -name '*.py' \
-not -path '*__pycache__*' | sort) \
<(cd "$tmp/share/da-cli" && find dacli -name '*.py' | sort) \
|| { echo "::error::install.sh did not mirror dacli/ faithfully"; exit 1; }
echo "::notice::install.sh output matches the source tree"
# ---------------------------------------------------------------------------
# Security / supply-chain jobs. Run on every push + PR; cheap, catches
# real bugs that lint+type+test can't.
# ---------------------------------------------------------------------------
secret-scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
# gitleaks needs full history to scan past commits, not just HEAD.
fetch-depth: 0
- name: Run gitleaks
# The gitleaks CLI directly, not gitleaks-action@v2: the action
# requires a GITLEAKS_LICENSE for org-owned repos, while the
# CLI itself is MIT. The exit code still gates CI.
# `gitleaks git` scans the full history (`detect` is deprecated
# since v8.19). Custom rules live in .gitleaks.toml, picked up
# automatically from the repo root.
run: |
case "$(uname -m)" in
aarch64|arm64) ARCH=arm64 ;;
*) ARCH=x64 ;;
esac
curl -sL "https://github.com/gitleaks/gitleaks/releases/download/v8.30.1/gitleaks_8.30.1_linux_${ARCH}.tar.gz" \
| tar -xz -C /tmp gitleaks
/tmp/gitleaks git --redact --verbose --exit-code 1 .
verified-secret-scan:
# TruffleHog complements gitleaks: its detectors VERIFY candidate
# credentials against the issuing provider's API (read-only), so a
# finding here means a currently-live secret, not a lookalike.
# --results=verified,unknown keeps the job quiet on dummies and
# docs placeholders. The CLI directly rather than the marketplace
# action: the action is Docker-based and needs a runner with
# Docker-action support; the plain binary runs anywhere (same
# approach as the gitleaks job above). --fail → exit 183 on any
# reportable finding, which gates CI.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: TruffleHog (verified/unknown findings only)
run: |
case "$(uname -m)" in
aarch64|arm64) ARCH=arm64 ;;
*) ARCH=amd64 ;;
esac
curl -sSfL "https://github.com/trufflesecurity/trufflehog/releases/download/v3.96.0/trufflehog_3.96.0_linux_${ARCH}.tar.gz" \
| tar -xz -C /tmp trufflehog
/tmp/trufflehog git file://. --results=verified,unknown --fail --no-update
codespell:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install codespell
# Into a venv: the runner image's system Python is externally
# managed (PEP 668) and rejects a bare `pip install`.
run: |
python3 -m venv /tmp/codespell-venv
/tmp/codespell-venv/bin/pip install --quiet codespell
echo "/tmp/codespell-venv/bin" >> "$GITHUB_PATH"
- name: Run codespell
# codespell catches typos in docs, code, and commit messages.
# Codespell's default dictionary is conservative; if it flags
# something intentional (e.g. a DA API field name), add it to
# the --ignore-words-list below rather than disabling the rule.
run: |
codespell \
--skip="./.git,./.venv-dev,./.ruff_cache,./.mypy_cache,./.pytest_cache,*.pyc,./coverage.xml,./da_cli.egg-info" \
--ignore-words-list="deviantart,userdata,signin,fave,faves" \
--check-filenames \
--check-hidden
pip-audit:
# Runs weekly (not on every push — too slow for the main pipeline).
# Catches CVEs in the dev toolchain (pytest/ruff/mypy transitive
# deps). da-cli itself has 0 runtime deps, so this is purely about
# the dev environment.
runs-on: ubuntu-latest
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install dev deps
run: |
python -m pip install --upgrade pip
pip install -e '.[dev]'
pip install pip-audit
- name: Audit dev dependencies
# Only scan the dev extras (runtime has 0 deps). pip-audit's
# exit code is non-zero if any CVE matches.
run: pip-audit --strict --desc
# ---------------------------------------------------------------------------
# Network integration tests (anonymous — CI-runnable forever)
# ---------------------------------------------------------------------------
markdownlint:
# The repo has configured markdownlint in .pre-commit-config.yaml
# since day one, but pre-commit never ran in CI — so the rules were
# advisory and the corpus had drifted to 56 violations. Run it here
# so "configured" and "enforced" mean the same thing.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: markdownlint
# Pinned to the same version .pre-commit-config.yaml uses, so
# local and CI agree. Newer versions add rules (e.g. MD060) that
# would fail a tree the pinned hook considers clean.
run: npx --yes markdownlint-cli@0.45.0 --config .markdownlint.yaml $(git ls-files '*.md')
link-check:
# Internal doc links only (relative file paths + #anchors) —
# deterministic, no network, gates every push/PR. External URLs are
# covered by link-check-external below.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install lychee
run: |
case "$(uname -m)" in
aarch64|arm64) ARCH=aarch64 ;;
*) ARCH=x86_64 ;;
esac
curl -sSfL "https://github.com/lycheeverse/lychee/releases/download/lychee-v0.24.2/lychee-${ARCH}-unknown-linux-gnu.tar.gz" \
| tar -xz -C /tmp --strip-components=1 "lychee-${ARCH}-unknown-linux-gnu/lychee"
- name: Check internal doc links and anchors
run: /tmp/lychee --config lychee.toml --offline --include-fragments --no-progress '**/*.md'
link-check-external:
# Live URLs go stale on the internet's schedule, not the repo's —
# weekly + manual trigger so a third-party outage can't block
# unrelated PRs. Exclusions and accepted status codes live in
# lychee.toml.
runs-on: ubuntu-latest
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
steps:
- uses: actions/checkout@v4
- name: Install lychee
run: |
case "$(uname -m)" in
aarch64|arm64) ARCH=aarch64 ;;
*) ARCH=x86_64 ;;
esac
curl -sSfL "https://github.com/lycheeverse/lychee/releases/download/lychee-v0.24.2/lychee-${ARCH}-unknown-linux-gnu.tar.gz" \
| tar -xz -C /tmp --strip-components=1 "lychee-${ARCH}-unknown-linux-gnu/lychee"
- name: Check all doc links (external included)
run: /tmp/lychee --config lychee.toml --no-progress '**/*.md'
cassette-replay:
# The contract layer: replays recorded DA responses from
# tests/integration/cassettes/. No network, no credentials, ~5 s —
# so unlike the live jobs below it can gate every push, and it is
# what actually catches a DA response-shape change.
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install dev + integration deps
run: |
python -m pip install --upgrade pip
pip install -e '.[dev,integration]'
- name: Cassette replay tests
run: pytest -m integration_cassette -v --no-cov
- name: Assert the marker still selects tests
# Guards the regression this suite was built to catch: an
# addopts --ignore once reduced every `-m integration*` run to
# zero collected tests, so the live job reported green while
# executing nothing. A count of 0 must fail, not pass.
run: |
n=$(pytest -m integration_cassette --collect-only -q --no-cov 2>/dev/null \
| grep -oE '^[0-9]+' | head -1)
if [ -z "$n" ] || [ "$n" -eq 0 ]; then
echo "::error::pytest -m integration_cassette collected 0 tests"
exit 1
fi
echo "::notice::integration_cassette collects $n tests"
network-anonymous:
# Uses client_credentials grant — no human, no 90-day token expiry.
# Needs DA_CLIENT_ID + DA_CLIENT_SECRET as GitHub Actions secrets.
# If those secrets aren't set, the tests skip gracefully.
#
# This is the only job that puts a real credential in the environment
# and then runs pytest — i.e. code from the branch under test. On a
# public repo anyone can open a PR, so the guard below refuses to run
# it for a pull request originating from a fork.
#
# GitHub already withholds secrets from fork PRs, so this is belt and
# braces rather than a fix. It is here because that protection is
# implicit: nothing in the file said "this job must never see
# untrusted code with a live secret", and the next person adding a
# secret to another job has no reason to know.
if: >-
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
needs: test # don't bother if unit tests are broken
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Install dev + integration deps
run: |
python -m pip install --upgrade pip
pip install -e '.[dev,integration]'
- name: Anonymous endpoint tests
env:
DA_CLIENT_ID: ${{ secrets.DA_CLIENT_ID }}
DA_CLIENT_SECRET: ${{ secrets.DA_CLIENT_SECRET }}
# Don't fail the whole workflow if DA is down or creds aren't set.
# The test fixtures handle the skip gracefully; surfacing a failure
# here only matters once secrets are actually configured.
continue-on-error: true
run: pytest -m integration_anonymous -v --no-cov
discoverability:
# Guards the metadata that makes the project findable at all. GitHub's
# default repo search matches only name, description and topics — not the
# README — so those fields are the entire discovery surface, and they rot
# silently: nothing else in CI notices a description that stopped being
# true or a `Typing :: Typed` classifier with no py.typed beside it.
#
# Offline mode only. The --github rules need a token and query live API
# metadata, which is not a property of the commit under test.
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
with:
# No token in the environment: this job runs no privileged step and
# a checkout that leaves credentials behind is a needless one.
persist-credentials: false
- uses: actions/setup-python@v5
with:
python-version: "3.13"
- name: Discoverability and metadata checks
run: python3 tools/check_discoverability.py
ci-gate:
# One check to require in branch protection, instead of fifteen.
#
# Two problems this solves. First, the `test` job is a matrix, so it
# reports as `test (3.10)` ... `test (3.14)`; requiring those by name
# means editing branch protection every time the matrix changes. Second,
# three jobs here are conditional (`pip-audit`, `link-check-external`,
# `network-anonymous`) and a required check that never reports blocks a
# PR from merging forever.
#
# `if: always()` so this runs even when a dependency failed — without it
# the gate would itself be skipped and report nothing, which is the
# failure mode it exists to prevent.
if: always()
needs:
- lint
- test
- integration
- smoke
- artifact
- secret-scan
- verified-secret-scan
- codespell
- pip-audit
- markdownlint
- link-check
- link-check-external
- cassette-replay
- network-anonymous
- discoverability
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Fail if any job failed or was cancelled
# `skipped` is deliberately tolerated: the three conditional jobs
# above skip by design on a normal push. Treating skipped as failure
# would make every push red; treating cancelled as success would let
# a timed-out run merge.
if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
run: |
echo "::error::One or more required jobs did not succeed."
echo '${{ toJSON(needs) }}'
exit 1
- name: All required jobs succeeded
run: echo "ok"