Skip to content

add dcsm gitignore command + dev tooling overhaul#2

Merged
igor47 merged 8 commits into
mainfrom
gitignore-command
Apr 26, 2026
Merged

add dcsm gitignore command + dev tooling overhaul#2
igor47 merged 8 commits into
mainfrom
gitignore-command

Conversation

@igor47

@igor47 igor47 commented Apr 25, 2026

Copy link
Copy Markdown
Owner

Summary

This PR started as a single feature (a dcsm gitignore subcommand to stop the parade of untracked decrypted files in git status) and grew into a small dev-workflow refresh along the way. Each concern is in its own commit so reviewing one at a time is easy.

Feature: dcsm gitignore

For each *.template file under any DCSM_TEMPLATE_* directory, finds the closest ancestor .gitignore (bounded by the template root) and writes the resulting decrypted filename into a managed block:

# BEGIN DCSM (managed by `dcsm gitignore` -- do not edit)
postgres.env
synapse/homeserver.yaml
# END DCSM
  • Paths are rendered relative to whichever .gitignore ends up managing them, so an intermediate .gitignore gets bare filenames while the root .gitignore gets full subpaths.
  • If no ancestor .gitignore exists anywhere within the template root, a single one is created at the template root rather than scattering one per template directory.
  • The command is idempotent — the file is only rewritten when the rendered block would actually change.

Dev environment

  • Drop .python-version; manage Python via mise.toml (auto-creates a project-local .venv).
  • Promote pyproject.toml to declare the project, with PyYAML as the runtime dep and mypy, ruff, types-PyYAML as dev extras.
  • Add just setup (installs .[dev] into the active venv via uv pip install).
  • Add ruff for lint + format (isort + bugbear on top of defaults), wired into just test and exposed via a just format recipe.
  • Apply ruff's format + autofix to existing sources; fix three pre-existing lint findings (B007 unused os.walk captures, B007 unused dict-items unpacking in get_secrets, B904 missing exception chaining in template_file).
  • Replace the old "Dev"/"Requirements" section with a "Developing" section walking through mise, just, just setup, just test, just format, and noting that CI does not currently run just test — only publish.yaml for Docker image build/push.
  • Drop the obsolete top-level version key from docker-compose.yml and from the example block in the README (modern docker compose warns about it on every invocation).

Test plan

  • just test — locally passes ruff check, ruff format --check, mypy --strict, 18 unit tests, and the e2e docker compose up.
  • dcsm gitignore end-to-end behavior covered by 5 unit tests:
    • mixed gitignores → each template's output lands in its closest gitignore
    • no gitignores anywhere → single .gitignore at template root with all entries
    • correct rendered relative paths (root gets full subpath, intermediate gets bare filename)
    • template directly in template root
    • idempotent rewrites
  • Smoke-test against a real consumer repo

Adds a new `dcsm gitignore` subcommand that walks the DCSM_TEMPLATE_*
directories and, for each `.template` file, finds the closest ancestor
.gitignore (bounded by the template root) and writes the resulting
filename into a managed `# BEGIN DCSM ... # END DCSM` block. If no
ancestor .gitignore exists, a single one is created at the template
root so a fresh tree collects all entries into one file.

The command is idempotent: it only rewrites a file when the contents
would actually change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igor47 igor47 requested a review from Copilot April 25, 2026 23:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new dcsm gitignore subcommand to automatically maintain .gitignore entries for decrypted outputs of *.template files, reducing git status noise after running dcsm.

Changes:

  • Implement dcsm gitignore to discover templates under DCSM_TEMPLATE_* directories, choose the nearest in-tree .gitignore, and update a managed ignore block.
  • Add unit + end-to-end tests covering proximity selection, block updates (create/append/replace/idempotent), and relative path rendering.
  • Document the new workflow in the README.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File Description
dcsm.py Adds template discovery + .gitignore proximity selection and managed-block update logic; wires new gitignore subcommand into main().
test.py Adds comprehensive tests for find_proximate_gitignore, update_gitignore, and the end-to-end gitignore command behavior.
README.md Documents the new dcsm gitignore command and the managed .gitignore block semantics.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dcsm.py Outdated
Comment on lines +184 to +185
original = path.read_text() if path.exists() else ''

Copilot AI Apr 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update_gitignore uses path.read_text() without an explicit encoding. For deterministic behavior across locales (and to avoid surprises in non-UTF-8 environments), consider passing encoding='utf-8' explicitly.

Copilot uses AI. Check for mistakes.
Comment thread dcsm.py Outdated
Comment on lines +209 to +210
path.write_text(replaced)
return True

Copilot AI Apr 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update_gitignore uses path.write_text(replaced) without an explicit encoding. Consider using encoding='utf-8' to keep the file content stable across different locale/default-encoding configurations.

Copilot uses AI. Check for mistakes.
Comment thread dcsm.py Outdated
Comment on lines +158 to +160
assert template.is_relative_to(template_root), (
f'{template} is not under template root {template_root}'
)

Copilot AI Apr 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

find_proximate_gitignore uses an assert template.is_relative_to(template_root) for input validation. Asserts can be disabled with Python optimizations (-O), which would skip this safety check and potentially let the function walk outside the intended root. Prefer an explicit runtime check that raises a ValueError (or similar) when template is not under template_root.

Suggested change
assert template.is_relative_to(template_root), (
f'{template} is not under template root {template_root}'
)
if not template.is_relative_to(template_root):
raise ValueError(
f'{template} is not under template root {template_root}'
)

Copilot uses AI. Check for mistakes.
Comment thread dcsm.py Outdated
Comment on lines +190 to +193
m = pattern.search(original)
if m:
before = original[:m.start()].rstrip('\n')
after = original[m.end():].lstrip('\n')

Copilot AI Apr 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update_gitignore only searches for and replaces the first managed block (pattern.search). If a .gitignore already contains multiple DCSM-managed blocks (e.g., from a merge conflict or previous bug), this will leave extra blocks behind and can produce confusing results. Consider detecting multiple blocks and either replacing all of them or raising an error that instructs the user to fix the file.

Suggested change
m = pattern.search(original)
if m:
before = original[:m.start()].rstrip('\n')
after = original[m.end():].lstrip('\n')
matches = list(pattern.finditer(original))
if matches:
before = original[:matches[0].start()].rstrip('\n')
after = original[matches[-1].end():].lstrip('\n')

Copilot uses AI. Check for mistakes.
Comment thread dcsm.py Outdated
Comment on lines +187 to +205
r'\n*' + re.escape(GITIGNORE_BEGIN) + r'\n.*?' + re.escape(GITIGNORE_END) + r'\n*',
re.DOTALL,
)
m = pattern.search(original)
if m:
before = original[:m.start()].rstrip('\n')
after = original[m.end():].lstrip('\n')
else:
before = original.rstrip('\n')
after = ''

parts: list[str] = []
if before:
parts.append(before + '\n\n')
parts.append(new_block)
if after:
parts.append('\n' + after)
replaced = ''.join(parts)

Copilot AI Apr 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update_gitignore trims surrounding newlines (rstrip('\n') / lstrip('\n')) and then re-inserts normalized spacing ('\n\n' before the block, '\n' before after). This can rewrite parts of the file outside the managed block (e.g., collapsing/expanding blank lines) even when the managed entries themselves are unchanged. If the goal is to only update the managed block, preserve the exact original prefix/suffix whitespace around the match, or document that the command canonicalizes formatting.

Suggested change
r'\n*' + re.escape(GITIGNORE_BEGIN) + r'\n.*?' + re.escape(GITIGNORE_END) + r'\n*',
re.DOTALL,
)
m = pattern.search(original)
if m:
before = original[:m.start()].rstrip('\n')
after = original[m.end():].lstrip('\n')
else:
before = original.rstrip('\n')
after = ''
parts: list[str] = []
if before:
parts.append(before + '\n\n')
parts.append(new_block)
if after:
parts.append('\n' + after)
replaced = ''.join(parts)
r'(?P<leading>\n*)'
+ re.escape(GITIGNORE_BEGIN)
+ r'\n.*?'
+ re.escape(GITIGNORE_END)
+ r'(?P<trailing>\n*)',
re.DOTALL,
)
m = pattern.search(original)
if m:
replaced = (
original[:m.start()]
+ m.group('leading')
+ new_block
+ m.group('trailing')
+ original[m.end():]
)
else:
trailing_newlines = len(original) - len(original.rstrip('\n'))
if not original:
separator = ''
elif trailing_newlines == 0:
separator = '\n\n'
elif trailing_newlines == 1:
separator = '\n'
else:
separator = ''
replaced = original + separator + new_block

Copilot uses AI. Check for mistakes.
Comment thread dcsm.py Outdated
Comment on lines +233 to +238
try:
rel = output.relative_to(ignore_path.parent)
except ValueError:
# output is not under the gitignore's directory -- fall back
# to an absolute-from-gitignore-dir style by skipping
rel = Path(output.name)

Copilot AI Apr 25, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In gitignore(), the try/except ValueError around output.relative_to(ignore_path.parent) should be unreachable given the current find_proximate_gitignore behavior (the chosen .gitignore is always an ancestor of the template/output). The fallback also only uses output.name, which is not an "absolute-from-gitignore-dir" style path as the comment suggests. Consider removing the try/except (or replacing it with an explicit invariant check) to keep the control flow and comment accurate.

Suggested change
try:
rel = output.relative_to(ignore_path.parent)
except ValueError:
# output is not under the gitignore's directory -- fall back
# to an absolute-from-gitignore-dir style by skipping
rel = Path(output.name)
if not output.is_relative_to(ignore_path.parent):
raise ValueError(
f'output path {output} is not under gitignore directory '
f'{ignore_path.parent}'
)
rel = output.relative_to(ignore_path.parent)

Copilot uses AI. Check for mistakes.
igor47 and others added 4 commits April 25, 2026 16:40
- Drop `.python-version` in favor of `mise.toml`, which pins python and
  auto-creates a `.venv` on entering the directory.
- Promote `pyproject.toml` to define the project (runtime dep: PyYAML;
  dev deps: mypy + types-PyYAML) so dependencies live in one place.
- Add `just setup` recipe which installs the project in editable mode
  with dev extras into the active venv via `uv pip install`.
- Ignore `.venv` and `dcsm.egg-info` produced by the local install.

Skipping mise tasks for now since the justfile already serves that role.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add `ruff` to dev dependencies in pyproject.toml.
- Configure isort (`I`) and flake8-bugbear (`B`) lint rules on top of
  ruff's defaults.
- Add `just format` recipe to autoformat sources and apply lint fixes.
- Wire `ruff check` and `ruff format --check` into `just test` so the
  test recipe gates on style as well as types and behavior.

This commit only adds the tooling; the existing source does not yet
satisfy the new checks. The follow-up commit fixes those issues.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Reformat both source files via `ruff format` (largely a quote-style
  swap from single to double).
- Sort imports per isort.
- Rename unused `os.walk` directory captures to `_dirs` (B007).
- Drop unused tuple unpacking in `get_secrets` -- iterate directly
  over the dict (B007).
- Chain the template-substitution exception with `from e` so the
  original cause is preserved in the traceback (B904).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the brief 'Dev' / 'Requirements' sections with a 'Developing'
section that covers:

- the mise + just toolchain and how the project-local venv is created
- `just setup` for installing runtime + dev deps from pyproject.toml
- what `just test` actually runs (ruff, mypy, unit tests, e2e compose)
- a note that there is no CI test gate today; only Docker publish runs
  in GitHub Actions

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igor47 igor47 changed the title add gitignore command to manage .gitignore for decrypted files add dcsm gitignore command + dev tooling overhaul Apr 25, 2026
The Compose Specification dropped the top-level `version` key; modern
`docker compose` warns about it on every invocation. Remove it from
`docker-compose.yml` and from the example block in the README.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igor47 igor47 force-pushed the gitignore-command branch from eb85931 to 5527c0d Compare April 25, 2026 23:47
igor47 and others added 2 commits April 25, 2026 16:49
- Pass `encoding="utf-8"` to `read_text`/`write_text` in
  `update_gitignore` so behavior is deterministic across locales.
- Replace the `assert template.is_relative_to(template_root)` in
  `find_proximate_gitignore` with an explicit `ValueError`. Asserts
  are stripped under `python -O`, which would silently let the
  function walk outside the intended root.
- Drop the unreachable `try/except ValueError` around
  `output.relative_to(ignore_path.parent)` in `gitignore()`. The
  invariant from `find_proximate_gitignore` (the chosen `.gitignore`
  is always at or above the template's directory) makes the call
  total. Replaced with a comment explaining the invariant.

Add a test that `find_proximate_gitignore` raises `ValueError` when
the template lives outside the template root.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
If a `.gitignore` ends up with more than one DCSM-managed block (e.g.
left behind by a merge conflict or earlier bug), `update_gitignore`
previously only rewrote the first block and left the others in place.

Switch from `pattern.search` to `pattern.finditer` and span from the
first match's start to the last match's end, so all stale blocks
collapse into one freshly-rendered block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@igor47 igor47 merged commit 889b2d7 into main Apr 26, 2026
1 check passed
@igor47 igor47 deleted the gitignore-command branch April 26, 2026 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants