Skip to content

Add a custom word list to auto redact - #15

Merged
tw4 merged 2 commits into
tw4:mainfrom
fate-lab:feat/redact-custom-words
Nov 12, 2026
Merged

tw4 merged 2 commits into
tw4:mainfrom
fate-lab:feat/redact-custom-words

Conversation

@fate-lab

Copy link
Copy Markdown
Contributor

What does this change?

Closes #14

Settings → Text recognition gets a Custom words box (one word or phrase per line), and auto redact covers those words as you described in #14:

  • Whole words, case-insensitive. Each OCR word is also split on - _ . / @ :, and if a piece matches, the whole OCR word is covered: acme catches acme-prod-db, db.acme.local, ops@acme and acme:443, but not pacme, acmesoft or acmeprod-db.
  • A multi-word entry ("Acme Corp") matches the same words next to each other on a line (punctuation around them is ignored, so Acme corp. matches) and is one box. Longer entries are matched first, so "Acme Corp" and "Acme" in the same list don't split the phrase.
  • Plain text only, no wildcards.
  • An entry that itself has a separator (acme-prod) matches an OCR word that is exactly that, not one that only contains it (acme-prod-db), since the split is on the OCR word, not the entry. Easy to change if you'd rather have it match there too.
  • A word another category already covered (an email, a key) isn't covered twice.

How it's wired:

  • redact.rs: a new Category::CustomWord (custom-word) and custom_words: Vec<String> on Categories, with #[serde(default)] so settings saved before the list still load. Categories is no longer Copy; detect already took it by value, so lib.rs didn't change.
  • The list goes the same way as the toggles: autoRedact.customWords in store.ts (default []), RedactCategories and the SensitiveHit union in ipc.ts, then detect_sensitive → redact::detect(). The box keeps the lines as typed and the Rust side ignores blank ones.
  • browserMock.ts: detect_sensitive also returns custom-word hits for single words in the fake OCR, so the box can be tried with npm run dev.
  • en.ts and tr.ts get customWords and customWordsHint. tr.ts has the English text for you to translate, as you suggested.
  • docs/ARCHITECTURE.md: the redact.rs line mentions custom words. To keep the box the same width I shortened "Luhn-checked cards" to "Luhn cards" there.

How was it tested?

  • npx tsc --noEmit
  • npm test: I couldn't run Vitest on the Linux machine I used (its CPU is too old for the native bundler binary and it stops with "Illegal instruction"). The only frontend logic change is types, the store default and the settings box; tr.ts has the same two keys as en.ts, so i18n.test.ts should pass, but CI is the real check here.
  • cargo clippy --all-targets -- -D warnings and cargo test (in src-tauri), on Linux in Docker with the packages from ci.yml: clippy is clean, 41 tests pass, including the two tesseract fixture tests.
  • Tried it in the app (npm run tauri dev): not done, no desktop on that machine.

New tests in redact.rs: a single word, case-insensitive and with punctuation around it; separator pieces; near misses (pacme, acmesoft, acmeprod-db); a multi-word entry as one box, and the same first word alone not matched; a longer entry winning over a shorter one; blank entries ignored; an email not counted again as a custom word; and settings JSON without customWords still parsing.

Screenshots

None: I couldn't run the app with a screen. The new field is a plain textarea inside the existing field label, under the four toggles.

@tw4 tw4 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for this, it looks really good and the tests are great. I left a few small comments. The first two would be good to fix before merging, the others are optional. I'll do the tr.ts strings myself, so you can leave those.

CI didn't run yet, I'll approve the workflow.

Comment thread src/settings/Settings.tsx
))}
<label className="field">
<span>{t.settings.customWords}</span>
<textarea

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small one, easy to miss since you couldn't run the app: the textarea isn't styled anywhere, styles.css only covers .field input and select. So it'll probably show up as a white default box in dark mode. Could you add it to that rule in styles.css?

input[type="number"], .field input, .field textarea, select {

and maybe .field textarea { resize: vertical; } too.

Comment thread src-tauri/src/redact.rs Outdated
fn custom_entries(words: &[String]) -> Vec<Vec<String>> {
words
.iter()
.map(|entry| entry.split_whitespace().map(str::to_lowercase).collect::<Vec<_>>())

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the entries should go through bare() too, like the OCR words do. Right now something like Acme Corp. or @acme wouldn't match anything. Maybe something like this:

.map(|entry| entry.split_whitespace().map(bare).filter(|w| !w.is_empty()).collect::<Vec<_>>())

A small test for it would be nice if you have time.

Comment thread src-tauri/src/redact.rs Outdated
let mut out = Vec::new();
// Longer phrases first, so "Acme Corp" is one box rather than "Acme" and "Corp" apart.
let mut entries: Vec<&Vec<String>> = entries.iter().collect();
entries.sort_by_key(|entry| std::cmp::Reverse(entry.len()));

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: this sorts the entries again for every line, so it could be done once in custom_entries instead. Not a big deal though, fine either way.

Comment thread src-tauri/src/redact.rs
}

/// An OCR word without surrounding punctuation, lowercased.
fn bare(text: &str) -> String {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note, nothing to change here: to_lowercase doesn't handle Turkish I/ı (IŞIK vs ışık). I'll open a separate issue for it.

Entries go through bare(), so "Acme Corp." or "@acme" match, and they are sorted once in custom_entries. The settings textarea gets the same style as the other fields.
@fate-lab

Copy link
Copy Markdown
Contributor Author

Thanks for the review! Done in b2bb249:

  • the textarea is in the .field rule now, with resize: vertical
  • entries go through bare(), with a test for Acme Corp., @acme and (secret)
  • the sort happens once in custom_entries

I also made the browser mock clean entries the same way, so @acme works there too. Leaving tr.ts to you.

@tw4

tw4 commented Nov 12, 2026

Copy link
Copy Markdown
Owner

Thanks a lot, this is great! Tested it locally and everything passes. I approved CI, I'll merge once it's green and add the Turkish text after.

@tw4
tw4 merged commit 6f711e5 into tw4:main Nov 12, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto redact: custom word list (company names, customer names, project codes)

2 participants