Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 13 additions & 14 deletions docs/autonomous-quality-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,29 +58,28 @@ commands consume verified state but do not duplicate review logic.
- Final export blocks while any current cue remains unaudited. The audit
must not claim that deterministic heuristics prove semantic correctness.

### 3. Honest completion QA
### 3. Optional visual diagnostics

- Add a QA report that separates mechanical evidence (current artifact hashes,
duration, non-empty video, rendered frame files) from visual observation.
- A model that cannot receive images may complete mechanical QA, but the result
must say that visual inspection was not performed; it must never report a
visual pass.
- Keep the QA report as an optional diagnostic that separates rendered-frame
evidence from visual observation.
- A model that cannot receive images skips visual QA. This is not a delivery
failure and does not reduce the assessed subtitle-content quality.
- A visual attestation, when available, is tied to the current MP4 and rendered
frame hashes. Changing the video invalidates the attestation.
- The bundled skill must use the structured QA result and must not infer visual
success from file existence alone.
- The bundled skill must not run visual QA in the default one-shot flow or infer
visual success from file existence alone.
- Select up to seven risk frames by boundaries, midpoint, source/target length,
source CPS, and short duration instead of relying on three uniform frames.
- A visual failure requires structured issue codes. Layout failures can be
remediated with the smaller, raised `fansub-compact` preset before reburn.
source CPS, and short duration when a user explicitly requests visual review.
- A visual result is advisory and never automatically selects `fansub-compact`
or triggers export/burn rework.

### 4. Hard delivery gate

- `openbbq --json delivery check` is the sole final readiness decision. Any
failed gate returns `ready:false` and a non-zero process exit.
- Aggregate ASR, fresh segmentation, deterministic translation checks,
full-context audit, exact bilingual ASS events, burn provenance, QA hashes,
and visual attestation without duplicating the underlying domain rules.
full-context audit, exact bilingual ASS events, burn provenance, and a
non-empty burned artifact without duplicating the underlying domain rules.
- `status` reports the same delivery result. A successful command or existing
MP4 alone never means delivery-ready.

Expand Down Expand Up @@ -156,7 +155,7 @@ commands consume verified state but do not duplicate review logic.

1. Detect word and segment-level ASR failures with metadata/caption evidence.
2. Require full-coverage, neighbor-bound semantic translation decisions.
3. Add risk-frame visual QA, structured issues, and compact remediation.
3. Keep risk-frame visual QA as an explicit, advisory diagnostic only.
4. Aggregate all facts into a non-zero hard delivery gate and status summary.
5. Keep deterministic checks read-only and every derived artifact hash-bound.
6. Update bundled skills/docs and run static, unit, and real-workspace regressions.
Expand Down
51 changes: 51 additions & 0 deletions docs/glossary-quality-loop.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Agent-driven glossary quality loop

## Goal

A one-shot subtitle request must give the agent enough structured evidence and
safe write operations to discover ASR mistakes from meaning and context, repair
the current source subtitles, and preserve reusable names or mishearings for
future videos.

Word probability is evidence for ordering only. It is not a correctness gate:
high-confidence words can still be semantically wrong.

## Workflow contract

1. `asr check/batch/apply` resolves detector-found issues as before.
2. `glossary audit` pages through every transcript segment and exposes its
surrounding text, word probabilities, overlapping reference caption, and
current glossary matches. The agent reviews meaning rather than accepting a
token only because its probability is high.
3. `asr amend` records a bounded, reasoned phrase correction even when no
detector produced an issue ID.
4. Reusable names, terms, and recurring ASR variants are written atomically with
`glossary apply`. One malformed or colliding change leaves the glossary
untouched.
5. `glossary suggest` mines the ASR-resolved transcript, not the immutable raw
transcript.
6. `segment` reports canonical term matches, alias corrections, and a clear
no-effect signal for the bound glossary.

## Compatibility and safety

- Existing glossary and ASR review files remain readable.
- Review and glossary writes are capped at 20 entries per operation.
- Every manual ASR correction requires `segment_id`, `find`, `replacement`, and
a non-empty reason.
- Updating a glossary for a workspace invalidates segmentation and all later
artifacts; it does not trigger translation, export, or burn automatically.
An idempotent no-op update leaves completed stages intact.
- Reference captions and probabilities are advisory evidence. The agent remains
responsible for contextual judgment and must not copy reference text blindly.

## Acceptance criteria

- A high-confidence error can be found in a context audit and corrected without
inventing a low-confidence issue ID.
- Adding `{"source": "hot take", "aliases": ["hot tick"]}` corrects the
current segmentation and is available to later workspaces.
- Re-running `glossary suggest` after an ASR replacement cannot re-surface the
replaced raw spelling.
- Segment JSON output distinguishes “glossary bound” from “glossary actually
matched or corrected text.”
72 changes: 53 additions & 19 deletions docs/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,45 @@ Use accept/replace for words and entities, and keep_first/drop for repeated
segments. Every decision requires a reason; phrase replacements also include
the exact `find` phrase and `replacement`. When fetch preserved a YouTube VTT,
the batch includes overlapping reference text. Repeat until `asr check` returns
`ready: true`; `segment` blocks unresolved or stale decisions.
`ready: true`; `segment` blocks unresolved or stale decisions. This gate only
closes detector-found issues. It does not certify high-confidence words.

After the detector gate, audit every transcript segment with semantic context:

```bash
openbbq --json glossary suggest --workspace workspaces/demo
openbbq --json glossary audit --workspace workspaces/demo --offset 0 --limit 20
```

Follow `next_offset` until `remaining` is zero. Audit items include previous and
next text, word probabilities, resolved/raw source, reference captions when
available, and glossary matches. The Agent decides from context; probability and
reference text are advisory evidence.

For a one-off error with no detector issue id, apply a bounded contextual patch:

```json
{"amendments":[{"segment_id":12,"find":"hot tick","replacement":"hot take","reason":"The surrounding sentence uses the idiom hot take."}]}
```

```bash
openbbq asr amend --workspace workspaces/demo asr-amendments.json
```

For reusable names and ASR variants, atomically update the bound glossary:

```json
{"terms":[{"source":"Andy Matuschak","aliases":["Annie Matushak"],"note":"researcher; confirmed ASR variant"}]}
```

```bash
openbbq glossary apply --workspace workspaces/demo glossary-terms.json
openbbq --json segment --workspace workspaces/demo
```

Segment output reports `glossary_matched_terms`,
`glossary_aliases_applied`, and `glossary_no_effect`. A bound glossary with no
matches is not treated as proof that terminology was maintained.

## Translate

Expand Down Expand Up @@ -175,26 +213,22 @@ for an intentional manual draft; an explicitly supplied ASS outside the
workspace remains supported. Successful burn also records the final MP4 hash
and the exact source-video and ASS hashes.

## Completion QA
## Completion Checks

```bash
openbbq --json qa render --workspace workspaces/demo
openbbq --json qa check --workspace workspaces/demo
openbbq --json delivery check --workspace workspaces/demo --to zh
```

`qa render` defaults to up to seven boundary, midpoint, long-line, high-CPS,
and short-duration risk frames. `mechanical_status: pass` proves that the current non-empty MP4, source video,
ASS, and rendered frame hashes agree. It is not a visual observation. Only
after actually opening every returned frame should a vision-capable reviewer
run `qa attest --result pass|fail --reason ...`. A failure also requires one or
more structured `--issue` values. Without image input, leave
`visual_status: not_performed` and disclose that visual inspection was not
performed.

Final `delivery check` is a hard gate combining ASR, deterministic translation,
full-context semantic review, export/burn freshness, and visual QA. Any failure
returns `ready:false` with a non-zero exit code.
full-context semantic review, exact bilingual ASS content, export/burn
freshness, burn provenance, and a non-empty MP4. Any failure returns
`ready:false` with a non-zero exit code.
Visual layout is not a default gate; non-vision models do not need to inspect
risk frames and are not penalized for skipping them.

`qa render`, `qa check`, and `qa attest` remain available as optional manual
diagnostics when the user explicitly requests visual review. They do not
automatically select an ASS preset or trigger a reburn.

## ASS Presets

Expand All @@ -207,8 +241,8 @@ openbbq export --workspace workspaces/demo --to zh --mode bilingual --format ass

- `default`: normal 16:9 horizontal video.
- `fansub`: more prominent translated line.
- `fansub-compact`: a smaller, raised bilingual stack for lower-third conflicts
and overlap remediation.
- `fansub-compact`: a smaller, raised bilingual stack used only when explicitly
requested; the default flow never auto-selects it from sampled frames.
- `mobile`: 9:16 vertical video with a vertical canvas and larger bottom safe
area.

Expand Down Expand Up @@ -278,10 +312,10 @@ openbbq fetch
openbbq extract-audio
openbbq transcribe
openbbq segment
openbbq asr check/batch/apply
openbbq asr check/batch/apply/amend
openbbq translate init/batch/apply/check/audit/audit-apply
openbbq review
openbbq glossary list/show/new/use/suggest
openbbq glossary list/show/new/use/suggest/audit/apply
openbbq export
openbbq burn
openbbq qa render/check/attest
Expand Down
63 changes: 48 additions & 15 deletions docs/usage.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,43 @@ openbbq asr apply --workspace workspaces/demo asr-decisions.json
词和实体使用 accept/replace;重复段可用 keep_first/drop。所有决定都必须写理由;
短语 replace 还要提供精确的 `find` 与 `replacement`。fetch 到 YouTube VTT 时,batch
会附带同时间的参考文字。
重复处理直到 `asr check` 返回 `ready: true`;未决或过期决定会阻止 `segment`。
重复处理直到 `asr check` 返回 `ready: true`;未决或过期决定会阻止 `segment`。该门禁
只覆盖检测器发现的问题,不代表高置信词一定正确。

检测器门禁通过后,必须结合语义上下文审计全部转写段:

```bash
openbbq --json glossary suggest --workspace workspaces/demo
openbbq --json glossary audit --workspace workspaces/demo --offset 0 --limit 20
```

沿 `next_offset` 翻页,直到 `remaining` 为 0。每项包含前后段、词级概率、已解析/原始
source、可用时的参考字幕和 glossary 命中。Agent 根据上下文判断;概率和参考文字只是
证据。

没有检测器 issue id 的一次性错误,用有界 amendment 修正:

```json
{"amendments":[{"segment_id":12,"find":"hot tick","replacement":"hot take","reason":"结合前后句可知这里是固定表达 hot take。"}]}
```

```bash
openbbq asr amend --workspace workspaces/demo asr-amendments.json
```

可复用的专名和 ASR 变体用原子 patch 更新已绑定 glossary:

```json
{"terms":[{"source":"Andy Matuschak","aliases":["Annie Matushak"],"note":"研究者;已确认的 ASR 变体"}]}
```

```bash
openbbq glossary apply --workspace workspaces/demo glossary-terms.json
openbbq --json segment --workspace workspaces/demo
```

segment 输出会报告 `glossary_matched_terms`、`glossary_aliases_applied` 和
`glossary_no_effect`。仅仅绑定 glossary 不能证明术语维护真的生效。

## 翻译

Expand Down Expand Up @@ -162,23 +198,19 @@ openbbq --json status --workspace workspaces/demo
外部 ASS 仍然受支持。成功 burn 还会记录最终 MP4,以及所用源视频和 ASS 的精确
内容哈希。

## 完成 QA
## 完成交付检查

```bash
openbbq --json qa render --workspace workspaces/demo
openbbq --json qa check --workspace workspaces/demo
openbbq --json delivery check --workspace workspaces/demo --to zh
```

`qa render` 默认选择最多 7 张首尾、中段、长句、高 CPS、短时长风险帧。
`mechanical_status: pass` 只证明当前非空 MP4、源视频、ASS 和截帧 hash 一致,
不等于看过画面。只有实际打开并检查返回的每一张 frame 后,有视觉输入能力的审核者
才能运行 `qa attest --result pass|fail --reason ...`。失败必须额外用一个或多个
`--issue` 记录结构化问题。没有图像输入能力时必须保留
`visual_status: not_performed`,并明确说明未执行视觉检查。

最终 `delivery check` 是硬门禁:它综合 ASR、翻译机械检查、全覆盖上下文审校、
export/burn freshness 与视觉 QA。任一失败都会返回 `ready:false` 和非零退出码。
双语 ASS 内容、export/burn freshness、烧录 provenance 与非空 MP4。任一失败都会返回
`ready:false` 和非零退出码。视觉排版不属于默认门禁;非多模态模型不需要查看风险帧,
也不会因此被判定失败。

`qa render`、`qa check` 和 `qa attest` 仍保留为用户明确要求时的可选人工诊断,
但不会自动切换 ASS 预设或触发重烧录。

## ASS 预设

Expand All @@ -191,7 +223,8 @@ openbbq export --workspace workspaces/demo --to zh --mode bilingual --format ass

- `default`:常规 16:9 横屏视频。
- `fansub`:译文行更醒目。
- `fansub-compact`:更小且上移的双语堆叠,用于下三分之一冲突或遮挡修复。
- `fansub-compact`:更小且上移的双语堆叠,仅在用户明确指定时使用;默认流程不会
根据抽帧自动选择它。
- `mobile`:面向 9:16 竖屏视频,使用竖屏画布和更大的底部安全区。

`mobile` 只改变渲染样式。目标语行容量由 `translate init` 的覆盖参数控制;如果仍
Expand Down Expand Up @@ -256,10 +289,10 @@ openbbq fetch
openbbq extract-audio
openbbq transcribe
openbbq segment
openbbq asr check/batch/apply
openbbq asr check/batch/apply/amend
openbbq translate init/batch/apply/check/audit/audit-apply
openbbq review
openbbq glossary list/show/new/use/suggest
openbbq glossary list/show/new/use/suggest/audit/apply
openbbq export
openbbq burn
openbbq qa render/check/attest
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "openbbq"
version = "0.0.7"
version = "0.0.8"
description = "Agent-friendly CLI for video translation and subtitle production."
readme = "README.md"
authors = [
Expand Down
2 changes: 1 addition & 1 deletion src/openbbq/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.0.7"
__version__ = "0.0.8"
Loading