Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,6 @@ tmp/
# Python
__pycache__/
*.pyc

# Local agent notes
CLAUDE.local.md
9 changes: 4 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
strict no backward compatibility no matter what, so feel free to make breaking changes as needed. Just make sure to
do `cargo clippy --all-targets -- -D warnings` after changes to make sure code style is correct and then `cargo test` as needed.
- strict no backward compatibility or legacy code paths no matter what, no change logs either.
- do `cargo clippy --all-targets -- -D warnings` after changes and then `cargo test` as needed.

- Use `./tmp` as the temporary working directory for debug output, scratch files, etc. It is gitignored.
- The project is a Cargo workspace: `crates/core` (`audio-pattern-detector-core`, the library) and `crates/cli` (`audio-pattern-detector`, the binary). Keep CLI-only dependencies (e.g. clap) out of the core crate. See `docs/development.md` for the code layout.
Expand All @@ -21,9 +21,8 @@ do `cargo clippy --all-targets -- -D warnings` after changes to make sure code s

## Debugging opus/lossy audio

- Use `--debug` and `--debug-dir <dir>` to save debug output per run. Use separate debug dirs for A/B comparisons.
- Debug graphs are not implemented yet (planned for v2, see `docs/roadmap.md`). Until then use the stderr diagnostics and the per-section peak dumps in `debug/cross_correlation_<clip>/`, which contain the MSE similarity and per-window Pearson r of every candidate.
- Debug audio sections in `audio_section/` can be listened to for ground truth verification. Don't assume old detection results are correct.
- There is no debug mode yet (planned for v2, see `docs/roadmap.md`).
- Listen to the audio around a candidate for ground truth verification. Don't assume old detection results are correct.
- Pattern clips should be extracted from the same encoding as the target audio (e.g. from an Opus stream when matching Opus audio). Denoise as a fallback when source-matched clips aren't available. See `docs/denoise-strategy.md`.

## Version bumping
Expand Down
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ resolver = "2"
members = ["crates/core", "crates/cli"]

[workspace.package]
version = "0.4.3"
version = "0.4.4"
edition = "2021"
authors = ["Andrew Chen <hello@andrewtheguy.com>"]

Expand Down
10 changes: 4 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Detects audio patterns specified by audio clips in target audio files. Designed

Useful for AI workflows to efficiently segment audio files before processing (e.g., OpenAI Whisper transcription preprocessing).

Detection is a two-step process. **Step 1** always runs FFT cross-correlation against the audio to find and center potential match locations. **Step 2** verifies each candidate using one of three paths chosen by clip type: normal verification (partitioned MSE + multi-window Pearson correlation), short-clip verification (single-window variant for clips under 0.5s), or marker-tone verification (narrowband spectral check for `.apd.toml` patterns like station beeps). Robust against lossy-encoded audio (Opus, AAC).
Detection is a two-step process. **Step 1** always runs FFT cross-correlation against the audio to find and center potential match locations. **Step 2** verifies each candidate using one of three paths chosen by clip type: normal verification (partitioned MSE + center-window Pearson correlation), short-clip verification (single-window variant for clips under 0.5s), or marker-tone verification (narrowband spectral check for `.apd.toml` patterns like station beeps). Robust against lossy-encoded audio (Opus, AAC).

Written in Rust: a single self-contained binary with no runtime dependencies (ffmpeg is only needed for non-WAV input files). Also available as a Python package with the same detector ([Python bindings](docs/python.md)).

Expand All @@ -25,7 +25,7 @@ cargo install --git https://github.com/andrewtheguy/audio_pattern_detector.git -
`ghcr.io/andrewtheguy/audio_pattern_detector:<tag>` is published with each release (`v<version>`, plus `latest` for releases from `main`). It contains only the binary at `/usr/local/bin/audio-pattern-detector` (no ffmpeg), so the usual use is copying it into another image:

```dockerfile
COPY --from=ghcr.io/andrewtheguy/audio_pattern_detector:v0.4.3 /usr/local/bin/audio-pattern-detector /usr/local/bin/audio-pattern-detector
COPY --from=ghcr.io/andrewtheguy/audio_pattern_detector:v<version> /usr/local/bin/audio-pattern-detector /usr/local/bin/audio-pattern-detector
```

The release workflow builds the image for amd64 and arm64 from the binaries it has already compiled (`runtime-prebuilt` target). To build it locally from source for the host architecture: `docker build --target runtime -t audio-pattern-detector .`
Expand Down Expand Up @@ -93,8 +93,6 @@ audio-pattern-detector show-config ./clips/rthk_beep.apd.toml
| `--chunk-seconds` | Seconds per chunk (default: 60, or "auto") |
| `--timestamp-format` | JSONL timestamp fields: `both` (default), `ms`, or `formatted` |
| `--height-min` | Minimum correlation peak height (default: 0.25; lower to find weak matches) |
| `--debug` | Enable debug mode (diagnostics, candidate audio, peak dumps; no charts yet) |
| `--debug-dir` | Base directory for debug output (default: ./tmp) |

## JSONL Output Format

Expand Down Expand Up @@ -136,8 +134,8 @@ For custom audio sources, implement `SampleSource` and drive `AudioPatternDetect
- **[Denoise Strategy](docs/denoise-strategy.md)** - How to denoise pattern clips for better matching with lossy-encoded or noisy audio
- **[Python Bindings](docs/python.md)** - Python API, installation and building the wheel
- **[Stdin Modes](docs/stdin-modes.md)** - WAV stdin and multiplexed stdin (IPC) with a Node.js example
- **[Development](docs/development.md)** - Building, linting, testing, code layout, debug output
- **[Roadmap](docs/roadmap.md)** - What v1 covers and the plan for debug charts in v2
- **[Development](docs/development.md)** - Building, linting, testing, code layout
- **[Roadmap](docs/roadmap.md)** - The plan for the debug mode and its charts in v2

## Development

Expand Down
2 changes: 0 additions & 2 deletions audio_pattern_detector.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@ class Detector:
seconds_per_chunk: int | None = 60,
target_sample_rate: int = 8000,
height_min: float | None = None,
debug: bool = False,
debug_dir: StrPath = "./tmp",
) -> Detector: ...
@property
def clip_names(self) -> list[str]: ...
Expand Down
10 changes: 0 additions & 10 deletions crates/cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -71,14 +71,6 @@ struct MatchArgs {
#[arg(long, value_name = "SECONDS", default_value = "60")]
chunk_seconds: String,

/// Debug mode: diagnostics on stderr plus candidate audio and peak dumps under --debug-dir
#[arg(long)]
debug: bool,

/// Base directory for debug output
#[arg(long, value_name = "DIR", default_value = "./tmp")]
debug_dir: PathBuf,

/// Override minimum correlation peak height (default: 0.25, lower to find weak matches)
#[arg(long, value_name = "HEIGHT")]
height_min: Option<f32>,
Expand Down Expand Up @@ -158,10 +150,8 @@ fn cmd_match(args: MatchArgs) -> Result<()> {
};

let options = MatchOptions {
debug_mode: args.debug,
seconds_per_chunk,
target_sample_rate: args.target_sample_rate,
debug_dir: args.debug_dir,
height_min: args.height_min,
};

Expand Down
30 changes: 0 additions & 30 deletions crates/cli/tests/cli_integration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
//! Internal detection logic is tested in the other integration test files.

use std::collections::BTreeSet;
use std::fs;
use std::io::Write;
use std::process::{Command, Output, Stdio};

Expand All @@ -30,9 +29,6 @@ const CBS_NEWS_CLIP_NAME: &str = "cbs_news";
const RAINBOW_INTRO_PATTERN: &str = "../../sample_audios/clips/天空下的彩虹intro.wav";
const PATTERN_FOLDER: &str = "../../sample_audios/clips";
const NONEXISTENT_FILE: &str = "nonexistent.wav";
/// Pattern name that would escape the debug directory if used as a path.
const TRAVERSAL_CLIP_NAME: &str = "../../escape";
const TRAVERSAL_SAFE_NAME: &str = ".._.._escape";
const DEFAULT_SAMPLE_RATE: u32 = 8000;

struct CliResult {
Expand Down Expand Up @@ -612,29 +608,3 @@ fn multiplexed_stdin_requires_no_pattern_file() {
let result = run_cli_stdin(&["match", "--multiplexed-stdin"], payload);
assert_eq!(result.code, 0);
}

#[test]
fn multiplexed_stdin_debug_output_stays_inside_debug_dir() {
let pattern_data = fs::read(CBS_NEWS_PATTERN).unwrap();
let audio_data = fs::read(CBS_NEWS_AUDIO).unwrap();
let payload = build_multiplexed_payload(&[(TRAVERSAL_CLIP_NAME, &pattern_data)], &audio_data);

let root = tempfile::tempdir().unwrap();
let debug_dir = root.path().join("nested").join("debug_out");
let result = run_cli_stdin(&["match", "--multiplexed-stdin", "--debug", "--debug-dir", debug_dir.to_str().unwrap()], payload);
assert_eq!(detected_clip_names(&result.events()), BTreeSet::from([TRAVERSAL_CLIP_NAME.to_string()]));

// Everything written under the temp root is inside debug_dir, in
// directories named after the sanitised clip name.
let mut top_level: Vec<String> =
fs::read_dir(root.path()).unwrap().map(|e| e.unwrap().file_name().to_string_lossy().into_owned()).collect();
top_level.sort();
assert_eq!(top_level, vec!["nested".to_string()]);
let nested: Vec<String> = fs::read_dir(root.path().join("nested"))
.unwrap()
.map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
.collect();
assert_eq!(nested, vec!["debug_out".to_string()]);
assert!(debug_dir.join("audio_section").join(TRAVERSAL_SAFE_NAME).is_dir());
assert!(debug_dir.join("debug").join(format!("cross_correlation_{TRAVERSAL_SAFE_NAME}")).is_dir());
}
2 changes: 1 addition & 1 deletion crates/core/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ pyo3 = { version = "0.28", optional = true, features = ["abi3-py39"] }
realfft = "3.5"
rustfft = "6.4"
serde.workspace = true
serde_json.workspace = true
toml = "1.1"

[dev-dependencies]
serde_json.workspace = true
tempfile.workspace = true
Loading
Loading