Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 10 additions & 7 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,25 +50,28 @@ allow).
manta/
├── Cargo.toml # workspace
├── crates/
│ ├── manta-input # IQ sources: SoapySDR, KiwiSDR client, file, audio
│ ├── manta-input # IQ sources: SoapySDR, HPSDR/Hermes, KiwiSDR client, file, audio
│ ├── manta-dsp # PFB channelizer, noise-floor estimation, envelope
│ ├── manta-decode # CW keying state machine, timing, Morse decode
│ ├── manta-spot # callsign validation, CQ/DE parse, dedupe, scoring
│ ├── manta-server # telnet cluster server + JSON/WebSocket stream
│ ├── manta-engine # orchestration: track lifecycle, decoder pool
│ ├── manta-testkit # synthetic CW generator, golden-IQ harness
│ └── manta-cli # `manta` binary: daemon + subcommands
│ ├── manta-cli # `manta` binary: daemon + subcommands
│ └── manta-soak-harness # 24h soak measurement harness (ROADMAP M2 gate),
│ # not shipped in the manta binary
```

Dependency graph (arrows = depends on):

```
manta-cli ──▶ manta-engine ──▶ manta-input ──▶ manta-dsp
│ ├──▶ manta-dsp ──────▶ coppa-dsp
│ ├──▶ manta-decode
│ └──▶ manta-spot ──────▶ manta-decode
──▶ manta-server
│ ├──▶ manta-dsp ──────▶ coppa-dsp
│ ├──▶ manta-decode
│ └──▶ manta-spot ──────▶ manta-decode
└──────────────────▶ manta-server
manta-testkit ──▶ manta-dsp, manta-decode, coppa-channel
manta-soak-harness ──▶ manta-dsp, manta-input, manta-engine, manta-testkit
```

M1 added `manta-input → manta-dsp` (the shared Hilbert transformer, used
Expand Down Expand Up @@ -318,7 +321,7 @@ validation (MAN-28). Dedupe (step 5) still applies.
own yet (decode-pipeline internals, not the network-facing surface
MAN-59 scoped to), and `manta --status` hitting a local control socket
for live stats is similarly not yet implemented. Prometheus text
endpoint (feature `metrics`): input overruns, active tracks, evictions,
endpoint (compiled in unconditionally, no feature flag): input overruns, active tracks, evictions,
decode rate, spots/min, per-stage queue depths, spot confidence
histogram — also aspirational for several of these fields; the
currently-implemented subset is `manta_spots_total`,
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ reachable from this environment.
## Documents (read in this order)

- `README.md` — goals and non-goals
- `ARCHITECTURE.md` — 8-crate workspace, data flow, channelizer/decoder/
- `ARCHITECTURE.md` — 9-crate workspace, data flow, channelizer/decoder/
validation/output design
- `docs/SPEC-decode-core.md` — implementation-level algorithm spec: exact
channelizer constants, noise-floor estimator, track state machine, decoder
Expand Down
234 changes: 147 additions & 87 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,22 @@
</p>

`manta` is a headless daemon written in Rust. It takes wideband IQ from a
commodity SDR, a KiwiSDR over the network, or a WAV file, channelizes the
whole passband with a polyphase filterbank, runs an independent CW decoder on
every signal it finds, validates the callsigns, and emits spots. Output is the
standard `DX de` cluster format over telnet plus a JSON Lines stream, so
existing aggregators (including the Reverse Beacon Network) and modern
consumers (such as [cqdx](https://cqdx.app)) can ingest it without changes.

No GUI. CLI, a config file, and metrics.
commodity SDR, an OpenHPSDR/Hermes device, a KiwiSDR over the network, or a
WAV file, channelizes the whole passband with a polyphase filterbank, runs an
independent CW decoder on every signal it finds, validates the callsigns, and
emits spots. Output is the standard `DX de` cluster format over telnet plus a
JSON Lines / WebSocket stream, so existing aggregators (including the Reverse
Beacon Network) and modern consumers (such as [cqdx](https://cqdx.app)) can
ingest it without changes.

No GUI. CLI, a TOML config file, and Prometheus metrics.

```console
$ telnet manta.example.org 7300
login: W1XYZ
de W5AU-# >
DX de W5AU-#: 14000.7 W1AW CW 7 dB 20 WPM CQ 1533Z
```

## Why

Expand All @@ -40,114 +48,161 @@ with documented, testable algorithms.

## Installation

No Rust toolchain, no cloning the source. Pick one:
```sh
cargo install --path crates/manta-cli --features hpsdr # Rust 1.85+
```

**Download a prebuilt binary** from the [Releases
page](https://github.com/HagaleTechnologies/manta/releases) — macOS
(Intel or Apple Silicon), Windows, or Linux (x86_64 or arm64, including
Raspberry Pi OS 64-bit). Unpack the archive and run the `manta` binary
inside it. **Linux binaries need `libasound2` installed** (audio input is
an unconditional dependency, even if you only ever use file, KiwiSDR, or
HPSDR input) — `sudo apt install libasound2` on Debian/Ubuntu/Raspberry Pi
OS, or the equivalent ALSA runtime package on other distros; without it
the binary fails to start.
That puts a `manta` binary in Cargo's bin directory (`~/.cargo/bin`
unless you moved `CARGO_HOME`), which a standard Rust install already has
on `PATH` — every command below assumes a bare `manta` resolves. To build
without installing, `cargo build --release -p manta-cli` leaves the
binary at `target/release/manta`; run that path instead.

**Or run the Docker image** (works anywhere Docker does, `linux/amd64` and
`linux/arm64`):
No tagged release yet, so there is no prebuilt binary or Docker image to
pull — build from source for now. Both publish automatically, for every
platform, from the first tag:

```sh
# once a release exists:
docker run --rm ghcr.io/hagaletechnologies/manta:latest --help
```

When running as a long-lived server (not `--help`), stop it with
`docker stop -t 30 <container>` — Docker's own default 10-second grace
period before SIGKILL is shorter than manta's supported drain window for
a slow client's final write (up to 25s), so the default can cut a
graceful shutdown off mid-drain.

Both are built by [`.github/workflows/release-publish.yml`](.github/workflows/release-publish.yml)
directly from each tagged release's commit — every published binary
traces to a specific, auditable source revision. (If the image above
returns an authorization error, the GHCR package needs its one-time
"make public" step in GitHub's package settings after the first real
release — see MAN-65. Windows binaries need the [Visual C++
Redistributable](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist)
installed if it isn't already.)
**Notes:**
- Linux binaries need `libasound2` installed (audio input is an
unconditional dependency, even if you only ever use file, KiwiSDR, or
HPSDR input) — `sudo apt install libasound2` on Debian/Ubuntu/Raspberry
Pi OS, or the equivalent ALSA runtime package elsewhere.
- Stop a long-running container with `docker stop -t 30 <container>` —
Docker's default 10-second grace period is shorter than manta's drain
window for a slow client's final write (up to 25 s), so the default
can cut a graceful shutdown off mid-drain.
- Input backends are cargo features, and a build that did not ask for
one has no flags for it — `--hpsdr-host` / `--soapy-driver` fail with
`error: unexpected argument` on a build without them. `hpsdr`
(OpenHPSDR/Hermes) has no native dependency, which is why the install
line above turns it on. `soapy` (RTL-SDR, Airspy, SDRplay, HackRF via
SoapySDR) needs the native SoapySDR system library installed first;
once you have it, add it: `--features hpsdr,soapy`.
- Windows binaries need the [Visual C++
Redistributable](https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist)
installed if it isn't already.

## 60-second demo

No SDR, no radio. Install, generate a synthetic golden vector, decode it:

**Building from source** (if you're developing manta itself, or need a
platform/feature combination the release matrix doesn't cover — the
`soapy` feature below, for instance, isn't in the official release
binaries since it needs the SoapySDR system library) still works exactly
as before, and is what the rest of this Quickstart assumes:

## Quickstart
```sh
cargo install --path crates/manta-cli --features hpsdr # puts `manta` on PATH
manta gen v1 --out /tmp/v1 # 120 s of synthetic CW, 20 WPM, +20 dB
manta decode /tmp/v1/v1.wav
# CQ DE W1AW W1AW K CQ CQ DE W1AW W1AW K CQ CQ DE W1AW W1AW K CQ …
```

Requires Rust 1.85 or newer.
Then point it at a real signal — a public KiwiSDR needs no hardware of
your own:

```sh
# Build
cargo build --release -p manta-cli
manta listen --kiwi-host kiwi.example.org --kiwi-freq 7030000
```

# Decode a synthetic golden vector from a file (deterministic, no hardware)
manta gen v1 --out /tmp/v1
manta decode /tmp/v1/v1.wav
File replay (`listen --source`) currently accepts only 48 kHz mono audio
and runs faster than realtime, so the hardware-free path above stops at
`decode`; a paced replay that can drive the servers below is being
worked on.

# Copy live CW from a public KiwiSDR on 40 m
manta listen --kiwi-host kiwi.example.org --kiwi-freq 7030000
`manta --help` lists every subcommand and flag.

# Copy from a local SDR via SoapySDR (build with --features soapy)
manta listen --soapy-driver driver=rtlsdr --soapy-freq 7030000 --soapy-rate 240000
## Run it as a node

# Copy from the default audio input (rig audio passband, 48 kHz)
manta listen
One flag starts the DX cluster telnet server, the JSON/WebSocket stream,
and the metrics endpoint alongside the decoder:

# Any of the above as JSON Lines instead of text
manta listen --json --kiwi-host kiwi.example.org --kiwi-freq 7030000
```toml
# server.toml
[server]
station_callsign = "W5AU" # your call; becomes `DX de W5AU-#:` and JSON `deCall`
bind_addr = "127.0.0.1" # 0.0.0.0 to accept remote clients
telnet_port = 7300
json_port = 7301
metrics_port = 7302
```

`manta --help` lists every subcommand and flag.
```sh
manta listen --kiwi-host kiwi.example.org --kiwi-freq 7030000 --server-config server.toml
telnet localhost 7300 # DX de … lines
nc localhost 7301 # one JSON object per spot
curl -s localhost:7302/metrics # Prometheus text
```

Forwarding to an upstream RBN-style collector is a `[[rbn_uplink]]`
block. **Set `dry_run = true` first** — the default is `false`, so an
uplink block starts transmitting as soon as you add it.

Before exposing any port beyond loopback, read
[docs/RUNBOOKS/network-exposure.md](docs/RUNBOOKS/network-exposure.md).

## Inputs

| Source | How | Status |
| --- | --- | --- |
| IQ / audio WAV file | `decode`, `listen --source` | Working |
| IQ / audio WAV file | `decode`, `listen --source` | Working (`decode` takes IQ; `listen --source` takes 48 kHz mono audio) |
| Sound card (rig audio passband) | `listen --device` | Working, 48 kHz input only |
| KiwiSDR over the network | `listen --kiwi-host` | Working |
| RTL-SDR, Airspy, SDRplay, HackRF, and anything else SoapySDR drives | `listen --soapy-driver`, feature `soapy` | Working, needs hardware soak |
| OpenHPSDR / Hermes (Hermes-Lite 2, Red Pitaya, QMTech) | `listen --hpsdr-host`, feature `hpsdr` — on in the install line above, no native dependency | Working; protocol verified against reference sources, not yet against hardware |
| RTL-SDR, Airspy, SDRplay, HackRF, anything SoapySDR drives | `listen --soapy-driver`, feature `soapy` — **not** in the install line above; needs the SoapySDR system library, then `--features hpsdr,soapy` | Working, needs hardware soak |

Targets Linux (x86-64 and ARM, Raspberry Pi 4 class), macOS, and Windows.
The CPU budget is a full 192 kS/s passband inside one Raspberry Pi 4 core,
enforced by criterion benches.

## Outputs

- Decoded text or JSON Lines on stdout today.
- RBN-format `DX de` spots over the DX cluster telnet protocol (port 7300)
and a JSON Lines / WebSocket stream (port 7301): in progress, see
[ROADMAP.md](ROADMAP.md) milestone M3.
All four ship today; `manta listen --server-config <file>` starts the
first three together.

- **DX cluster telnet server** (`:7300`) — standard login prompt and
RBN-format `DX de` lines, with enough command grammar (`sh/dx`,
`set/dx/filter`) for stock clients. This is the RBN/aggregator
compatibility surface.
- **JSON Lines / WebSocket stream** (`:7301`) — one full-fidelity spot
object per line; a raw TCP client and a WebSocket client share the
port. This is the [cqdx](https://cqdx.app) ingest surface.
- **Outbound RBN uplink** — forwards validated spots to an upstream
collector in the same `DX de` wire format, with reconnect and multiple
simultaneous targets. Working against a collector; **not yet validated
against RBN's live ingest**.
- **Prometheus metrics** (`:7302/metrics`) — spot, client, uplink and
source-health counters. Some gauges are still placeholders;
ARCHITECTURE §8 says which.

`decode` and `listen` also print decoded text or `--json` events on
stdout. That output is a debugging aid, not a stable interface — the
servers above are.

The decode path is deterministic: the same file in produces byte-identical
spot logs out. That is a hard requirement, and CI enforces it with golden
test vectors.

## Status

Pre-1.0. What exists and what does not:

- **Done:** single-signal decode from files and live audio (M1); the full
wideband pipeline of polyphase channelizer, detector, track manager, and
decoder pool, with SoapySDR and KiwiSDR inputs (M2 sub-projects); callsign
validation, CQ/DE parsing, cty.dat and SCP cross-checks, dedupe, wired into
the engine (M3, in part).
- **Open acceptance gates:** the Raspberry Pi 4 CPU-budget measurement and a
24 h live-SDR soak both need physical hardware.
- **Next:** the telnet and JSON spot servers, TOML config, metrics, and an RBN
parity benchmark on recorded contest IQ.
- **Known limits:** the classical decoder loses copy under heavy HF fading on
a few golden vectors (issues #25 and #28). Closing that gap is the M4 ML
fusion stage, gated on beating the classical baseline under simulated
fading.
Pre-1.0, and pre-first-release. What is true today:

- **Shipped:** the full wideband pipeline — polyphase channelizer,
noise-floor detector, track manager, decoder pool — with five input
sources, callsign validation (cty.dat, SCP, CQ/DE parsing, dedupe), and
all four output surfaces above.
- **Not yet measured:** the RBN parity benchmark (recall vs. RBN on
recorded contest IQ), the Raspberry Pi 4 CPU budget, and a 24-hour
live-SDR soak. The first needs reference data; the other two need
physical hardware. The CPU budget's desktop leg *has* been measured and
currently reads as a **fail** — ≈0.53x–0.58x realtime against a <0.5x
budget, pending a clean rerun on a quiet machine ([ROADMAP.md](ROADMAP.md) M2).
- **Known limits:** the classical decoder loses copy under heavy HF
fading on several golden vectors, and at low SNR the validator still
admits occasional bogus callsigns from noise. Closing the fading gap is
classical-DSP work in flight, with ML fusion behind it at M4.
- **No tagged release yet**, so the container image above is empty until
the first tag.

[ROADMAP.md](ROADMAP.md) has the milestone breakdown with acceptance
criteria.
Expand All @@ -173,16 +228,21 @@ criteria.

## Documentation

- [ARCHITECTURE.md](ARCHITECTURE.md): the seven-crate workspace, data flow,
and the channelizer, decoder, validation, and output design.
- [docs/SPEC-decode-core.md](docs/SPEC-decode-core.md): the
implementation-level algorithm spec. Channelizer constants, noise-floor
estimator, track state machine, decoder equations, confidence formulas,
determinism rules, golden vectors, and the config-key table.
- [ROADMAP.md](ROADMAP.md): milestones M0 to M4 with acceptance criteria.
- [docs/DECISIONS/](docs/DECISIONS/): dated design decisions and
implementation pins.
- [wiki/INDEX.md](wiki/INDEX.md): accumulated gotchas and cross-references.
**Running a node:** [docs/RUNBOOKS/network-exposure.md](docs/RUNBOOKS/network-exposure.md)
(exposing the servers safely) · the "Run it as a node" section above ·
`manta <subcommand> --help` for every flag.

**Contributing:** [ARCHITECTURE.md](ARCHITECTURE.md) — the nine-crate
workspace, data flow, and the channelizer, decoder, validation and output
design · [docs/DECISIONS/](docs/DECISIONS/) — dated design decisions and
implementation pins · [wiki/INDEX.md](wiki/INDEX.md) — accumulated
gotchas.

**Algorithms and research:** [docs/SPEC-decode-core.md](docs/SPEC-decode-core.md) —
channelizer constants, noise-floor estimator, track state machine,
decoder equations, confidence formulas, determinism rules, golden
vectors, config-key table · [ROADMAP.md](ROADMAP.md) — milestones M0 to
M4 with acceptance criteria.

## Related projects

Expand Down
18 changes: 12 additions & 6 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,11 +112,18 @@ repetition gate, dedupe) is complete as a standalone crate -- see
-decode-core.md §7.1 (V11-V15). It is now wired into `manta-engine`'s
batch (`decode_samples`/`decode_wav`) and streaming (`listen`) pipelines,
both emitting real `Spot`s -- see
`docs/superpowers/specs/2026-07-26-m3-engine-wiring-design.md`. Remaining
M3 sub-projects: `manta-server` (telnet + JSON/WebSocket output, TOML
config, metrics), and the RBN parity benchmark (needs ≥ 2 h of recorded
contest-weekend IQ with RBN reference spots -- a data dependency not yet
resolved).
`docs/superpowers/specs/2026-07-26-m3-engine-wiring-design.md`.
`manta-server` has also landed: the telnet cluster server, the
JSON/WebSocket stream, TOML config, the metrics endpoint, and the
outbound RBN uplink are all shipped and tested. Remaining M3 sub-projects:
the RBN parity benchmark (needs ≥ 2 h of recorded contest-weekend IQ with
RBN reference spots -- a data dependency not yet resolved) and the
7-day unattended soak below.

**RBN admission** is pre-1.0 work, not deferred: the near-term path is a
Skimmer-Server-compatible handshake so existing aggregators can accept a
manta node, with direct-uplink admission negotiated from a position of
having live nodes.

## M4 — ML decoder stage (research-dependent)

Expand All @@ -135,6 +142,5 @@ confidence weighting. ONNX/candle inference, feature-gated.

- RTTY/FT4-adjacent modes on the same channelizer.
- Multi-SDR single-daemon orchestration.
- Upstream conversation with RBN operators about accepting manta nodes.
- Spot quality feedback loop: cqdx-side confirmation (same call spotted by other
nodes) fed back to tune validation thresholds.
Loading
Loading