IQ Capture and Record Analyze#28
Conversation
Helpful for: - Extra signal processing through numpy or matplotlib - Recording samples with pyhubblenetwork - Testing Satellite hardware with IQ samples from real devices - Determining the frequency of a continuous tone Signed-off-by: hunterhubble <hunter@hubble.com>
- Created /record_analyze API endpoint to record IQ samples, attempt to decode from them, then provides a log of statistics to match our satellite protocol in a plain text format - Statistics include frequency, amplitude, and timing per symbol. From this, we can tell if it matches frequency hopping, see any amplitude drop-offs, and observe timing violations - Also refactored some code to turn into functions I can call in analyze.py Signed-off-by: hunterhubble <hunter@hubble.com>
There was a problem hiding this comment.
Pull request overview
Adds capture-and-diagnostics capabilities to the SDR web service by introducing endpoints to export raw IQ and generate a plain-text analysis report, plus refactors shared symbol/timing utilities to support the new analysis flow.
Changes:
- Added
GET /api/iq_captureto download a.npyrecording of recent IQ samples. - Added
GET /api/record_analyzeand newstream_web/analysis.pyto decode and produce per-symbol timing/frequency/amplitude diagnostics. - Refactored/shared symbol-grid and per-symbol FFT/amplitude helpers across processor/spectrogram/analysis.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/stream_web/timing.py | Adds reusable helpers for dominant per-symbol frequency and RMS dBFS amplitude/noise floor. |
| src/stream_web/spectrogram.py | Uses shared symbol-grid helper and refactors per-symbol frequency computation to the new helper. |
| src/stream_web/processor.py | Uses shared packet symbol-grid helper when computing corrected symbol edges. |
| src/stream_web/config.py | Adds packet_symbol_grid() and rotated_hop_sequence() utilities shared across modules. |
| src/stream_web/app.py | Adds /api/iq_capture and /api/record_analyze, including ring-buffer capture logic and concurrency lock. |
| src/stream_web/analysis.py | New offline diagnostic pipeline: decode, per-symbol analyses, and report formatting. |
| CLAUDE.md | Documents the new RX endpoints. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| sr, sps, hop = config.SAMPLE_RATE, config.samples_per_symbol, config.NUM_SYM_PER_HOP | ||
| ch = rotated[0] | ||
| freqs = [dominant_symbol_freq(iq[s:s + sps], sr) for s, _ in edges] | ||
|
|
| buf = io.BytesIO() | ||
| np.save(buf, segment) | ||
| buf.seek(0) | ||
| fname = f"iq_{int(time.time())}_{seconds}s.npy" | ||
| resp = send_file(buf, mimetype="application/octet-stream", | ||
| as_attachment=True, download_name=fname) |
| @app.route("/api/iq_capture", methods=["GET"]) | ||
| def api_iq_capture(): | ||
| seconds, err = _parse_seconds() | ||
| if err: | ||
| return err |
| - `GET /api/iq_capture?seconds=N` — record N s (1–60) of raw IQ forward from now; returns a `.npy` (complex64) download with `X-Sample-Rate-Hz` / `X-Center-Freq-Hz` / `X-N-Samples` headers | ||
| - `GET /api/record_analyze?seconds=N` — record N s (1–30) and return a plain-text diagnostic report file: per-symbol timing (drift/rate/duration/gap) and per-symbol frequency vs expected channel window for the representative packet, plus synth-res/chipset, amplitude/SNR, RS corrections, regime + diagnosis; noise/mis-hits filtered by preamble strength; backs `hubblenetwork sat record` |
HongNguyen635
left a comment
HongNguyen635
left a comment
There was a problem hiding this comment.
can you please address copilot's comment above?
Also, in the pr summary:
PER-SYMBOL TIMING (duration, gap; drift measured vs an ideal 8000 us symbol / 800 us gap grid)
Symbol 0: duration=8026.88 us, gap=771.84 us, drift=+0.00 us, rate=+0.00 us/sym
Symbol 1: duration=8028.16 us, gap=773.12 us, drift=+0.00 us, rate=+0.00 us/sym
this feels a bit off, 8026.88 + 771.84 = 8798.72. So why is the drift = 0.00 us?
is this gap supposed to be for symbol 1 then?
| _WINDOW_OVERLAP_S = 0.6 # decoder-window overlap; > the longest packet (~0.53 s) | ||
| _DEDUP_WINDOW_S = 0.5 # duplicates (same seq/auth/payload) collapse within this span |
There was a problem hiding this comment.
are these constants the same as defined in config.py?
| def packet_symbol_grid(decode_info: dict) -> tuple[int, int, int]: | ||
| """Return ``(n_sym, slot, sym_len)`` for a decoded-packet / decode_info dict: | ||
| total symbols (preamble + header + PDU), samples per symbol slot, and symbol length.""" | ||
| ver = decode_info.get("phy_ver", 1) | ||
| slot = slot_samples.get(ver, slot_samples[1])["slot"] | ||
| n_sym = PREAMBLE_LEN + NUM_HEADER_SYMS + (decode_info.get("num_pdu_symbols") or 0) | ||
| return n_sym, slot, samples_per_symbol | ||
|
|
||
|
|
||
| def rotated_hop_sequence(channel_num, hop_seq_idx) -> list[int] | None: |
There was a problem hiding this comment.
do you think adding these 2 functions here makes sense?
I assumed config.py is supposed to be for constants.
HongNguyen635
left a comment
HongNguyen635
left a comment
There was a problem hiding this comment.
another comment: maybe it's worth it to think back about why we have the sepration between hubble_satnet_decoder and sdr-docker. I forgot to pay attention to it until now, but isn't these timing analysis that is protocol specific supposed to be in the satnet decoder? and the docker is just there to capture RF -> call satnet decoder -> serves the data?
| from .timing import correct_symbol_edges, dominant_symbol_freq, symbol_amplitudes_dbfs | ||
|
|
||
| log = logging.getLogger(__name__) | ||
| _SYM_GRID_MS, _GAP_GRID_MS = 8.0, 0.8 |
There was a problem hiding this comment.
Do we want to pull this put into config.py as well? or is it something you can import from hubble_satnet_decoder constants.py?
from my understanding, hubble_satnet_decoder defined the protocol, so these things are not supposed to be re-defined here. When protocol changes, you're going to have to change these.
Since we're talking about protocol change, maybe it's relevant to think about (not now, something to think about as you go on). When we're adding more and more things that are specific to a certain protocol instead of protocol agnostic, can we reuse this file / debug when there's a new protocol?
There was a problem hiding this comment.
That is a good point, I'll spend some more time thinking about how to organize all this, I think the protocol specific things can/should be in hubble-satnet-decoder.
|
@HongNguyen635 That's a good point, the current design has the drift based off of the timing of the first symbol, then the rest of the symbols are measured off of that. It's also taken from the midpoint of the symbol relative to the midpoint of the symbols of a 'perfect' template, so while it does look strange that a symbol is too long With that being said, I am re-thinking this design, because the first symbol could be too long, like with this instance, and that should be reported. Good catch |
Added GET /api/iq_capture to return a /npy file of IQ samples for a given duration (default 10s, max 30s)
Added GET /api/record_analyze to record IQ samples and print statistics to logs, including the frequency, timing, amplitude, and snr of each symbol. To note, this does not use the existing GET /api/td_info, because we will probably want to analyze packets that don't decode in a future iteration. The logs are printed in plaintext. (default 10s, max 30s)
I also refactored some code that I repeat in analyze.py into their own functions