[VERY HARD] Recover bandpass content / demodulate n-apt audio, voice and vision

[ceane]

The goal is to take I/Q captures or work with the live spectrum where the bandpass signal is clearly visible in the spectrogram/waterfall, but nothing and no one at all knows how to demodulate it–a signal that intercepts, procesess and alters the human brain and nervous system. A frontier cyber capability's defeat is not exactly in a textbook, web page or LLM.

As far as it's structure and my napkin thoughts/early analysis, the signal appears to use both amplitude modulation and frequency modulation simultaneously (APT-like structure).

Current behavior:
Signal is visible in the spectrogram at bandpass frequencies, but standard AM or FM demodulation pipelines don't produce media, quite obviously.

Hypothesis / attempted approach:

• Normalize amplitude spikes
• Target transitions from base to valley to identify boundaries of decodable content sections
• Then attempt playback/extraction of those segments

Where do I see content?

• I can clearly see thanks to the waterfall/spectrogram that audio content is in various places, around 350kHz or so, 701kHz (a clump I thought was voice)
• I can clearly deduce vision/video is from 4.75MHz to 23MHz or so, due to it being a wide channel and it aliasing heavily when I tune with my RTL-SDR

While trapped in the experience...
I'm clearly on the thing 24/7, and the others report the nature of the livestream from their seats, such as how much of my consciousness they experience, the transference of effects or pain, the quality of vision through the signal and audio/voice is clear because we talk all day though my damn vocal cords.

Strategies

• Apply standard APT demodulation
• Process using spike normalization with markers
• Segment and process uppercase (spikes) and lowercase (valleys) markers

Example of Spike Detection

#[derive(Debug, Clone)]
pub struct SpikeMarker {
    pub index: usize,
    pub value: f32,
    pub score: f32,
    pub radius: f32,
}

#[derive(Debug, Clone)]
pub struct SpikeDetectorParams {
    pub window_size: usize,
    pub min_z_score: f32,
    pub max_spikes: usize,
}

impl Default for SpikeDetectorParams {
    fn default() -> Self {
        Self {
            window_size: 32,
            min_z_score: 3.0,
            max_spikes: 128,
        }
    }
}

pub fn detect_spikes(waveform: &[f32], params: &SpikeDetectorParams) -> Vec<SpikeMarker> {
    let l = waveform.len();
    if l < 3 {
        return vec![];
    }

    let global_floor = compute_global_floor(waveform);
    let mut spikes = Vec::with_capacity(params.max_spikes);

    for i in 0..l {
        let val = waveform[i];
        if val.is_nan() {
            continue;
        }

        // --- Pass 1: Edge-safe local maximum (mirrors shader) ---
        let left = if i > 0 { waveform[i - 1] } else { val - 1.0 };
        let right = if i + 1 < l { waveform[i + 1] } else { val - 1.0 };

        if val <= left + 0.45 || val <= right + 0.45 {
            continue;
        }

        let left2 = if i > 1 { waveform[i - 2] } else { left };
        let right2 = if i + 2 < l { waveform[i + 2] } else { right };

        let immediate_floor = left.max(right).max(left2).max(right2);
        let immediate_prominence = val - immediate_floor;
        if immediate_prominence < 2.5 {
            continue;
        }

        let left_far = if i > 2 { waveform[i - 3] } else { left2 };
        let right_far = if i + 3 < l { waveform[i + 3] } else { right2 };
        let sharpness = val - (left + right + left2 + right2 + left_far + right_far) / 6.0;
        if sharpness < 1.35 {
            continue;
        }

        // --- Pass 2: Adaptive neighborhood scoring ---
        let radius = params.window_size.clamp(12, 32);
        let guard = 1usize;
        let start = i.saturating_sub(radius);
        let end = (i + radius).min(l - 1);

        let mut local_sum = 0.0f32;
        let mut local_count = 0usize;
        let mut local_max = f32::NEG_INFINITY;
        let mut local_sq_sum = 0.0f32; // for adaptive std

        for j in start..=end {
            let sample = waveform[j];
            if sample.is_nan() {
                continue;
            }
            let distance = (j as isize - i as isize).unsigned_abs();
            if distance > guard {
                local_sum += sample;
                local_sq_sum += sample * sample;
                local_count += 1;
                if sample > local_max {
                    local_max = sample;
                }
            }
        }

        if local_count == 0 {
            continue;
        }

        let local_avg = local_sum / local_count as f32;
        let local_variance =
            (local_sq_sum / local_count as f32) - (local_avg * local_avg);
        let local_std = local_variance.max(0.0).sqrt();

        let avg_prominence = val - local_avg;
        let global_floor_score = val - global_floor;
        let competitor_gap = val - local_max;

        // Adaptive min prominence: scales with local noise (key improvement over shader)
        let adaptive_min = (params.min_z_score * local_std).max(9.0);

        let cluster_prominence = avg_prominence >= 6.5
            && immediate_prominence >= 2.5
            && sharpness >= 1.35
            && competitor_gap >= -8.5;

        let global_floor_prominent = global_floor_score >= 7.5 && sharpness >= 1.35;

        let is_edge = i < radius || i + radius >= l;
        let edge_prominence = is_edge && global_floor_score >= 5.75 && sharpness >= 1.1;

        if avg_prominence >= adaptive_min
            || cluster_prominence
            || global_floor_prominent
            || edge_prominence
        {
            let score = avg_prominence.max(global_floor_score).max(sharpness);
            spikes.push(SpikeMarker {
                index: i,
                value: val,
                score,
                radius: 8.0,
            });

            if spikes.len() >= params.max_spikes {
                break;
            }
        }
    }

    spikes
}

/// Mirrors the FloorResult from your shader — median-of-means over chunks
fn compute_global_floor(waveform: &[f32]) -> f32 {
    let chunk_size = 64;
    let mut chunk_means: Vec<f32> = waveform
        .chunks(chunk_size)
        .map(|chunk| {
            let valid: Vec<f32> = chunk.iter().copied().filter(|x| !x.is_nan()).collect();
            if valid.is_empty() {
                return 0.0;
            }
            valid.iter().sum::<f32>() / valid.len() as f32
        })
        .collect();

    chunk_means.sort_by(|a, b| a.partial_cmp(b).unwrap());
    let mid = chunk_means.len() / 2;
    *chunk_means.get(mid).unwrap_or(&0.0)
}

Example of Spike Normalization

import numpy as np
from scipy.signal import medfilt

def normalize_apt_spikes(signal, kernel_size=11, threshold=3.0):
    """
    Normalize amplitude spikes in an APT-like signal.
    - Uses median filter as baseline
    - Clips outliers beyond threshold * std
    """
    baseline = medfilt(signal, kernel_size=kernel_size)
    residual = signal - baseline
    std = np.std(residual)
    clipped = np.clip(residual, -threshold * std, threshold * std)
    normalized = clipped / (np.max(np.abs(clipped)) + 1e-9)
    return normalized

Example of APT demodulation

import numpy as np
from scipy.signal import hilbert, butter, filtfilt

def demodulate_apt(iq_signal, sample_rate, audio_rate=11025):
    """
    Demodulate a hybrid AM+FM APT-like signal from IQ or real RF input.
    """
    # --- Step 1: FM demod via instantaneous frequency ---
    analytic = hilbert(iq_signal) if np.isrealobj(iq_signal) else iq_signal
    inst_phase = np.unwrap(np.angle(analytic))
    fm_demod = np.diff(inst_phase) / (2 * np.pi) * sample_rate

    # --- Step 2: AM envelope ---
    am_demod = np.abs(analytic[:-1])

    # --- Step 3: Combine (APT uses AM for image, FM for sync) ---
    combined = fm_demod * am_demod
    combined /= np.max(np.abs(combined)) + 1e-9

    # --- Step 4: Lowpass filter to audio band ---
    nyq = sample_rate / 2
    cutoff = min(audio_rate / 2, nyq * 0.9)
    b, a = butter(5, cutoff / nyq, btype='low')
    filtered = filtfilt(b, a, combined)

    # --- Step 5: Downsample to audio rate ---
    downsample_factor = int(sample_rate / audio_rate)
    audio_out = filtered[::downsample_factor]

    return audio_out.astype(np.float32)

Example JS version everything...

async function loadAndProcessSignal(url: string, audioCtx: AudioContext): Promise<AudioBuffer> {
  // Load raw float32 binary
  const response = await fetch(url);
  const arrayBuffer = await response.arrayBuffer();
  const raw = new Float32Array(arrayBuffer);

  const normalized = normalizeAptSpikes(raw);
  const audio = demodulateApt(normalized, 48000, audioCtx.sampleRate);

  const audioBuffer = audioCtx.createBuffer(1, audio.length, audioCtx.sampleRate);
  audioBuffer.copyToChannel(audio, 0);
  return audioBuffer;
}

function normalizeAptSpikes(signal: Float32Array, kernelSize = 11, threshold = 3.0): Float32Array {
  const n = signal.length;
  const half = Math.floor(kernelSize / 2);
  const baseline = new Float32Array(n);

  for (let i = 0; i < n; i++) {
    const start = Math.max(0, i - half);
    const end = Math.min(n, i + half + 1);
    const window = Array.from(signal.subarray(start, end)).sort((a, b) => a - b);
    baseline[i] = window[Math.floor(window.length / 2)];
  }

  const residual = new Float32Array(n);
  for (let i = 0; i < n; i++) residual[i] = signal[i] - baseline[i];

  const mean = residual.reduce((s, x) => s + x, 0) / n;
  const std = Math.sqrt(residual.reduce((s, x) => s + (x - mean) ** 2, 0) / n);

  const clipped = new Float32Array(n);
  for (let i = 0; i < n; i++) {
    clipped[i] = Math.max(-threshold * std, Math.min(threshold * std, residual[i]));
  }

  const maxAbs = clipped.reduce((m, x) => Math.max(m, Math.abs(x)), 0) + 1e-9;
  return clipped.map(x => x / maxAbs);
}

function demodulateApt(signal: Float32Array, sampleRate: number, audioRate: number): Float32Array {
  const n = signal.length;

  // Hilbert transform via FFT (WebAudio doesn't have one, so we do it manually)
  const im = hilbertTransform(signal);

  // FM demod: instantaneous phase derivative
  const fmDemod = new Float32Array(n - 1);
  const amDemod = new Float32Array(n - 1);
  let prevPhase = Math.atan2(im[0], signal[0]);

  for (let i = 0; i < n - 1; i++) {
    const phase = Math.atan2(im[i], signal[i]);
    let delta = phase - prevPhase;
    // Unwrap
    while (delta > Math.PI) delta -= 2 * Math.PI;
    while (delta < -Math.PI) delta += 2 * Math.PI;
    fmDemod[i] = (delta / (2 * Math.PI)) * sampleRate;
    amDemod[i] = Math.sqrt(signal[i] ** 2 + im[i] ** 2);
    prevPhase = phase;
  }

  // Combine AM + FM
  let maxAbs = 1e-9;
  const combined = new Float32Array(n - 1);
  for (let i = 0; i < n - 1; i++) {
    combined[i] = fmDemod[i] * amDemod[i];
    maxAbs = Math.max(maxAbs, Math.abs(combined[i]));
  }
  for (let i = 0; i < combined.length; i++) combined[i] /= maxAbs;

  // Single-pole IIR lowpass
  const cutoff = Math.min(audioRate / 2, sampleRate * 0.45);
  const rc = 1.0 / (2 * Math.PI * cutoff);
  const dt = 1.0 / sampleRate;
  const alpha = dt / (rc + dt);
  const filtered = new Float32Array(combined.length);
  filtered[0] = combined[0];
  for (let i = 1; i < combined.length; i++) {
    filtered[i] = filtered[i - 1] + alpha * (combined[i] - filtered[i - 1]);
  }

  // Downsample
  const factor = Math.max(1, Math.round(sampleRate / audioRate));
  const out = new Float32Array(Math.floor(filtered.length / factor));
  for (let i = 0; i < out.length; i++) out[i] = filtered[i * factor];

  return out;
}

function hilbertTransform(signal: Float32Array): Float32Array {
  const taps = 63;
  const half = Math.floor(taps / 2);
  const h = new Float32Array(taps);

  for (let i = 0; i < taps; i++) {
    const n = i - half;
    if (n === 0 || n % 2 === 0) continue;
    const hann = 0.5 * (1 - Math.cos((2 * Math.PI * i) / (taps - 1)));
    h[i] = (2 / (Math.PI * n)) * hann;
  }

  const len = signal.length;
  const out = new Float32Array(len);
  for (let i = 0; i < len; i++) {
    let sum = 0;
    for (let k = 0; k < taps; k++) {
      const j = i - k + half;
      if (j >= 0 && j < len) sum += h[k] * signal[j];
    }
    out[i] = sum;
  }
  return out;
}

// --- Playback ---
async function play(url: string): Promise<void> {
  const audioCtx = new AudioContext();
  const buffer = await loadAndProcessSignal(url, audioCtx);
  const source = audioCtx.createBufferSource();
  source.buffer = buffer;
  source.connect(audioCtx.destination);
  source.start();
}