Signal Intelligence & Electronic Warfare Analysis System
COMPLETE TECHNICAL DOCUMENTATION
⬛ SECTION 1
SENTINAL (Signal Intelligence & Electronic Warfare Analysis System) is a real-time, multi-sensor military intelligence platform designed to detect, classify, track, and analyse adversary radar emitters and communications networks on a battlefield. It fuses Electronic Intelligence (ELINT) with Communications Intelligence (COMINT) to present commanders with a complete Recognized Electronic Picture (REP) and automated mission assessments.
The system is built around two simulated Electronic Warfare (EW) sensor networks --- EW System 1 and EW System 2 --- each comprising four geographically distributed sensors that passively detect radar pulses and communication emissions from enemy platforms within a 100×100 km operational area.
SENTINAL serves the following military intelligence objectives:
-
ELINT Collection: Detect, locate, and classify adversary radar emitters (ground surveillance, naval, AEW, fighter tracking radars) using passive radio frequency (RF) sensing.
-
COMINT Analysis: Monitor adversary communication networks, detect frequency hopping, identify jamming events, and assess communications anomalies.
-
SIGINT Correlation: Cross-correlate ELINT tracks with COMINT network activity to determine which radar platforms share communication networks --- revealing command relationships.
-
Mission Management: Automatically detect, classify, and track adversary tactical missions (air patrols, strike packages, electronic warfare operations, etc.) and score their threat level.
-
Common Operating Picture (COP): Provide a unified, timestamped situational display that integrates sensor data, tracks, missions, and alerts for operational commanders.
| Capability | Technology | Output |
|---|---|---|
| Radar Emitter Detection | DBSCAN clustering on PDW features | Emitter positions (x, y) with uncertainty |
| Radar Classification | Random Forest ML classifier | 4 radar types: GS / Naval / AEW / Fighter |
| Position Estimation | Angle-of-Arrival (AoA) triangulation | Geolocated emitter coordinates |
| Track Continuity | Kalman Filter (constant velocity) | Smoothed tracks with velocity & prediction |
| COMINT Clustering | Agglomerative Clustering on frequency | Network groups with statistics |
| Anomaly Detection | Isolation Forest (online learning) | Flagged anomalous transmissions |
| Mission Detection | Rule-based classifier + heuristics | Named missions with threat scores |
| SIGINT Correlation | Spatial + frequency scoring | Radar-COMINT platform associations |
| Media Analysis | YOLOv5 + Kalman Box Tracker | Object-tracked annotated video/images |
| Text Intelligence | NLTK NER + VADER Sentiment | Entities, keywords, sentiment labels |
⬛ SECTION 2
SENTINAL follows a pipeline architecture with five distinct layers. Data originates from synthetic sensor simulation, flows through a message broker, is processed by an intelligence engine, persisted to a database, and finally presented through a web-based dashboard.
{width="7.0in" height="4.530555555555556in"}
| Layer | Component | Technology | Role |
|---|---|---|---|
| 1 --- Data Generation | Dataset generators (EW1, EW2) | Python / NumPy / Pandas | Simulate realistic radar and COMINT data |
| 2 --- Streaming | Kafka Producer + Kafka Broker | Apache Kafka / Docker | Real-time message streaming by timestamp |
| 3 --- Intelligence Engine | Backend API Server | Python / Flask / scikit-learn | ML processing, tracking, mission analysis |
| 4 --- Persistence | PostgreSQL Database | PostgreSQL / psycopg2 | Store processed frames, summaries, analyses |
| 5 --- Presentation | Frontend Dashboard | HTML / CSS / JavaScript | Interactive COP and intelligence panels |
| 5b --- ELT Control | ICIC Interface Software (Adaptor) | HTML Single-Page App | Trigger ELT pipeline, manage system state |
The complete data flow through SENTINAL is described step by step below:
-
Dataset generators (datasetgeneration_v6_advancecomint.py, data_generation_v9_EW2.py) produce CSV files containing simulated ELINT and COMINT sensor readings for 300 simulation timesteps.
-
The Kafka Producer (kafka_producer.py) reads these CSVs, groups rows by timestamp, and publishes them as JSON messages to two Kafka topics: ew1.raw and ew2.raw.
-
The Backend Server (elint_server_new.py) consumes the CSV data (or reads it directly from disk), applies the full signal processing pipeline (clustering, classification, fusion, Kalman tracking, mission management), and stores per-frame results.
-
Processed frame data is persisted to PostgreSQL in three tables: frames, system_summaries, and analysis_cache.
-
The ICIC Interface Software (adaptor.html) triggers the ELT pipeline by calling POST /api/db/pg-reprocess followed by POST /api/ingest/reload, which loads the data from PostgreSQL into server RAM.
-
The Frontend Dashboard (index.html + script.js) queries the REST API at regular intervals to retrieve frames, tracks, COMINT data, and mission information, rendering them on the interactive display.
SENTINAL supports two Electronic Warfare system configurations simultaneously:
| Parameter | EW System 1 (EW1) | EW System 2 (EW2) |
|---|---|---|
| Sensor Count | 4 sensors | 4 sensors |
| Emitter Count | 4 emitters | 8 emitters |
| Emitter Types | 2 static, 2 moving | 5 static, 3 fighter jets (mobile) |
| Radar Types | GS, Naval, AEW, Fighter | GS, Naval, AEW, Fighter (×3 jets) |
| COMINT Networks | NET_A (VHF), NET_B (UHF) | NET_A, NET_B, NET_C (S-Band) |
| Dataset File | new_realistic_elint_comint_dataset_v2.csv | ew_system2_elint_comint_dataset_v4.csv |
| Kafka Topic | ew1.raw | ew2.raw |
| Display Colour | Cyan (#00c8ff) | Green (#00ff9d) |
| DBSCAN Epsilon | 0.3 | 0.3 |
| Association Distance | 15 km | 12 km |
⬛ SECTION 3
SENTINAL generates synthetic but physically realistic sensor datasets that model radar emitters (ELINT) and communication networks (COMINT) on a 100×100 km tactical map. Two generators produce separate datasets for the two EW systems.
Files: datasetgeneration_v6_advancecomint.py (EW1) | data_generation_v9_EW2.py (EW2)
-
Map Size: 100×100 km tactical grid (unitless coordinates 0--100).
-
Simulation Time: 300 timesteps at DT=1 second intervals (5 minutes of operation).
-
Random Seed: EW1 uses seed 42; EW2 uses seed 99 for reproducibility.
-
Sensors: 4 passive sensors at fixed positions (corners for EW1; trapezoid for EW2).
-
Emitters: 4 (EW1) or 8 (EW2) radar emitters with configurable movement.
EW System 1 places sensors at the four corners of the operational area, maximising AOA triangulation coverage. EW System 2 uses a trapezoid layout with sensors at (10,10), (90,10), (20,50), (80,50) to provide asymmetric coverage reflecting a more realistic deployed configuration.
For every (sensor, emitter, timestep) combination, the following ELINT features are computed and recorded, each corrupted with realistic Gaussian noise:
| Feature | Symbol | Noise Model | Description |
|---|---|---|---|
| Frequency | freq | σ = 0.5 MHz | Carrier frequency of the radar signal |
| Pulse Repetition Interval | PRI | σ = 2.0 µs | Time between consecutive radar pulses |
| Pulse Width | PW | σ = 0.15 µs | Duration of each transmitted radar pulse |
| Bandwidth | BW | Deterministic | Signal bandwidth in MHz |
| Peak Power | PP | Deterministic | Transmitted power in dBm |
| Scan Rate | SR | Deterministic | Antenna rotation rate in RPM |
| Duty Cycle | DC | Deterministic | Ratio of pulse-on to pulse period |
| Amplitude (RSSI) | amp | σ = 1.5 dB | Received signal strength (path-loss model) |
| Angle of Arrival | AoA | σ = 2.0° | Bearing from sensor to emitter |
| Time of Arrival | ToA | Calculated | Propagation delay at speed of light |
| Doppler Shift | doppler | σ = 1.0 Hz | Velocity-induced frequency shift |
| Signal-to-Noise Ratio | SNR | σ = 2.0 dB | Signal quality metric |
| Estimated Position | est_x, est_y | σ = 1.2 km | Noisy position estimate at each sensor |
| True Position | true_x, true_y | Ground truth | Exact emitter location (for evaluation) |
The path-loss model used for amplitude and RSSI follows the standard log-distance formula:
RSSI = peak_power − 20 × log10(distance + ε) + N(0, σ)
where ε is a small constant (1e-6) to prevent log(0) errors and N(0,σ) represents Gaussian noise.
Each emitter belongs to one of four radar types. The Pulse Descriptor Word (PDW) parameters used by the classifier must clearly separate these types in feature space:
| Radar Type | Code | Frequency Band | Freq (MHz) | PRI (µs) | PW (µs) | Scan Rate |
|---|---|---|---|---|---|---|
| Ground Surveillance Radar | 0 | UHF (rf_band=0) | ~950 | ~1200 | ~2.5 | 30 RPM |
| Naval Radar | 1 | L-Band (rf_band=1) | ~1300 | ~900 | ~1.8 | 25 RPM |
| AEW Radar | 2 | S-Band (rf_band=2) | ~3200 | ~600 | ~1.0 | 60 RPM |
| Fighter Tracking Radar | 3 | X-Band (rf_band=3) | ~9400 | ~300 | ~0.5 | 0 (tracking) |
Important: The fighter jets in EW System 2 (emitters E6, E7, E8) are deliberately designed with PDW values close to the EW1 fighter (E4) so the trained Random Forest model correctly classifies them as radar_type=3. This alignment between EW1 training data and EW2 test data is a key design decision.
Emitters have configurable velocity vectors (vx, vy). Static emitters (ground radars, naval units) have vx=vy=0. Mobile emitters follow linear trajectories, while E4 (AEW in EW1) uses a sinusoidal north-south oscillation simulating a racetrack orbit:
y += sin(t / 25) × 2 (applied to AEW emitter)
Fighter jets in EW2 move diagonally across the map at 0.3-0.4 km/s, simulating high-speed intercept profiles.
Each emitter belongs to one of two or three frequency-hopping communication networks. The COMINT simulation models tactical radio communication:
| Network | Frequency Band | Channels (MHz) | Hop Dwell (s) | Pattern Period (s) | EW Systems |
|---|---|---|---|---|---|
| NET_A | VHF/UHF (~890 MHz) | 890, 895, 900, 905, 910 | 2 | 8 | EW1, EW2 |
| NET_B | UHF (~1800 MHz) | 1780, 1790, 1800, 1810, 1820 | 3 | 10 | EW1, EW2 |
| NET_C | S-Band (~3100 MHz) | 3100, 3110, 3120, 3130, 3140 | 2 | 6 | EW2 only |
The simulation includes scripted tactical events that test the system's analytical capabilities:
| Time Window | Event | Affected Network | Effect on Data |
|---|---|---|---|
| t = 140--160 | Frequency anomaly | NET_B | Frequency drifts 600--1000 MHz above normal band |
| t = 150--170 | Communications jamming | NET_A, NET_B | SNR decreases 5--12 dB; comm_jammed flag = True |
| t = 200--230 | Communication blackout | NET_A, NET_B | All networks inactive (comm_active = False) |
When a network is active, the following COMINT fields are populated:
-
comm_frequency --- current hop channel frequency
-
comm_freq_band --- LOW / MID / HIGH band classification
-
comm_rssi --- received signal strength of the communication link
-
comm_snr --- signal-to-noise ratio of the communication link
-
comm_active --- Boolean flag indicating network transmission
-
comm_modulation --- modulation type: BPSK, QPSK, QAM, OFDM
-
comm_symbol_rate --- symbol rate in ksps
-
comm_bandwidth --- channel bandwidth (derived from symbol rate)
-
comm_traffic_load --- traffic intensity (Beta distribution, 0--1)
-
comm_packet_rate --- packets per second (load × 100)
-
comm_ber --- bit error rate (derived from SNR)
-
comm_packet_loss --- binary packet loss indicator
-
comm_jammed --- Boolean jamming indicator
-
comm_anomaly --- Boolean frequency anomaly indicator
-
comm_logical_id --- alternate network ID at t % 120 == 0 (simulates call-sign rotation)
-
comm_burst_id --- burst sequence number
When a network is silent, all numerical COMINT fields are set to NaN and string fields to "NONE".
⬛ SECTION 4
Apache Kafka provides the real-time data streaming layer between the dataset generators and the backend server. It decouples data production from consumption, enabling SENTINAL to ingest live sensor feeds in a production deployment while the simulation uses pre-generated CSV files.
The Kafka infrastructure is containerised using Docker Compose (docker-compose.yml) and consists of three services:
Apache Zookeeper manages Kafka broker metadata and leader election. It runs on port 2181 on the kafka-net Docker bridge network.
A single Kafka broker (confluentinc/cp-kafka:7.5.0) handles message storage and delivery. Key configuration parameters:
| Parameter | Value | Purpose |
|---|---|---|
| KAFKA_LISTENERS | PLAINTEXT://0.0.0.0:9092, PLAINTEXT_INTERNAL://0.0.0.0:29092 | Dual listener --- external (host) and internal (Docker) |
| KAFKA_ADVERTISED_LISTENERS | PLAINTEXT://localhost:9092, PLAINTEXT_INTERNAL://kafka:29092 | Advertised addresses for each listener |
| KAFKA_LOG_RETENTION_HOURS | 2 | Messages expire after 2 hours --- saves disk space |
| KAFKA_MESSAGE_MAX_BYTES | 20971520 | 20 MB maximum message size --- allows large frame batches |
| KAFKA_AUTO_CREATE_TOPICS_ENABLE | true | Topics created automatically on first publish |
| KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR | 1 | Single-node configuration --- no replication |
Provectus Kafka UI runs on port 8080 and provides a web-based administration interface for monitoring topics, consumer groups, and message throughput.
File: kafka_producer.py
The Kafka producer reads both EW system CSV datasets and streams them to their respective topics.
| Topic | Source Dataset | Message Type |
|---|---|---|
| ew1.raw | new_realistic_elint_comint_dataset_v2.csv | Timestamped batch of EW1 sensor rows |
| ew2.raw | ew_system2_elint_comint_dataset_v4.csv | Timestamped batch of EW2 sensor rows |
Each Kafka message represents one timestamp group --- all sensor readings for a single simulation step:
{
"system_id" : "ew1",
"timestamp" : 120,
"row_count" : 16,
"rows" : [ { ...full CSV row... }, ... ]
}
NaN and infinity values are converted to JSON null before serialisation using the _clean() helper to ensure valid JSON.
-
Acknowledgements: acks="all" --- waits for leader acknowledgement before considering a message sent.
-
Max Request Size: 20 MB --- matches broker configuration.
-
Retry Logic: Up to 6 connection attempts with 5-second delays to allow Docker containers to start.
-
Send Delay: SEND_DELAY = 0.0 --- streams as fast as possible; configurable for rate-controlled simulation.
⬛ SECTION 5
The radar classifier is a supervised machine learning model that identifies the type of radar emitter from its Pulse Descriptor Word (PDW) features. It is trained once offline and loaded by the backend server at startup for real-time inference.
File: train_classifier_v2.py → produces radar_classifier_v2.pkl
The model is a Random Forest Classifier with 300 decision trees, maximum depth 20. Random Forest was chosen because:
-
It is robust to noisy, correlated features (common in RF sensor data).
-
It provides class probability estimates (predict_proba), enabling confidence-weighted decisions.
-
It naturally handles the multi-class, imbalanced training scenario.
-
Feature importance scores identify the most discriminating PDW parameters.
Exactly nine features are used. This list is critical --- the backend CLASS_FEATURES constant must match exactly:
| Feature | Unit | Discriminating Power | Notes |
|---|---|---|---|
| frequency | MHz | VERY HIGH | Best single discriminator --- each type occupies distinct band |
| pri | µs | HIGH | Pulse repetition interval varies significantly by mode |
| pulse_width | µs | HIGH | Shorter PW correlates with higher-frequency fighters |
| bandwidth | MHz | MEDIUM | Wider BW for higher-frequency radars |
| scan_rate | RPM | HIGH | Tracking radars have scan_rate = 0 (no mechanical scan) |
| duty_cycle | ratio | MEDIUM | Derived from PW and PRI --- partially redundant |
| modulation_type | integer | MEDIUM | Encoded: 0=AM, 1=FM, 2=pulse-Doppler |
| doppler | Hz | LOW-MED | Non-zero for mobile emitters; noisy |
| snr | dB | LOW | Range-dependent; useful as quality weight |
Design Note: platform_type (whether the emitter is ground/ship/airborne) is intentionally excluded from features. The backend does not pass it to predict_proba(), so the model must classify purely from RF characteristics --- mimicking real-world scenarios where platform identity is unknown.
| Class Code | Radar Type Name | Representative Emitter | Typical Frequency |
|---|---|---|---|
| 0 | Ground Surveillance Radar | EW1: E1, EW2: E1 | 800--1000 MHz (UHF) |
| 1 | Naval Radar | EW1: E2, EW2: E2 | 1200--1400 MHz (L-Band) |
| 2 | Airborne Early Warning Radar | EW1: E3, EW2: E5 | 3100--3300 MHz (S-Band) |
| 3 | Fighter Tracking Radar | EW1: E4, EW2: E6-E8 | 9000--9600 MHz (X-Band) |
-
Load dataset: realistic_elint_dataset.csv (EW1 full dataset with ground truth labels).
-
Stratified 80/20 train/test split with random_state=42.
-
Train RandomForestClassifier(n_estimators=300, max_depth=20, n_jobs=1).
-
Evaluate: Accuracy, classification report (precision/recall/F1 per class), confusion matrix.
-
Sanity check: Feed a synthetic type-3 PDW (freq=9400, PRI=300, PW=0.5, BW=50, SR=0) --- must predict class 3.
-
Save model to radar_classifier_v2.pkl via joblib.
The model achieves high accuracy (~95%+) on EW1 data and is designed to generalise to EW2 fighter jet signatures because the EW2 dataset was specifically tuned to match the type-3 PDW distribution the model learned from EW1 E4.
⬛ SECTION 6
File: elint_server_new.py --- Flask REST API, Build-1 + Build-2
The backend server is the intelligence core of SENTINAL. It is a Flask application that performs the complete signal processing pipeline from raw sensor data to tactical intelligence products. It exposes 50+ REST API endpoints consumed by both the frontend dashboard and the ICIC Interface Software adaptor.
For each EW system, the backend runs the following processing pipeline over all 300 timesteps at startup (or when triggered by the adaptor):
The CSV dataset is loaded into a pandas DataFrame and sorted by timestamp. Feature availability is detected (COMINT V1 vs V2 columns). A StandardScaler is fitted on the CLUSTER_FEATURES to normalise for DBSCAN.
A temporal fusion window of 5 timesteps is maintained (FUSION_WINDOW=5). At each timestep t, the fusion window contains rows from t-4 to t. This increases the number of pulse descriptor words (PDWs) available for clustering, reducing false negatives for low-PRF radars that may not trigger every timestep.
DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is applied to the normalised CLUSTER_FEATURES:
CLUSTER_FEATURES = ["frequency", "pri", "pulse_width", "bandwidth", "scan_rate", "duty_cycle"]
Each cluster represents a group of PDWs likely originating from the same radar emitter. DBSCAN parameters:
-
epsilon (eps): 0.3 --- maximum distance between two points to be considered neighbours (in normalised feature space).
-
min_samples: 3 --- minimum number of PDWs to form a cluster core point.
-
Points labelled -1 are noise (no-emitter detections) and are discarded.
For each valid cluster, the mean of the nine CLASS_FEATURES is computed and passed to the Random Forest classifier. This returns:
-
radar_type --- predicted class (0=GS, 1=Naval, 2=AEW, 3=Fighter).
-
confidence --- probability of the predicted class (0--100%).
-
radar_name --- human-readable name from RADAR_NAMES dictionary.
For each cluster, sensor-level measurements are grouped by sensor_id and the mean AoA (Angle of Arrival) per sensor is computed. A least-squares triangulation is performed to estimate the emitter's 2D position:
For each sensor i: nx = -sin(AoA_i), ny = cos(AoA_i)
Solve: [nx, ny] · [ex, ey]ᵀ = nx·sx + ny·sy (over all sensors)
A minimum of 2 sensors with detections of the same cluster is required. The AoA residual (mean angular error after triangulation) is computed as a quality metric.
The COMINT block function processes all detections at each timestep:
-
Active detections (with valid comm_frequency) are separated from silent ones.
-
Agglomerative Clustering with distance threshold 120 MHz groups emitters by frequency into network clusters.
-
An Isolation Forest is trained online during the first LEARNING_WINDOWS=25 timesteps to learn the baseline COMINT feature distribution.
-
After training, the Isolation Forest labels any network cluster as anomalous (score = -1) if its features deviate significantly from the baseline.
-
Network statistics (frequency mean/std, RSSI, SNR, BER, traffic load, modulation) are computed per cluster.
A constant-velocity Kalman filter maintains continuous tracks across timesteps:
| Parameter | Value | Description |
|---|---|---|
| State vector | [x, y, vx, vy] | 2D position + velocity |
| Measurement | [x, y] | Triangulated position from AoA |
| Process noise Q | σ=0.08 | Small --- emitters move slowly or are static |
| Measurement noise R | σ=2.0 | Position uncertainty from AoA triangulation |
| Association distance | 15 km (EW1) / 12 km (EW2) | Maximum distance for track-detection association |
| Trail subsampling | Every 3rd frame | Stored history points for trail rendering |
Association uses nearest-neighbour gating: each new detection is assigned to the nearest existing track within the association distance. Unmatched detections spawn new tracks.
The MissionManager processes the list of active tracks at each timestep and generates tactical mission assessments. This is a sophisticated heuristic system operating in three stages:
8a. Track Grouping
Tracks are grouped into missions using a multi-criterion approach:
-
Tracks sharing the same COMINT network (comm_active=True, same cluster ID) are grouped together.
-
Remaining tracks are assigned to the nearest group centroid within GROUPING_RADIUS=45 km.
-
Nearby groups are merged iteratively until no two groups are within 45 km of each other.
-
Mixed mobile/static groups are split if mobile and static centroids are more than SPLIT_THRESHOLD=38 km apart.
8b. Mission Type Classification
Mission types are determined by a rule-based classifier examining the entity composition, speed, and signals data:
| Mission Type | Trigger Rules | Base Threat | Color |
|---|---|---|---|
| ELECTRONIC_WARFARE | Jamming detected | 0.78 | #c07aff |
| FORCE_PACKAGE | 4+ tracks, 2+ mobile, 3+ COMINT, 2+ fighters + mix | 0.70 | #ff8800 |
| COMBAT_AIR_PATROL | AEW + fighters | 0.62 | #ff3355 |
| STRIKE_PACKAGE | 2+ fighters, speed>0.3, coherence>0.6 | 0.85 | #ff0000 |
| FIGHTER_PATROL | 1+ fighter | 0.40 | #ff6600 |
| ISR_SURVEILLANCE | AEW only | 0.28 | #00c8ff |
| AIR_DEFENSE | 2+ ground/naval | 0.50 | #ffb800 |
| NAVAL_OPERATION | Naval present | 0.42 | #4db8ff |
| GROUND_SURVEILLANCE | Ground radar only | 0.22 | #00ff9d |
| UNKNOWN | No clear pattern | 0.20 | #888888 |
8c. Threat Score Calculation
The threat score (0.0--1.0) is computed from a weighted combination of factors:
score = base_threat(mission_type)
+ 0.12 × jamming_active
+ 0.08 × anomaly_detected
+ min(fighter_count × 0.05, 0.20)
+ min(avg_speed × 0.07, 0.10)
+ min((entity_count-1) × 0.025, 0.10)
+ heading_coherence × 0.06
Threat levels map score ranges: LOW (<0.30), MEDIUM (<0.55), HIGH (<0.75), CRITICAL (≥0.75).
Each processed timestep is stored in an in-memory frame_cache dictionary, then bulk-upserted to PostgreSQL using psycopg2.extras.execute_values for efficiency. The frame structure contains:
-
timestamp --- simulation timestep
-
tracks --- list of Kalman-tracked emitter objects with all ELINT and COMINT fields
-
sensors --- sensor positions from the current frame
-
n_clusters, n_tracks, n_comint_active, n_anomalies, n_jammed --- summary counts
-
comint_networks --- network cluster statistics dictionary
-
missions --- list of active mission assessments from MissionManager
-
max_threat_level, n_critical --- rapid-access threat summary
The EOB aggregates the full dataset into a per-emitter intelligence picture. For each emitter (keyed by true_emitter_id), it computes: radar name, type, platform, RF band, frequency range, mean PRI and PW, first/last seen timestamps, mobility assessment, track distance, and threat level. This provides a static reference of all detected enemy electronic assets.
The JAU analyses the scripted jamming window (t=150--170) by comparing pre-jam, during-jam, and post-jam COMINT statistics (SNR, BER, packet loss). It computes jamming effectiveness as the fractional SNR degradation and estimates recovery time by detecting when post-jam SNR returns to 85% of baseline.
The SIGINT correlation engine pairs ELINT tracks with COMINT network clusters at each timestep, computing a composite score (0.0--1.0) from:
-
Network ID match (score +0.50) --- track's true_net matches COMINT cluster ID.
-
Spatial colocation (score +0.30) --- track within 20 km of COMINT centroid.
-
Spatial proximity (score +0.15) --- track within 40 km.
-
Frequency band match (score +0.20) --- radar and COMINT share same band.
-
Co-jamming signature (score +0.10) --- both track and network are jammed simultaneously.
Pairs scoring ≥0.65 are flagged as likely same platform, enabling identification of radar-equipped communications units.
⬛ SECTION 7
SENTINAL persists all processed intelligence to PostgreSQL using three tables. The schema is auto-created on startup using IF NOT EXISTS DDL, making it safe to run multiple times:
| Column | Type | Description |
|---|---|---|
| system_id | TEXT NOT NULL | EW system identifier ("ew1" or "ew2") |
| timestamp | INTEGER NOT NULL | Simulation timestep (0--300) |
| data | JSONB NOT NULL | Complete frame object: tracks, missions, COMINT networks |
| processed_at | TIMESTAMPTZ | When this frame was saved or last updated |
Primary key: (system_id, timestamp). Bulk upserted using execute_values with page_size=200 for performance.
| Column | Type | Description |
|---|---|---|
| system_id | TEXT NOT NULL | Primary key --- one row per EW system |
| comint_summary | JSONB | Per-network aggregate statistics across all frames |
| mission_summary | JSONB | Per-mission lifetime record (phases seen, peak threat) |
| config | JSONB | System configuration parameters (label, colour, DBSCAN params) |
| Column | Type | Description |
|---|---|---|
| system_id | TEXT NOT NULL | EW system identifier |
| cache_type | TEXT NOT NULL | "eob" or "jau" |
| data | JSONB NOT NULL | Pre-computed EOB or JAU analysis result |
Connection parameters are configured via environment variables with defaults:
SENTINAL_PG_HOST → localhost
SENTINAL_PG_PORT → 5432
SENTINAL_PG_DB → sentinal
SENTINAL_PG_USER → sentinal_user
SENTINAL_PG_PASSWORD → sentinal_pass
The backend follows a clear priority for data loading on startup:
-
If AUTOLOAD_OVERRIDE flag (--autoload) is active AND PostgreSQL has existing data for a system AND --reprocess is NOT set: load directly from PostgreSQL (fastest path).
-
If PostgreSQL has no data OR --reprocess is set: process from CSV → save to PostgreSQL.
-
Default (no flags): server starts empty --- the ICIC Interface Software adaptor is the only trigger.
⬛ SECTION 8
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/systems | List all loaded EW systems with metadata and frame counts |
| POST | /api/switch | Switch the active EW system {system: "ew1"|"ew2"} |
| GET | /api/timestamps | Get list of all available timestamps for a system |
| GET | /api/sensors | Get sensor positions for the first frame of active system |
| GET | /api/health | System health snapshot (components, alerts, RAM/CPU) |
| GET | /api/health/history | Last N health snapshots (rolling 30-second interval) |
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/frame/<t> | Get complete frame at timestep t: tracks, COMINT, missions |
| GET | /api/all_frames | Get all frames as one bulk JSON (large response ~50MB) |
| GET | /api/cop?t=<t> | Common Operating Picture at t: emitters, missions, alerts, FEBA |
| GET | /api/cop/animation | Lightweight animation frames (t_start/t_end/step params) |
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/elint/prediction?t=<t>&steps=<n> | Kalman trajectory prediction for N future steps |
| GET | /api/elint/patterns/temporal | Time-binned ELINT statistics (mean freq, SNR, detection count) |
| GET | /api/elint/patterns/spectral | Frequency distribution by radar type + histogram |
| GET | /api/elint/patterns/spatial?t=<t> | Spatial density grid and emitter positions at timestep t |
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/comint/summary | Per-network aggregate statistics across all timesteps |
| GET | /api/comint/clusters?t=<t> | Active network clusters at timestep t with spatial grouping |
| GET | /api/comint/traffic?network=NET_A | Traffic load / BER / jamming timeline for a network |
| GET | /api/comint/prediction?t=<t>&steps=<n> | Predicted network activity windows for N future steps |
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/missions?t=<t> | Auto-detected missions at t + all manual missions |
| POST | /api/missions | Create a manual mission (operator-annotated) |
| GET | /api/missions/<id> | Get single mission by ID |
| PUT | /api/missions/<id> | Update a manual mission (notes, status, phase, threat) |
| DELETE | /api/missions/<id> | Delete a manual mission |
| GET | /api/mission/summary | All-time mission summary: lifetime, phases, peak threats |
| GET | /api/missions/<id>/analysis | Detailed mission analysis: AOP, FEBA, track history, timeline |
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/sigint/correlation?t=<t> | ELINT-COMINT correlation scores at timestep t |
| GET | /api/sigint/report | Aggregated SIGINT report over sampled timesteps |
| GET | /api/eob | Full Electronic Order of Battle for active system |
| GET | /api/eob/summary | EOB summary: counts by type, mobility, threat level |
| GET | /api/eob/combined | De-duplicated EOB combining both EW systems |
| GET | /api/jau/analysis | Jamming Analysis Unit report (cached) |
| GET | /api/jau/timeline | Timestep-level jamming timeline with per-network SNR/BER |
The backend maintains curated reference libraries used for cross-matching detected emitters against known threat signatures:
| Endpoint Prefix | Library Name | Contents |
|---|---|---|
| /api/global/radar-library | Radar Library | 8 known radar systems (P-18, Top Plate, Zaslon, etc.) with freq/PRI ranges |
| /api/global/bts-library | BTS Library | 3 base transceiver station profiles for NET_A, NET_B, NET_C |
| /api/global/vulnerable-assets | Vulnerable Assets | 3 own-force assets (supply base, radar station, mobile HQ) with priority |
| /api/global/formation-hq | Formation HQ | 2 headquarters locations at corps and brigade level |
| /api/global/satellite-footprints | Satellite Footprints | 3 satellites (Cartosat-3, RISAT-2BR1, GSAT-7A) with coverage polygons |
| /api/global/comm-library | COMINT Emitter Library | 6 communication emitter profiles from tactical VHF to covert burst |
| /api/global/threat-library | Threat Library | 8 threat profiles with countermeasure recommendations |
All libraries support GET (list), POST (create), GET/<id> (retrieve), PUT/<id> (update), DELETE/<id> (remove). The radar-library and comm-library also have /match endpoints that find the closest entry given detected parameters.
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/db/status | RAM store sizes, PostgreSQL table stats, memory usage |
| GET | /api/db/pg-stats | Live PostgreSQL row counts and table sizes |
| POST | /api/db/pg-reprocess | Force reprocess from CSV and save to PostgreSQL |
| POST | /api/ingest/reload | Load data from PostgreSQL into server RAM (adaptor trigger) |
| GET | /api/db/backup | Download JSON backup of all in-memory stores |
| POST | /api/db/restore | Restore in-memory stores from backup JSON |
| DELETE | /api/db/purge?store=<name> | Purge frames from PostgreSQL or clear in-memory stores |
| Method | Endpoint | Description |
|---|---|---|
| GET | /api/gis/coverage?t=<t> | Sensor coverage circles, overlap zones, 10×10 grid coverage heatmap |
| POST | /api/gis/shortest-path | Route planning with A* pathfinding avoiding high-threat zones |
| GET | /api/export/geojson?t=<t>&layers=all | Export COP as GeoJSON (EPSG:4326 lat/lng) with emitter/mission/FEBA layers |
| GET | /api/reports/ew-summary | EW system summary report (JSON or CSV format) |
| GET | /api/reports/mission/<id> | Detailed mission analysis report with recommendations |
The backend implements a role-based access control (RBAC) system with three roles:
| Role | Level | Access |
|---|---|---|
| operator | 0 | Basic data read access |
| commander | 1 | Operator access + threat library management |
| admin | 2 | Full access including user management, purge, backup/restore |
In development mode (DEV_MODE=True), all role checks pass automatically. For production, JWT tokens are supported via flask_jwt_extended. Pre-configured users: operator1/op1pass, commander1/cmd1pass, admin1/adm1pass.
⬛ SECTION 9
File: media_blueprint.py --- Flask Blueprint registered as /api/media/*
The Media Analysis Module extends SENTINAL with computer vision and natural language processing capabilities. It provides military analysts with the ability to process surveillance video footage, analyse reconnaissance images, and extract intelligence from intercepted text communications.
Video processing uses YOLOv5 (You Only Look Once v5) for real-time object detection. The detector supports custom weight files (.pt) placed in the weights/ directory, enabling domain-specific models trained on military vehicle datasets.
-
Confidence threshold: 0.40 --- detections below this are discarded.
-
IoU threshold: 0.45 --- non-maximum suppression threshold.
-
Image size: 640×640 pixels --- YOLOv5 standard inference size.
-
Device: Automatic GPU (CUDA) detection, falls back to CPU.
A custom Kalman filter-based multi-object tracker maintains consistent track IDs across frames, preventing ID swaps when objects cross or occlude each other:
-
State vector: [x1, y1, x2, y2, vx1, vy1, vx2, vy2] --- bounding box corners + velocities.
-
Track assignment: Hungarian algorithm (scipy linear_sum_assignment) minimising weighted distance/IoU cost.
-
Track confirmation: min_hits=3 --- a track must be matched 3 consecutive frames before being reported.
-
Track death: max_age=20 --- tracks unmatched for 20 frames are deleted.
-
Smooth output: Exponential moving average (α=0.6) on bounding box coordinates.
Processed video is encoded as H.264 MP4, which is browser-compatible for streaming playback. The system supports two encoding paths:
-
Path A (ffmpeg available): Write MJPEG to temporary AVI → re-encode with libx264/H.264 for optimal quality.
-
Path B (no ffmpeg): Write directly using OpenCV mp4v/avc1 codec.
HTTP Range request support is implemented manually in the /result endpoint, enabling browsers to seek within video without re-downloading.
Video processing is asynchronous. Each uploaded video creates a job with a unique 8-character ID. Jobs progress through states: queued → loading_model → processing → re_encoding → done (or error). The client polls /api/media/video/status/<job_id> to check progress.
Single images can be analysed without creating a persistent job. POST /api/media/image/analyze accepts an image file, runs YOLOv5 detection, draws annotated bounding boxes, and returns the result as a base64-encoded JPEG plus structured detection data (bounding boxes, class names, confidence scores, class counts).
SENTINAL includes an NLP pipeline for analysing intercepted text communications, built on NLTK to avoid native extension dependencies:
NLTK Part-of-Speech tagging and NE chunking extract named entities from text. Entity types are mapped to standard labels with colour codes for the frontend:
| Entity Type | NLTK Source | Color | Examples |
|---|---|---|---|
| PERSON | PERSON | #00d4e8 | "General Ahmad", "Lt Col Singh" |
| ORG | ORGANIZATION | #a78bfa | "Alpha Company", "3rd Brigade" |
| GPE | GPE / GSP | #10b981 | "Sector 7", "Northern Valley" |
| LOC | LOCATION | #34d399 | "Ridge Line Alpha", "River Crossing" |
| FAC | FACILITY | #f59e0b | "Radar Station Bravo", "FOB Delta" |
Frequency-based keyword extraction with POS filtering (nouns, verbs, adjectives) and stop-word removal identifies operationally significant terms. Results are ranked by frequency with normalised weight scores.
VADER (Valence Aware Dictionary and sEntiment Reasoner) computes compound sentiment scores (-1 to +1) with labels NEGATIVE / NEUTRAL / POSITIVE. The "subjectivity" proxy is computed as 1 − VADER neutrality score.
The /api/media/text/cluster endpoint accepts 2--20 documents and returns per-document analysis plus a TF-IDF cosine similarity matrix and K-Means cluster labels (k=min(3, n_docs)).
⬛ SECTION 10
File: adaptor.html --- Standalone HTML single-page application
The ICIC (ICIC Interface and Control) Interface Software is the operator-facing ELT (Extract, Load, Transform) pipeline controller. It is the sole authorised trigger for loading data into the backend server in production mode. This design ensures data provenance --- no data reaches the server's analysis engine without explicit operator action through this interface.
The adaptor orchestrates the two-step ELT pipeline:
-
Operator opens adaptor.html in a browser and enters the backend API URL.
-
The adaptor pings the backend health endpoint to confirm connectivity.
-
Operator selects one or both EW systems and clicks "Run ELT Pipeline".
-
Adaptor calls POST /api/db/pg-reprocess --- backend processes CSV datasets and saves all frames to PostgreSQL. This may take 30-120 seconds.
-
Adaptor calls POST /api/ingest/reload --- backend loads processed data from PostgreSQL into RAM, making it available for API queries.
-
Frontend dashboard is now operational --- all API endpoints return live data.
The adaptor displays a real-time status dashboard with four indicator panels:
-
Backend API: Tests connectivity to /api/health endpoint.
-
Kafka Broker: Checks Kafka component status from health response.
-
PostgreSQL: Checks database component status from health response.
-
Data Loaded: Checks if SYSTEM_DATA is populated (server_ready flag).
-
Decoupled Control: The adaptor is a separate HTML file that communicates exclusively via REST API --- no shared code with the main frontend.
-
CORS-first: The backend adds permissive CORS headers to all responses, enabling the adaptor to run from any origin (file://, localhost, or a separate server).
-
Operator Override: The --autoload flag disables the adaptor-controlled mode for development/debugging, restoring automatic loading.
⬛ SECTION 11
Files: index.html | style.css | script.js
The SENTINAL frontend is a tactical intelligence dashboard built as a single-page web application using vanilla HTML, CSS, and JavaScript (no frameworks). It communicates exclusively with the backend REST API and presents all intelligence products in a dark, military-grade interface.
The interface follows a consistent dark tactical aesthetic:
-
Colour scheme: Pure black background (#000) with deep navy panels, blue accent (#1F7CF4), cyan highlights (#38BDF8), violet for COMINT, green for SIGINT.
-
Typography: System UI fonts at small sizes (8--14px) with heavy use of letter-spacing and ALL CAPS labels --- mimicking military display terminals.
-
Scanline overlay: A subtle repeating CSS gradient creates a retro CRT scanline effect across the entire interface.
-
Background grid: Fine 60px grid lines at ultra-low opacity provide spatial reference without distracting from content.
| Panel | Function | Primary Data Source |
|---|---|---|
| System Selector | Switch between EW1 and EW2 | GET /api/systems |
| Timeline Scrubber | Navigate through 300 timesteps | GET /api/timestamps |
| Tactical Map / COP | Interactive 2D map with tracks, sensors, FEBA | GET /api/cop?t=<t> |
| Track Table | Live list of all active ELINT tracks | GET /api/frame/<t> |
| COMINT Panel | Network clusters, traffic stats, anomalies | GET /api/comint/clusters |
| Mission Panel | Active mission list with threat scores | GET /api/missions?t=<t> |
| EOB Panel | Electronic Order of Battle reference | GET /api/eob |
| SIGINT Panel | Radar-COMINT correlation pairs | GET /api/sigint/correlation |
| JAU Panel | Jamming analysis chart and timeline | GET /api/jau/analysis |
| Statistics Panel | Temporal, spectral, spatial analytics | GET /api/stats |
| Media Panel | Video/image/text intelligence analysis | GET /api/media/* |
| Admin Panel | Database management, health, logs | GET /api/db/status |
The dashboard supports both manual frame stepping and automated playback:
-
Frame-by-frame navigation using previous/next buttons.
-
Auto-play mode cycles through all timesteps at configurable speed.
-
Timeline scrubber allows direct jump to any timestamp.
-
All panels update synchronously when the active timestamp changes.
⬛ SECTION 12
| Component | Version | Installation |
|---|---|---|
| Python | 3.10+ | python.org or conda |
| Docker Desktop | Latest | docker.com --- for Kafka infrastructure |
| Node.js | 16+ | nodejs.org --- for npm/docx if needed |
| PostgreSQL | 13+ | Local install or Docker |
| FFmpeg | Any | ffmpeg.org --- optional, for H.264 video encoding |
Install all Python dependencies using pip:
pip install flask flask-cors numpy pandas scikit-learn joblib
pip install psycopg2-binary kafka-python opencv-python torch torchvision scipy nltk
docker compose up -d
Wait 20--30 seconds for Kafka to become fully ready. Access Kafka UI at http://localhost:8080.
python datasetgeneration_v6_advancecomint.py # EW System 1
python data_generation_v9_EW2.py # EW System 2
Each script produces a CSV file in the current directory (~4800--14400 rows).
python train_classifier_v2.py
Requires a realistic_elint_dataset.csv in the ELINT_classification_model_training/ directory. Produces radar_classifier_v2.pkl.
python kafka_producer.py
Reads both CSV datasets and publishes them to ew1.raw and ew2.raw topics.
python elint_server_new.py
Server starts on http://localhost:5000. In default mode (no flags), RAM is empty --- the adaptor must be used to trigger data loading. To auto-load from DB: python elint_server_new.py --autoload. To force full reprocess: python elint_server_new.py --autoload --reprocess.
Open adaptor.html in a browser. Set API URL to http://localhost:5000. Click "Run ELT Pipeline". Wait for completion (the two-step reprocess → reload sequence).
Open index.html in a browser (or serve via a local HTTP server). The dashboard connects to localhost:5000 and displays all intelligence panels.
| Flag | Effect |
|---|---|
| (none) | Default: server starts empty, adaptor-controlled mode |
| --autoload | Load data from PostgreSQL on startup (bypasses adaptor requirement) |
| --reprocess | Force CSV reprocessing even if PostgreSQL has existing data |
| --autoload --reprocess | Full reprocess from CSV + save to DB + load to RAM on startup |
⬛ SECTION 13
Every row in both EW1 and EW2 CSV datasets represents a single (sensor, emitter, timestep) observation. All 44 columns are described below:
| Column Name | Type | Description |
|---|---|---|
| timestamp | int | Simulation time step (0 to 300) |
| sensor_id | str | Sensor identifier: S1, S2, S3, or S4 |
| sensor_x / sensor_y | float | Fixed sensor coordinates on the 100×100 km map |
| est_x / est_y | float | Noisy position estimate at this sensor (σ=1.2 km Gaussian) |
| true_x / true_y | float | Ground-truth emitter position (not available to algorithm) |
| frequency | float | Measured carrier frequency (MHz) with noise (σ=0.5 MHz) |
| pri | float | Pulse Repetition Interval (µs) with noise (σ=2.0 µs) |
| pulse_width | float | Pulse width (µs) with noise (σ=0.15 µs) |
| bandwidth | float | Signal bandwidth (MHz), deterministic |
| peak_power | float | Emitter peak power (dBm), deterministic |
| scan_rate | float | Antenna rotation rate (RPM), 0 for tracking radars |
| duty_cycle | float | PW/PRI ratio, deterministic |
| amplitude | float | Received RSSI (dBm) --- path-loss model + noise |
| aoa | float | Angle of Arrival from sensor to emitter (degrees) + noise |
| toa | float | Time of Arrival --- simulation time + propagation delay |
| doppler | float | Doppler frequency shift (Hz) from emitter velocity |
| snr | float | Signal-to-noise ratio (dB) at this sensor |
| modulation_type | int | Modulation: 0=AM, 1=FM/pulse, 2=pulse-Doppler |
| radar_type | int | Ground truth class: 0=GS, 1=Naval, 2=AEW, 3=Fighter |
| platform_type | int | Platform: 0=ground, 1=ship/mobile, 2=airborne |
| rf_band | int | RF band: 0=VHF/UHF, 1=L-Band, 2=S-Band, 3=X-Band |
| scan_type | int | Scan: 0=pencil/non-scanning, 1=mechanically scanning |
| radar_mode | int | Mode: 0=search, 1=track |
| comm_frequency | float | Current comm frequency (MHz) --- NaN when silent |
| comm_freq_band | str | Band: LOW/MID/HIGH/NONE |
| comm_network_id | str | Network: NET_A, NET_B, NET_C, or NONE |
| comm_rssi | float | Comm signal RSSI (dBm) --- NaN when silent |
| comm_active | bool | True if network is transmitting at this timestep |
| comm_burst_id | int | Burst sequence count (-1 when silent) |
| comm_anomaly | bool | True if frequency anomaly detected (NET_B t=140-160) |
| comm_snr | float | Comm link SNR (dB) --- NaN when silent |
| comm_modulation | str | Modulation type: BPSK/QPSK/QAM/OFDM/NONE |
| comm_symbol_rate | float | Symbol rate (ksps) --- NaN when silent |
| comm_bandwidth | float | Channel bandwidth (kHz) --- NaN when silent |
| comm_traffic_load | float | Traffic intensity 0--1 (Beta distribution) |
| comm_packet_rate | float | Packets/second = traffic_load × 100 |
| comm_ber | float | Bit error rate --- derived from comm_snr |
| comm_packet_loss | int | Binary packet loss (Bernoulli based on BER) |
| comm_jammed | bool | True during jamming window (t=150--170) |
| comm_logical_id | str | Logical network ID (alternates at t%120==0) |
| true_emitter_id | str | Ground truth emitter: E1 through E8 |
⬛ SECTION 14
| Term | Full Form / Definition |
|---|---|
| AEW | Airborne Early Warning --- high-altitude radar aircraft for early detection |
| AoA | Angle of Arrival --- bearing angle from a sensor to a transmitter |
| BER | Bit Error Rate --- fraction of received bits that are erroneous |
| COMINT | Communications Intelligence --- intelligence from communications interception |
| COP | Common Operating Picture --- unified situational awareness display |
| DBSCAN | Density-Based Spatial Clustering of Applications with Noise |
| DC | Duty Cycle --- ratio of pulse-on time to total period (PW/PRI) |
| DLRL | Defence Laboratory for Research on Languages (referenced in JIM/JIS stubs) |
| ECM | Electronic Counter-Measures --- jamming and deception techniques |
| ELINT | Electronic Intelligence --- intelligence from non-communication RF emissions |
| EOB | Electronic Order of Battle --- catalogue of adversary electronic assets |
| ELT | Extract, Load, Transform --- the data pipeline managed by the adaptor |
| EW | Electronic Warfare --- use of electromagnetic spectrum for military advantage |
| FEBA | Forward Edge of the Battle Area --- tactical boundary line |
| GCI | Ground Controlled Intercept --- ground-based radar directing fighter aircraft |
| ICIC | In-Country Intelligence Cell / Interface and Control (adaptor naming) |
| ISR | Intelligence, Surveillance, and Reconnaissance |
| JAU | Jamming Analysis Unit --- subsystem analysing communications jamming events |
| JIM | Joint Intelligence Module --- planned CDR/IMSI analysis component |
| JIS | Joint Intelligence System --- planned call detail record analysis |
| NER | Named Entity Recognition --- NLP technique for extracting proper nouns |
| PDW | Pulse Descriptor Word --- set of measured parameters characterising a radar pulse |
| PRI | Pulse Repetition Interval --- time between consecutive radar pulses (µs) |
| RBAC | Role-Based Access Control --- access governed by user role assignments |
| REP | Recognized Electronic Picture --- comprehensive picture of electromagnetic environment |
| RF | Radio Frequency --- electromagnetic frequencies used in radar and communications |
| RSSI | Received Signal Strength Indicator --- measured power of received signal |
| SAR | Synthetic Aperture Radar --- radar imaging using aircraft or satellite motion |
| SENTINAL | Signal Intelligence & Electronic Warfare Analysis System (project codename) |
| SIGINT | Signals Intelligence --- intelligence from any electromagnetic signals |
| SNR | Signal-to-Noise Ratio --- signal power relative to background noise (dB) |
| VADER | Valence Aware Dictionary and sEntiment Reasoner --- NLP sentiment tool |
| YOLO | You Only Look Once --- single-pass real-time object detection framework |
⬛ SECTION 15
-
Synthetic Data Only: All datasets are synthetically generated with simplified physics models. Real sensor data would require calibrated antenna patterns, multi-path propagation models, and hardware-specific noise characteristics.
-
Single-Node Kafka: The Docker Compose configuration uses a single-broker Kafka cluster with replication factor 1. Production deployment requires a multi-broker cluster for fault tolerance.
-
In-Memory Mission State: MissionManager state is not persisted between server restarts. Post-restart, mission IDs restart from MSN-000.
-
JIM/JIS Stubs: Phone call detail record (CDR) analysis and IMSI tracking endpoints return simulated data only, pending classified input data from DLRL.
-
Voice/Image Analysis Stubs: The /api/analysis/voice and /api/analysis/image endpoints return simulated results, pending Wav2Vec2/Whisper and ENVI/YOLOv8 integration.
-
No Real-Time Kafka Consumer: The backend currently reads CSV files directly rather than consuming from Kafka topics. A Kafka consumer thread would enable true streaming operation.
-
Single-Threaded Processing: The ELT pipeline processes timesteps sequentially. Parallel processing would significantly reduce the 30-120 second reprocessing time.
-
Real Kafka consumer integration with the backend for true streaming intelligence.
-
Multi-hypothesis tracking (MHT) to handle track ambiguity in dense emitter environments.
-
Frequency deinterleaving to separate interleaved pulses from co-channel emitters.
-
3D position estimation using ToA-based multilateration to complement AoA triangulation.
-
Integration with DLRL classified data for JIM CDR analysis and voice transcription.
-
BERT-based named entity recognition for higher NER accuracy on military text.
-
RESTful mission collaboration --- multi-operator simultaneous mission annotation.
-
GIS integration with real geographic coordinate systems and terrain data.
⬛ SECTION 16
| File | Purpose | Key Dependencies |
|---|---|---|
| datasetgeneration_v6_advancecomint.py | EW System 1 dataset generator | numpy, pandas |
| data_generation_v9_EW2.py | EW System 2 dataset generator | numpy, pandas |
| train_classifier_v2.py | Train Random Forest radar classifier | scikit-learn, joblib |
| kafka_producer.py | Stream CSV data to Kafka topics | kafka-python, pandas |
| docker-compose.yml | Kafka + Zookeeper + Kafka UI containers | Docker, Confluent |
| elint_server_new.py | Main backend intelligence API server | flask, sklearn, psycopg2, numpy |
| media_blueprint.py | Video/image/text analysis Flask Blueprint | opencv, torch, nltk, scipy |
| adaptor.html | ICIC Interface Software ELT controller | Vanilla HTML/JS (standalone) |
| index.html | Main frontend dashboard shell | HTML, references style.css + script.js |
| style.css | Frontend CSS design system | Pure CSS (no frameworks) |
| script.js | Frontend application logic and API client | Vanilla JavaScript |
| radar_classifier_v2.pkl | Trained Random Forest model (generated) | joblib |
| new_realistic_elint_comint_dataset_v2.csv | EW1 processed dataset (generated) | Pandas CSV |
| ew_system2_elint_comint_dataset_v4.csv | EW2 processed dataset (generated) | Pandas CSV |
END OF DOCUMENT
SENTINAL v5.4 --- Technical Documentation | Generated from full source code analysis