graph TB
subgraph ROBOT["🤖 Boston Dynamics Spot"]
CAM_FL[Front Left Fisheye]
CAM_FR[Front Right Fisheye]
CAM_L[Left Fisheye]
CAM_R[Right Fisheye]
CAM_B[Back Fisheye]
CAM_H[Hand Color]
CAM_D[Depth Cameras]
SENSORS[IMU / Odometry / Joint States]
end
subgraph BACKEND["⚙️ Backend - Python Flask"]
SDK[Spot SDK Bridge]
PARVS_ENGINE[PARVS Streaming Engine]
REST[REST API<br/>Control / Telemetry]
WS_SERVER[WebSocket Server]
end
subgraph FRONTEND["🖥️ Frontend - React"]
PARVS_CLIENT[PARVS WebSocket Client]
CANVAS[Canvas Compositor]
UI[Command Center UI]
METRICS_DISPLAY[Metrics Overlay]
end
ROBOT -->|Spot SDK gRPC| SDK
SDK -->|Raw Frames| PARVS_ENGINE
SDK -->|Robot State| PARVS_ENGINE
SDK -->|Telemetry| REST
PARVS_ENGINE -->|Binary Tile Packets| WS_SERVER
WS_SERVER <-->|WebSocket| PARVS_CLIENT
REST <-->|HTTP JSON| UI
PARVS_CLIENT -->|Decoded Tiles| CANVAS
CANVAS -->|Rendered Frames| UI
PARVS_CLIENT -->|Bandwidth Stats| METRICS_DISPLAY
UI -->|Mouse Position| PARVS_CLIENT
UI -->|Movement Commands| REST
style ROBOT fill:#1a1a2e,stroke:#e94560,color:#fff
style BACKEND fill:#16213e,stroke:#0f3460,color:#fff
style FRONTEND fill:#0f3460,stroke:#533483,color:#fff
style PARVS_ENGINE fill:#e94560,stroke:#fff,color:#fff
style PARVS_CLIENT fill:#e94560,stroke:#fff,color:#fff
style CANVAS fill:#533483,stroke:#fff,color:#fff
graph TD
A[📷 Camera Capture<br/>Raw Frame 1280×720] --> B[🔺 Pyramid Generator]
B --> L0[Layer 0<br/>80×60<br/>~1.5 KB]
B --> L1[Layer 1<br/>320×240<br/>~6 KB]
B --> L2[Layer 2<br/>640×480<br/>~18 KB]
B --> L3[Layer 3<br/>1280×720<br/>~50 KB]
L0 --> C[🔲 Tile Segmenter<br/>8×6 Grid = 48 Tiles]
L1 --> C
L2 --> C
L3 --> C
C --> D[🎯 S-ROI Priority Engine]
RS[🤖 Robot State] --> D
RS --> |Velocity Vector| D1[Movement Alignment<br/>w₁ = 0.30]
RS --> |Depth Data| D2[Obstacle Proximity<br/>w₂ = 0.25]
PF[📊 Previous Frame] --> D3[Temporal Change<br/>w₃ = 0.20]
MP[🖱️ Mouse Position] --> D4[Operator Attention<br/>w₄ = 0.10]
RS --> |Arm Joints| D5[Arm Workspace<br/>w₅ = 0.15]
D1 --> D
D2 --> D
D3 --> D
D4 --> D
D5 --> D
D --> E[📊 Priority Heatmap<br/>P_i ∈ 0.0 - 1.0]
E --> F[📡 Bandwidth Controller]
BW_REPORT[📈 Client BW Report] --> F
F --> G[💰 Budget Allocator<br/>& Tile Selector]
G --> H{Keyframe?}
H -->|Every Nth frame| I[🔑 Keyframe<br/>Send ALL assigned tiles]
H -->|Other frames| J[Δ Delta Encoder<br/>Send CHANGED tiles only]
I --> K[📦 Packet Serializer<br/>Binary Protocol]
J --> K
K --> M[🔌 WebSocket TX<br/>Priority-Ordered]
style A fill:#2196F3,stroke:#fff,color:#fff
style B fill:#FF9800,stroke:#fff,color:#fff
style C fill:#9C27B0,stroke:#fff,color:#fff
style D fill:#E91E63,stroke:#fff,color:#fff
style E fill:#E91E63,stroke:#fff,color:#fff
style F fill:#4CAF50,stroke:#fff,color:#fff
style G fill:#00BCD4,stroke:#fff,color:#fff
style K fill:#FF5722,stroke:#fff,color:#fff
style M fill:#F44336,stroke:#fff,color:#fff
style L0 fill:#4CAF50,stroke:#fff,color:#fff
style L1 fill:#8BC34A,stroke:#fff,color:#fff
style L2 fill:#CDDC39,stroke:#000,color:#000
style L3 fill:#FFEB3B,stroke:#000,color:#000
graph LR
RAW[Raw Frame<br/>1280 × 720<br/>~2.7 MB raw] --> RESIZE
subgraph RESIZE["cv2.resize + cv2.imencode"]
direction TB
R0["resize → 80×60<br/>imencode JPEG q=60"] --> OUT0["Layer 0<br/>~1.5 KB<br/>⏱️ < 1ms"]
R1["resize → 320×240<br/>imencode JPEG q=70"] --> OUT1["Layer 1<br/>~6 KB<br/>⏱️ < 1ms"]
R2["resize → 640×480<br/>imencode JPEG q=80"] --> OUT2["Layer 2<br/>~18 KB<br/>⏱️ < 1ms"]
R3["No resize (original)<br/>imencode JPEG q=85"] --> OUT3["Layer 3<br/>~50 KB<br/>⏱️ < 2ms"]
end
OUT0 --> TILES[Tile Segmenter]
OUT1 --> TILES
OUT2 --> TILES
OUT3 --> TILES
style RAW fill:#2196F3,stroke:#fff,color:#fff
style OUT0 fill:#4CAF50,stroke:#fff,color:#fff
style OUT1 fill:#8BC34A,stroke:#fff,color:#fff
style OUT2 fill:#CDDC39,stroke:#000,color:#000
style OUT3 fill:#FFEB3B,stroke:#000,color:#000
graph TD
subgraph INPUTS["Priority Inputs"]
I1["🏃 Movement Alignment<br/>cos(angle between velocity<br/>vector and tile direction)<br/>w₁ = 0.30"]
I2["⚠️ Obstacle Proximity<br/>min depth value in tile<br/>region from depth camera<br/>w₂ = 0.25"]
I3["🔄 Temporal Change<br/>MSE between current tile<br/>and previous frame tile<br/>w₃ = 0.20"]
I4["🖱️ Operator Attention<br/>Gaussian around mouse<br/>cursor position<br/>w₄ = 0.10"]
I5["🦾 Arm Workspace<br/>Projection of arm<br/>end-effector to image<br/>w₅ = 0.15"]
end
I1 --> SCORE
I2 --> SCORE
I3 --> SCORE
I4 --> SCORE
I5 --> SCORE
SCORE["∑ P(i) = w₁×f₁ + w₂×f₂ + w₃×f₃ + w₄×f₄ + w₅×f₅"]
SCORE --> CLASS{Classification}
CLASS -->|"P(i) > 0.7"| CRIT["🔴 CRITICAL<br/>Layers 0-3<br/>Full Quality"]
CLASS -->|"0.4 < P(i) ≤ 0.7"| IMP["🟡 IMPORTANT<br/>Layers 0-2<br/>Standard Quality"]
CLASS -->|"P(i) ≤ 0.4"| LOW["🟢 LOW<br/>Layers 0-1<br/>Thumbnail Quality"]
style SCORE fill:#E91E63,stroke:#fff,color:#fff
style CRIT fill:#F44336,stroke:#fff,color:#fff
style IMP fill:#FF9800,stroke:#fff,color:#fff
style LOW fill:#4CAF50,stroke:#fff,color:#fff
block-beta
columns 8
block:header:8
title["Robot Moving FORWARD — Obstacle FRONT-RIGHT — Arm STOWED"]
end
T00[".2 🟢"] T01[".3 🟢"] T02[".4 🟡"] T03[".5 🟡"] T04[".6 🟡"] T05[".7 🔴"] T06[".5 🟡"] T07[".3 🟢"]
T08[".3 🟢"] T09[".4 🟡"] T10[".6 🟡"] T11[".7 🔴"] T12[".8 🔴"] T13[".9 🔴"] T14[".7 🔴"] T15[".4 🟡"]
T16[".3 🟢"] T17[".5 🟡"] T18[".7 🔴"] T19[".9 🔴"] T20[".9 🔴"] T21[".8 🔴"] T22[".6 🟡"] T23[".4 🟡"]
T24[".2 🟢"] T25[".4 🟡"] T26[".5 🟡"] T27[".7 🔴"] T28[".7 🔴"] T29[".6 🟡"] T30[".5 🟡"] T31[".3 🟢"]
T32[".2 🟢"] T33[".3 🟢"] T34[".4 🟡"] T35[".5 🟡"] T36[".5 🟡"] T37[".4 🟡"] T38[".3 🟢"] T39[".2 🟢"]
T40[".1 🟢"] T41[".2 🟢"] T42[".3 🟢"] T43[".3 🟢"] T44[".3 🟢"] T45[".3 🟢"] T46[".2 🟢"] T47[".1 🟢"]
style T05 fill:#F44336,color:#fff
style T11 fill:#F44336,color:#fff
style T12 fill:#F44336,color:#fff
style T13 fill:#F44336,color:#fff
style T14 fill:#F44336,color:#fff
style T18 fill:#F44336,color:#fff
style T19 fill:#E91E63,color:#fff
style T20 fill:#E91E63,color:#fff
style T21 fill:#F44336,color:#fff
style T27 fill:#F44336,color:#fff
style T28 fill:#F44336,color:#fff
graph TD
subgraph MEASURE["📊 Measurement"]
M1[bytes_received]
M2[round_trip_time]
M3[frames_dropped]
M4[jitter]
end
M1 --> EWMA
M2 --> EWMA
M3 --> EWMA
M4 --> EWMA
subgraph EWMA_BLOCK["📈 EWMA + Predictive Trend"]
EWMA["bw(t) = α × measured(t)<br/>+ (1-α) × bw(t-1)<br/><br/>α = 0.3"]
EWMA --> TREND["trend(t) = bw(t) - bw(t-1)"]
TREND --> PREDICT["predicted(t+1) = bw(t)<br/>+ β × trend(t)<br/><br/>β = 0.5"]
end
PREDICT --> DECISION
subgraph DECISION["🎛️ Quality Decision Matrix"]
Q1["> 500 KB/s<br/>All Layers • 15 FPS"]
Q2["> 200 KB/s<br/>Layers 0-2 • 10 FPS"]
Q3["> 50 KB/s<br/>Layers 0-1 • 8 FPS"]
Q4["< 50 KB/s<br/>Layer 0 only • 5 FPS"]
end
DECISION --> OUTPUT["Output:<br/>max_layer<br/>target_fps<br/>byte_budget"]
OUTPUT --> ALLOCATOR[Budget Allocator]
style EWMA fill:#4CAF50,stroke:#fff,color:#fff
style TREND fill:#8BC34A,stroke:#fff,color:#fff
style PREDICT fill:#CDDC39,stroke:#000,color:#000
style Q1 fill:#4CAF50,stroke:#fff,color:#fff
style Q2 fill:#8BC34A,stroke:#fff,color:#fff
style Q3 fill:#FF9800,stroke:#fff,color:#fff
style Q4 fill:#F44336,stroke:#fff,color:#fff
sequenceDiagram
participant C as Camera
participant DE as Delta Encoder
participant RF as Reference Frame
participant WS as WebSocket
Note over C,WS: KEYFRAME (t=0)
C->>DE: Raw Frame 0
DE->>RF: Store as reference
DE->>WS: All 48 tiles (~40 KB)
Note over C,WS: DELTA (t=1)
C->>DE: Raw Frame 1
DE->>RF: Compare with reference
Note right of DE: 12/48 tiles changed<br/>36 tiles unchanged (skip)
DE->>WS: 12 changed tiles (~8 KB)
Note over C,WS: DELTA (t=2)
C->>DE: Raw Frame 2
DE->>RF: Compare with reference
Note right of DE: 8/48 tiles changed
DE->>WS: 8 changed tiles (~5 KB)
Note over C,WS: DELTA (t=3-14)
C->>DE: Frames 3-14
DE->>RF: Compare with reference
DE->>WS: Only changed tiles
Note over C,WS: KEYFRAME (t=15) — Full refresh
C->>DE: Raw Frame 15
DE->>RF: Replace reference
DE->>WS: All 48 tiles (~40 KB)
graph LR
subgraph PACKET["Binary Packet Format (13 + N bytes)"]
direction TB
H1["msg_type<br/>1 byte<br/>0x01-0x06"]
H2["camera_id<br/>1 byte<br/>0-5"]
H3["frame_no<br/>4 bytes<br/>uint32"]
H4["tile_id<br/>1 byte<br/>0-47"]
H5["layer<br/>1 byte<br/>0-3"]
H6["flags<br/>1 byte<br/>bitfield"]
H7["payload_length<br/>4 bytes<br/>uint32"]
H8["payload<br/>N bytes<br/>JPEG data"]
end
subgraph MSG_TYPES["Message Types"]
MT1["0x01 TILE_DATA<br/>Server → Client"]
MT2["0x02 FRAME_START<br/>Server → Client"]
MT3["0x03 FRAME_END<br/>Server → Client"]
MT4["0x04 BW_REPORT<br/>Client → Server"]
MT5["0x05 MOUSE_POS<br/>Client → Server"]
MT6["0x06 QUALITY_CFG<br/>Server → Client"]
end
subgraph FLAGS["Flag Bits"]
F0["bit 0: is_keyframe"]
F1["bit 1: is_delta"]
F2["bit 2: priority_critical"]
F3["bit 3: priority_important"]
end
style PACKET fill:#1a1a2e,stroke:#e94560,color:#fff
style MSG_TYPES fill:#16213e,stroke:#0f3460,color:#fff
style FLAGS fill:#0f3460,stroke:#533483,color:#fff
gantt
title Frame Transmission Order — Priority Interleaved
dateFormat X
axisFormat %L ms
section Critical L0
CRIT tiles Layer 0 :crit, 0, 3
section Critical L1
CRIT tiles Layer 1 :active, 3, 8
section Important L0
IMP tiles Layer 0 :5, 10
section Critical L2
CRIT tiles Layer 2 :crit, 10, 20
section Important L1
IMP tiles Layer 1 :active, 15, 25
section Low L0
LOW tiles Layer 0 :20, 28
section Critical L3
CRIT tiles Layer 3 :crit, 25, 45
section Important L2
IMP tiles Layer 2 :active, 35, 55
section Low L1
LOW tiles Layer 1 :50, 65
graph TD
WS["🔌 WebSocket Connection<br/>ws://host:8000/parvs"] --> PARSER
subgraph PARSER["📦 Packet Parser"]
PARSE_HEADER["Parse 13-byte header"]
PARSE_HEADER --> ROUTE{msg_type?}
ROUTE -->|TILE_DATA| TILE_HANDLER["Route to FrameBuffer"]
ROUTE -->|FRAME_START| FS_HANDLER["Initialize frame assembly"]
ROUTE -->|FRAME_END| FE_HANDLER["Mark frame complete"]
ROUTE -->|QUALITY_CFG| QC_HANDLER["Update quality display"]
end
TILE_HANDLER --> FB
subgraph FB["📋 Frame Buffer (per camera)"]
direction TB
TILES_ARR["tile_buffer[48]"]
DECODE["createImageBitmap()<br/>Async, off main thread"]
DIRTY["dirty flag tracking"]
TILES_ARR --> DECODE
DECODE --> DIRTY
end
FB --> COMPOSITOR
subgraph COMPOSITOR["🎨 Canvas Compositor"]
CANVAS_EL["HTML5 Canvas<br/>1280 × 720"]
RAF["requestAnimationFrame loop"]
DRAW["For each dirty tile:<br/>ctx.drawImage(bitmap,<br/>x, y, tileW, tileH)"]
UPSCALE["Browser bilinear<br/>interpolation<br/>(automatic upscaling)"]
RAF --> DRAW
DRAW --> UPSCALE
UPSCALE --> CANVAS_EL
end
COMPOSITOR --> DISPLAY["🖥️ Operator Display"]
subgraph FEEDBACK["📡 Feedback Channel"]
BW_MEASURE["Measure RTT + throughput"]
BW_SEND["Send BW_REPORT<br/>every 500ms"]
MOUSE_TRACK["Track mouse position"]
MOUSE_SEND["Send MOUSE_POS<br/>throttled 100ms"]
end
FE_HANDLER --> BW_MEASURE
BW_MEASURE --> BW_SEND
BW_SEND --> WS
MOUSE_TRACK --> MOUSE_SEND
MOUSE_SEND --> WS
style WS fill:#F44336,stroke:#fff,color:#fff
style COMPOSITOR fill:#9C27B0,stroke:#fff,color:#fff
style FB fill:#2196F3,stroke:#fff,color:#fff
style FEEDBACK fill:#4CAF50,stroke:#fff,color:#fff
pie title Bandwidth Budget Distribution (Robot Moving Forward, Arm Unstowed)
"Front Composite (Active)" : 50
"Hand Color (Arm Active)" : 24
"Left Fisheye" : 8
"Right Fisheye" : 8
"Back Fisheye" : 5
"Depth Camera" : 5
sequenceDiagram
participant F as Front Camera
participant H as Hand Camera
participant L as Left Camera
participant R as Right Camera
participant B as Back Camera
participant WS as WebSocket
Note over F,WS: Single WebSocket — Interleaved by Priority
rect rgb(244, 67, 54)
F->>WS: Front CRITICAL L0 tiles
end
rect rgb(233, 30, 99)
H->>WS: Hand CRITICAL L0 tiles
end
rect rgb(244, 67, 54)
F->>WS: Front CRITICAL L1 tiles
end
rect rgb(76, 175, 80)
L->>WS: Left L0 tiles (all)
R->>WS: Right L0 tiles (all)
end
rect rgb(244, 67, 54)
F->>WS: Front CRITICAL L2 tiles
end
rect rgb(233, 30, 99)
H->>WS: Hand IMPORTANT L1 tiles
end
rect rgb(244, 67, 54)
F->>WS: Front L3 tiles
end
rect rgb(158, 158, 158)
B->>WS: Back L0 tiles (all)
end
Note over F,WS: Front always has data first → lowest perceived latency
gantt
title Frame Delivery Timeline Comparison
dateFormat X
axisFormat %L ms
section Current System (HTTP Polling)
Nothing visible :done, 0, 200
Full frame arrives :crit, 200, 220
Decode + render :active, 220, 230
section PARVS System
Layer 0 (blocky preview) :crit, 0, 19
Layer 1 (preview) :active, 19, 40
Layer 2 (standard) :40, 70
Layer 3 (full quality) :70, 120
stateDiagram-v2
[*] --> Excellent: BW > 500 KB/s
Excellent: 🟢 Excellent WiFi
Excellent: All Layers • 15 FPS
Excellent: All cameras updated
Good: 🟡 Good WiFi
Good: Layers 0-2 • 10 FPS
Good: Primary + secondary cameras
Poor: 🟠 Poor WiFi
Poor: Layers 0-1 • 8 FPS
Poor: Primary camera only
Critical: 🔴 Critical WiFi
Critical: Layer 0 only • 5 FPS
Critical: Primary camera thumbnail
Excellent --> Good: BW drops < 500 KB/s
Good --> Excellent: BW rises > 500 KB/s
Good --> Poor: BW drops < 200 KB/s
Poor --> Good: BW rises > 200 KB/s
Poor --> Critical: BW drops < 50 KB/s
Critical --> Poor: BW rises > 50 KB/s
Note right of Excellent: Predictive trend detection\ntriggers transitions BEFORE\nframes start dropping
graph TD
subgraph SPOT_SDK["Spot SDK Data Sources"]
VEL["get_robot_state()<br/>→ velocity (vx, vy, vz)"]
DEPTH["get_depth_image()<br/>→ depth map"]
ARM["get_arm_state()<br/>→ joint positions"]
POSE["get_robot_state()<br/>→ body pose (roll, pitch, yaw)"]
end
subgraph PARVS_PRIORITY["PARVS Priority Computation"]
MA["Movement Alignment<br/><br/>For each tile center (tx, ty):<br/>angle = atan2(ty - cy, tx - cx)<br/>alignment = cos(angle - vel_angle)<br/>score = max(0, alignment)"]
OP["Obstacle Proximity<br/><br/>For each tile region:<br/>min_depth = min(depth_map[tile])<br/>score = 1.0 - clamp(min_depth / 3.0)"]
TC["Temporal Change<br/><br/>For each tile:<br/>diff = MSE(current_tile, prev_tile)<br/>score = clamp(diff / threshold)"]
OA["Operator Attention<br/><br/>For each tile center:<br/>dist = distance(tile_center, mouse_pos)<br/>score = gaussian(dist, σ=200px)"]
AW["Arm Workspace<br/><br/>Project arm end-effector to image:<br/>dist = distance(tile_center, ee_proj)<br/>score = gaussian(dist, σ=150px)"]
end
VEL --> MA
DEPTH --> OP
ARM --> AW
POSE --> MA
subgraph CLIENT_INPUT["Client Feedback"]
MOUSE["Mouse cursor<br/>position (x, y)"]
end
MOUSE --> OA
MA --> COMBINE["Weighted Sum<br/>P(i) = 0.30×MA + 0.25×OP<br/>+ 0.20×TC + 0.10×OA + 0.15×AW"]
OP --> COMBINE
TC --> COMBINE
OA --> COMBINE
AW --> COMBINE
COMBINE --> HEATMAP["48-tile Priority Heatmap"]
style SPOT_SDK fill:#1a1a2e,stroke:#e94560,color:#fff
style PARVS_PRIORITY fill:#16213e,stroke:#0f3460,color:#fff
style CLIENT_INPUT fill:#0f3460,stroke:#533483,color:#fff
style COMBINE fill:#E91E63,stroke:#fff,color:#fff
style HEATMAP fill:#F44336,stroke:#fff,color:#fff
graph LR
subgraph BACKEND["backend/"]
subgraph PARVS_BE["parvs/"]
SE[stream_engine.py<br/>Main orchestrator]
PG[pyramid_generator.py<br/>Frame decomposition]
TS[tile_segmenter.py<br/>8×6 grid operations]
PE[priority_engine.py<br/>S-ROI scoring]
BC[bandwidth_controller.py<br/>EWMA + prediction]
DE_FILE[delta_encoder.py<br/>Keyframe + delta]
PP[packet_protocol.py<br/>Binary format]
CM[camera_multiplexer.py<br/>Multi-cam interleave]
MET[metrics.py<br/>Performance logging]
end
APP[app.py<br/>+WebSocket endpoint]
RC[robot_controller.py<br/>Spot SDK bridge]
end
subgraph FRONTEND_DIR["frontend/src/"]
subgraph PARVS_FE["parvs/"]
PC[PARVSClient.js<br/>WebSocket client]
FB_FILE[FrameBuffer.js<br/>Tile storage]
CC[CanvasCompositor.js<br/>Canvas rendering]
MO[MetricsOverlay.js<br/>Debug display]
PCF[PARVSCameraFeed.js<br/>Drop-in replacement]
end
end
subgraph DOCS_DIR["docs/"]
ARCH[PARVS_ARCHITECTURE.md]
PROTO[PARVS_PROTOCOL.md]
BENCH[PARVS_BENCHMARKS.md]
end
APP --> SE
RC --> SE
SE --> PG --> TS --> PE --> BC --> DE_FILE --> PP --> CM
PC --> FB_FILE --> CC --> PCF
MO --> PCF
style PARVS_BE fill:#E91E63,stroke:#fff,color:#fff
style PARVS_FE fill:#E91E63,stroke:#fff,color:#fff
style SE fill:#F44336,stroke:#fff,color:#fff
style PC fill:#F44336,stroke:#fff,color:#fff
graph TD
subgraph EXISTING["Existing Components (Unchanged)"]
CP[ControlPanel.js<br/>WASD Movement]
AC[ArmControl.js<br/>Arm Position]
GC[GripperControl.js<br/>Open/Close]
TEL[Telemetry.js<br/>Robot State]
SC[SpeedControl.js<br/>Speed Profiles]
end
subgraph REPLACED["Replaced Components"]
OLD_CF["❌ CameraFeed.js<br/>(HTTP polling, 5 FPS)"]
NEW_CF["✅ PARVSCameraFeed.js<br/>(WebSocket, 8-15 FPS)"]
OLD_CF -.->|replaced by| NEW_CF
end
subgraph NEW["New PARVS Components"]
PARVS_C[PARVSClient.js]
FRAME_B[FrameBuffer.js]
CANVAS_C[CanvasCompositor.js]
METRICS_O[MetricsOverlay.js]
end
subgraph BACKEND_INT["Backend Integration"]
APP_PY[app.py]
WS_EP["NEW: /parvs WebSocket"]
PARVS_ENG[parvs/stream_engine.py]
APP_PY --> WS_EP
WS_EP --> PARVS_ENG
end
CP -->|movement commands| PARVS_ENG
AC -->|arm state| PARVS_ENG
TEL -->|robot velocity| PARVS_ENG
NEW_CF --> PARVS_C
PARVS_C --> FRAME_B
FRAME_B --> CANVAS_C
subgraph FALLBACK["Fallback Mode"]
FB_CHECK{WebSocket<br/>connected?}
FB_CHECK -->|Yes| NEW_CF
FB_CHECK -->|No| OLD_POLL["HTTP Polling<br/>(automatic fallback)"]
end
style OLD_CF fill:#F44336,stroke:#fff,color:#fff
style NEW_CF fill:#4CAF50,stroke:#fff,color:#fff
style REPLACED fill:#fff3e0,stroke:#FF9800
style NEW fill:#e8f5e9,stroke:#4CAF50
style FALLBACK fill:#fff9c4,stroke:#FFC107
graph TD
subgraph CONDITIONS["Network Conditions (Linux tc)"]
C1["🟢 Excellent<br/>10 Mbit, 5ms delay"]
C2["🟡 Good<br/>2 Mbit, 15ms delay"]
C3["🟠 Poor<br/>500 Kbit, 50ms delay"]
C4["🔴 Degrading<br/>2M → 200K over 30s"]
C5["⚡ Unstable<br/>1M, 20±40ms jitter"]
end
subgraph BASELINES["Comparison Systems"]
B_A["A: HTTP Polling<br/>(current system)"]
B_B["B: Adaptive JPEG<br/>(quality-only)"]
B_C["C: WebRTC<br/>(industry standard)"]
B_D["D: PARVS-NoROI<br/>(ablation)"]
B_E["E: PARVS-NoPrediction<br/>(ablation)"]
B_F["F: PARVS-Full<br/>(proposed)"]
end
subgraph METRICS["Evaluation Metrics"]
M1["⏱️ Time-to-First-Pixel<br/>(TTFP)"]
M2["⏱️ Time-to-Full-Quality<br/>(TTFQ)"]
M3["📊 Bandwidth<br/>(bytes/sec)"]
M4["🖼️ SSIM / PSNR<br/>(image quality)"]
M5["📉 Frame Drop Rate"]
M6["👤 Task Completion Time<br/>(user study)"]
M7["📋 NASA-TLX Score<br/>(user study)"]
end
CONDITIONS --> |each condition ×| BASELINES
BASELINES --> |measured by| METRICS
METRICS --> PAPER["📄 IEEE RA-L Paper"]
style B_F fill:#E91E63,stroke:#fff,color:#fff
style PAPER fill:#F44336,stroke:#fff,color:#fff
sequenceDiagram
actor OP as Operator
participant UI as React UI
participant PC as PARVS Client
participant WS as WebSocket
participant SE as Stream Engine
participant PG as Pyramid Gen
participant PE as Priority Engine
participant BC as BW Controller
participant SDK as Spot SDK
participant SPOT as Spot Robot
Note over OP,SPOT: Initialization
OP->>UI: Open Command Center
UI->>PC: Initialize PARVS
PC->>WS: Connect ws://host:8000/parvs
WS->>SE: New client connected
SE->>SDK: Start camera capture loop
SDK->>SPOT: Request camera streams
Note over OP,SPOT: Steady State — Frame Pipeline
loop Every frame (66ms at 15 FPS)
SPOT->>SDK: Raw camera frame
SDK->>SE: Frame + robot state
SE->>PG: Generate 4-layer pyramid
PG->>SE: Layers 0-3 (JPEG encoded)
SE->>PE: Score 48 tile priorities
Note right of PE: Uses velocity, depth,<br/>arm state, mouse pos,<br/>frame diff
PE->>SE: Priority heatmap
SE->>BC: Request budget allocation
Note right of BC: EWMA + trend prediction<br/>from client BW reports
BC->>SE: max_layer, FPS, budget
SE->>SE: Delta encode (skip unchanged tiles)
SE->>WS: Binary tile packets (priority order)
WS->>PC: Receive tile packets
PC->>PC: createImageBitmap (async)
PC->>UI: Update canvas (progressive)
Note left of UI: t=19ms: blocky preview<br/>t=40ms: preview sharp<br/>t=80ms: standard<br/>t=120ms: full quality
end
Note over OP,SPOT: Feedback Loop
loop Every 500ms
PC->>WS: BW_REPORT (RTT, throughput, drops)
WS->>BC: Update bandwidth estimate
end
loop On mouse move (throttled 100ms)
OP->>UI: Move mouse
UI->>PC: Mouse position
PC->>WS: MOUSE_POS packet
WS->>PE: Update operator attention map
end
Note over OP,SPOT: Operator Moves Robot Forward
OP->>UI: Press W (forward)
UI->>SDK: Move command
SDK->>SPOT: Walk forward
SPOT->>SDK: Updated velocity vector
SDK->>PE: New movement direction
Note right of PE: Front-center tiles<br/>become CRITICAL<br/>Back tiles become LOW