CAN Data Processing & ESP32 Playback Pipeline

This repository provides a complete workflow for CAN bus log processing, decoding, uploading to a Hasura GraphQL backend, and playback on an ESP32 over OTA. The key orchestrator is pipeline.sh, which automates most of the workflow, leaving the ESP32 to handle the playback and events asynchronously.

---

Features

• Automatic log decoding: Converts CAN logs into structured JSON using a DBC file.
• Hasura GraphQL integration: Uploads decoded CAN messages to a Hasura backend.
• ESP32 OTA playback: Replays logs on ESP32 with built-in CAN transmission and asynchronous event handling.
• Blink feedback: ESP32 LED shows alive status while processing CAN events.
• Minimal manual steps: Once logs are processed, the ESP32 handles events automatically.

---

Full Architecture Overview

flowchart TD
    A["Raw CAN Logs"] -->|pipeline.sh| B["Decode CAN Logs (Python)"]
    B --> C["Decoded JSON"]
    C --> D["Upload to Hasura GraphQL"]
    D --> E["ESP32 OTA Playback"]
    E --> F["CAN Bus"]
    E --> G["Status LED"]
    F --> H["CAN Devices / Network"]

Loading

The pipeline orchestrates decoding, JSON creation, and Hasura upload. Once the ESP32 firmware is running, it handles playback and events autonomously.

---

ESP32 Playback Flow

sequenceDiagram
    participant ESP as ESP32
    participant CAN as CAN Bus
    participant LED as Status LED

    ESP->>ESP: Load C struct logs
    loop For each CAN event
        ESP->>CAN: Send CAN frame
        ESP->>LED: Blink
    end
    ESP->>ESP: Wait for next cycle

Loading

---

Getting Started

Prerequisites

• Python 3.8+ with cantools installed:

pip install cantools requests

• Hasura GraphQL endpoint running.
• ESP32 development environment (Arduino IDE or PlatformIO).

Running the Pipeline

1. Place your DBC file and CAN logs in their respective directories.
2. Run the orchestrator:

./pipeline.sh <can-log-file>

pipeline.sh decodes logs, creates JSON in decoded_output_jsons/, and uploads to Hasura automatically.

3. Flash the ESP32 or use the OTA server:

   • Access ESP32 via its web interface (http://<esp32-ip>).
   • Upload firmware if needed.
   • Logs will automatically play back and handle events asynchronously.

---

ESP32 Features

• Non-blocking LED indicator (init-blink.h)
• Asynchronous CAN log playback (init-can.h)
• Web-based OTA server (init-server.h)
• Event-driven design: once firmware is running, no further manual intervention is required for log replay.

---

Notes

• JSON outputs are automatically stored in decoded_output_jsons/ (gitignored).
• pipeline.sh is the main orchestrator; all other steps can run independently once JSON files exist.
• ESP32 uses js-async to schedule CAN events without blocking other tasks.