MQTTOTA is an SDK that revolutionizes firmware management for ESP32-based IoT devices. By leveraging the power of MQTT/MQTTS protocols, it provides a seamless, secure, and scalable solution for Over-The-Air updates in distributed IoT ecosystems. Whether you're managing a handful of devices or thousands across global deployments, MQTTOTA ensures reliable firmware delivery with enterprise-level security and robust error handling.
- What's New in v1.2.0
- Overview
- Key Features
- Installation
- Dependencies
- Basic Configuration
- Security — HMAC-SHA256 (v1.2.0)
- Standard MQTT Configuration
- MQTTS (Secure) Configuration
- Message Formats
- Advanced Configuration
- Diagnostics and Troubleshooting
- Complete API
- Performance Considerations
- Best Practices
- Complete Workflows
- Backend Implementation
- Broker Compatibility
- License
- Contact
This release resolves 8 issues identified in a technical audit of v1.1.0. All changes are backward-compatible unless noted.
| Issue | Description | Solution |
|---|---|---|
| #1 | _calculateSHA256() always returned "" — SHA-256 was declared but never computed |
Implemented with mbedtls_sha256 — real incremental digest |
| #2 | verifyFirmwareSignature() always returned true — HMAC was never verified |
Real HMAC-SHA256 via mbedtls_md_hmac. New API: setSecurityKey() / requireSignature() |
| Issue | Description | Solution |
|---|---|---|
| #3 | DynamicJsonDocument(32768) allocated 32 KB on the heap on every message |
All JSON documents migrated to StaticJsonDocument on the stack |
| #6 | getFreeOTASpace() was commented out with a compilation error note |
Implemented correctly using esp_ota_get_next_update_partition()->size |
| Issue | Description | Solution |
|---|---|---|
| #4 | delay(2000) / delay(3000) before ESP.restart() blocked the FreeRTOS scheduler |
Replaced with a non-blocking timer polled in handle(). handle() must be called every loop() |
| #5 | Two parallel state variables: _otaInProgress and _otaContext.inProgress could go out of sync |
Unified into _otaContext.inProgress. isUpdateInProgress() now reads a single source |
| #7 | _setState() was only called in a few places — getCurrentState() returned stale values |
_setState() is now called at every transition: IDLE → RECEIVING → DECODING → VALIDATING → WRITING → COMPLETING → SUCCESS/ERROR |
| #8 | esp_ota_mark_app_valid_cancel_rollback() was never called automatically |
Called in begin() whenever the running partition is an OTA partition |
In v1.0.1 the device restarted automatically via a blocking
delay()+ESP.restart()after a successful OTA.In v1.2.0 the restart is non-blocking and is driven by a timer polled in
handle(). If you do not callhandle()in yourloop(), the device will not restart automatically after OTA.Action required: Ensure
ota.handle()is called everyloop()iteration.
void loop() {
mqttClient.loop();
ota.handle(); // Required: drives restart timer + timeout watchdog
}MQTTOTA is a robust SDK designed for ESP32 IoT deployments. It supports both single-message and chunked firmware transfers, integrates directly with the ESP-IDF OTA partition API (esp_ota_ops.h), and since v1.2.0 provides real cryptographic verification with SHA-256 and HMAC-SHA256 via mbedtls (bundled with ESP32 Arduino).
- Full OTA — Single MQTT message with complete firmware
- Chunked OTA — Fragmented transfer for large firmware files with strict sequence validation
- Native ESP-IDF OTA — Uses
esp_ota_begin/write/enddirectly; no dependency onUpdate.hlayer
- MQTTS — Encrypted transport via TLS (delegated to the external MQTT client)
- Real SHA-256 — Integrity digest computed with
mbedtls_sha256on every chunk - Real HMAC-SHA256 — Firmware origin authentication via
setSecurityKey()+requireSignature() - Image header verification — Validates ESP32 magic number and segment count before writing
- Configurable timeout — 7-minute watchdog cancels hung updates
- 9-state machine —
IDLE → RECEIVING → DECODING → VALIDATING → WRITING → COMPLETING → SUCCESS/ERROR/ABORTED - Event callbacks —
onProgress,onError,onSuccess,onStateChange - OTA statistics — bytes, chunk count, error count, average speed (KB/s)
- Non-blocking restart (v1.2.0) — restart scheduled after OTA without blocking
loop()
- Download
MQTTOTA.handMQTTOTA.cpp - Create
MQTTOTA/folder inArduino/libraries/ - Copy files there and restart Arduino IDE
lib_deps =
https://github.com/JorgeGBeltre/MQTTOTA.git#include <Arduino.h>
#include <WiFi.h>
#include <WiFiClientSecure.h>
#include <ArduinoJson.h> // v6.19+ (ArduinoJson v7 not yet tested)
#include <Update.h>
#include <mbedtls/sha256.h> // v1.2.0 — bundled with ESP32 Arduino, no extra install
#include <mbedtls/md.h> // v1.2.0 — bundled with ESP32 Arduino, no extra install
#include "esp_ota_ops.h"
#include "esp_app_format.h"
#include "esp_partition.h"No additional library installs are needed for mbedtls — it ships with the ESP32 Arduino core.
#define MQTT_OTA_JSON_SIZE 8192 // Stack size for JSON parser (v1.2.0: was 32768 heap)
#define MQTT_OTA_BUFFSIZE 1024 // Write buffer per chunk
#define MQTT_OTA_TIMEOUT_MS 420000 // Total OTA timeout (7 min)
#define MQTT_OTA_MAX_CHUNK_SIZE 65536 // Max decoded bytes per MQTT chunk
#define MQTT_OTA_MIN_MEMORY 40000 // Min free heap before starting OTA
#define MQTT_OTA_MAX_RETRIES 3 // Chunk retry limit
#define MQTT_OTA_HMAC_KEY_SIZE 64 // Max HMAC key length (bytes)
#define MQTT_OTA_RESTART_DELAY_MS 3000 // Non-blocking restart delay (ms)Stack note:
StaticJsonDocument<MQTT_OTA_JSON_SIZE>lives on the stack ofloop(). The default 8 KB fits comfortably in the ESP32's 8 KBloop()stack. If you use FreeRTOS tasks with custom stacks, ensure the task stack is ≥ 10 KB.
#include "MQTTOTA.h"
MQTTOTA ota;
void setup() {
Serial.begin(115200);
ota.begin("MyDevice", "1.0.0");
// begin() also calls esp_ota_mark_app_valid_cancel_rollback()
// automatically when running from an OTA partition (v1.2.0)
}
void loop() {
ota.handle(); // Required every loop — drives restart timer and timeout
}v1.2.0 adds real firmware authentication. Without a configured key the behavior is identical to v1.0.1 (backward compatible).
void setup() {
ota.begin("MyDevice", "1.0.0");
// Set shared secret (must match the key used by your backend to sign firmware)
ota.setSecurityKey("my-super-secret-key-at-least-32-chars");
// Optional: reject any OTA message that does not carry a valid HMAC
ota.requireSignature(true);
}Your backend must compute HMAC-SHA256(decoded_firmware_bytes, key) and send it as a hex string in the checksum field (or in a User Property if using MQTTOTAv5).
# Python example
import hmac, hashlib
key = b"my-super-secret-key-at-least-32-chars"
data = open("firmware.bin", "rb").read()
sig = hmac.new(key, data, hashlib.sha256).hexdigest()
print(sig) # 64-char hex string → include in OTA message// Legacy overload — backward compat, does not compute HMAC without raw data
bool verifyFirmwareSignature(const String& signature);#include "MQTTOTA.h"
#include <PubSubClient.h>
#include <WiFi.h>
WiFiClient wifiClient;
PubSubClient mqttClient(wifiClient);
MQTTOTA ota;
const char* ssid = "your_SSID";
const char* password = "your_PASSWORD";
const char* mqttServer= "broker.hivemq.com";
const int mqttPort = 1883;
const char* otaTopic = "devices/my_device/ota";
void mqttCallback(char* topic, byte* payload, unsigned int length) {
String message;
for (unsigned int i = 0; i < length; i++) message += (char)payload[i];
ota.processMessage(String(topic), message);
}
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) { delay(500); }
mqttClient.setServer(mqttServer, mqttPort);
mqttClient.setCallback(mqttCallback);
while (!mqttClient.connect("my_device")) { delay(5000); }
mqttClient.subscribe(otaTopic);
ota.begin("MyDevice", "1.0.0");
ota.setMQTTConfig(
[](const char* topic, const String& msg) { mqttClient.publish(topic, msg.c_str()); },
[]() { return mqttClient.connected(); },
otaTopic
);
ota.onProgress([](int pct, const String& ver) {
Serial.printf("OTA %d%% — v%s\n", pct, ver.c_str());
});
ota.onError([](const String& err, const String& ver) {
Serial.printf("OTA error: %s (v%s)\n", err.c_str(), ver.c_str());
});
ota.onSuccess([](const String& ver) {
Serial.printf("OTA complete: v%s\n", ver.c_str());
});
}
void loop() {
if (!mqttClient.connected()) { /* reconnect */ }
mqttClient.loop();
ota.handle(); // Required: non-blocking restart + timeout watchdog
}#include "MQTTOTA.h"
#include <WiFiClientSecure.h>
#include <PubSubClient.h>
WiFiClientSecure wifiClient;
PubSubClient mqttClient(wifiClient);
MQTTOTA ota;
const char* rootCA = \
"-----BEGIN CERTIFICATE-----\n" \
"... your CA certificate ...\n" \
"-----END CERTIFICATE-----\n";
void setup() {
Serial.begin(115200);
// ... WiFi setup ...
wifiClient.setCACert(rootCA);
mqttClient.setServer("your-secure-broker.com", 8883);
mqttClient.setCallback([](char* topic, byte* payload, unsigned int len) {
String msg;
for (unsigned int i = 0; i < len; i++) msg += (char)payload[i];
ota.processMessage(String(topic), msg);
});
// ... connect and subscribe ...
ota.begin("MySecureDevice", "1.0.0");
// Recommended in production:
ota.setSecurityKey("production-hmac-secret-key-here");
ota.setSecurityMode(SECURITY_HMAC_SHA256);
ota.setMQTTConfig(
[](const char* t, const String& m) { mqttClient.publish(t, m.c_str()); },
[]() { return mqttClient.connected(); },
"devices/secure/ota"
);
}
void loop() {
mqttClient.loop();
ota.handle();
}{
"EventType": "UpdateFirmwareDevice",
"Details": {
"FirmwareVersion": "1.1.0",
"Base64": "<complete_firmware_base64>",
"IsError": false,
"ErrorMessage": null
}
}{
"EventType": "UpdateFirmwareDevice",
"Details": {
"FirmwareVersion": "1.2.0",
"Base64Part": "<chunk_base64>",
"PartIndex": 1,
"TotalParts": 10,
"IsError": false,
"ErrorMessage": null,
"sha256": "<expected_sha256_hex>",
"hmac_sig": "<expected_hmac_hex>",
"ecdsa_sig": "<expected_ecdsa_base64>"
}
}// ota/progress (throttled to multiples of 10%)
{ "device": "ABC123", "version": "1.1.0", "progress": 50, "timestamp": 123456 }
// ota/error
{ "device": "ABC123", "version": "1.1.0", "error": "SHA-256 mismatch", "timestamp": 123456 }
// ota/success
{ "device": "ABC123", "version": "1.1.0", "success": true, "timestamp": 123456 }
// ota/state (every state transition)
{ "device": "ABC123", "state": 5, "state_name": "WRITING", "timestamp": 123456 }void setup() {
ota.begin("MyDevice", "2.0.0");
// Security (v1.2.0)
ota.setSecurityKey("your-shared-secret");
ota.requireSignature(true);
// Transfer mode
ota.enableChunkedOTA(true); // Default: true
ota.setChunkSize(2048);
// Resilience
ota.setMaxRetries(5);
ota.setAutoReset(true); // Schedule non-blocking restart after OTA
ota.enableVersionCheck(true); // Reject if version matches current
ota.enableRollbackProtection(true);
// MQTT
ota.setMQTTConfig(publishFn, connectedFn, "devices/my/ota");
// Callbacks
ota.onProgress([](int pct, const String& ver) { /* ... */ });
ota.onError([](const String& err, const String& ver) { /* ... */ });
ota.onSuccess([](const String& ver) { /* ... */ });
ota.onStateChange([](uint8_t state) {
Serial.printf("OTA state changed: %d\n", state);
});
}void loop() {
ota.handle();
static unsigned long lastCheck = 0;
if (millis() - lastCheck > 30000) {
MQTTOTA::logMemoryStatus(); // free/min/maxAlloc heap
Serial.printf("OTA space: %zu B\n", ota.getFreeOTASpace()); // v1.2.0
lastCheck = millis();
}
if (!ota.isUpdateInProgress()) {
handleSensors();
publishTelemetry();
}
}ota.printDiagnostics();
// Output:
// === MQTTOTA Diagnostics ===
// DeviceID : ESP32_OTA_TEST
// Device : ESP32_OTA_TEST
// Version : 1.0.0
// State : IDLE
// Progress : 0%
// Received : 0 B
// Sig req : yes
// Key set : yes
// Heap — free=180000 minFree=120000 maxAlloc=90000
// Running : ota_0 @ 0x00010000
// Stats — chunks=0 errors=0 speed=0.0 KB/s
// ===========================ota.onError([](const String& error, const String& version) {
if (error.indexOf("memory") != -1) {
Serial.println("Low memory — free resources and retry");
MQTTOTA::logMemoryStatus();
} else if (error.indexOf("timeout") != -1) {
Serial.println("OTA timed out — check MQTT connection");
} else if (error.indexOf("SHA-256") != -1) {
Serial.println("Integrity check failed — firmware rejected");
} else if (error.indexOf("HMAC") != -1) {
Serial.println("Authentication failed — check HMAC key");
} else if (error.indexOf("sequence") != -1) {
Serial.println("Chunk out of order — restart the transfer");
}
});IDLE
└─► RECEIVING (begin() partition found, esp_ota_begin OK)
└─► DECODING (base64 decoded successfully)
└─► VALIDATING (image header + SHA-256/HMAC check)
└─► WRITING (esp_ota_write per chunk)
└─► COMPLETING (all chunks received)
├─► SUCCESS → pending non-blocking restart
└─► ERROR → cleanup, back to IDLE
void begin(const String& deviceName, const String& firmwareVersion);
// Also calls esp_ota_mark_app_valid_cancel_rollback() on OTA partitions (v1.2.0)
void handle();
// REQUIRED every loop() — drives restart timer and timeout watchdog (v1.2.0)
void setMQTTConfig(publishFn, isConnectedFn, otaTopic = "ota");
void setPartitionName(const String& partitionName = "");void setSecurityKey(const char* key); // Set HMAC-SHA256 key (max 64 bytes)
void requireSignature(bool required = true); // Reject unsigned payloads if true
// Verify HMAC on raw decoded bytes (recommended)
bool verifyFirmwareSignature(const uint8_t* data, size_t len, const String& expectedHex);
// Legacy overload — backward compat, does not compute HMAC without raw data
bool verifyFirmwareSignature(const String& signature);void enableChunkedOTA(bool enable = true);
void setChunkSize(size_t chunkSize);
void setAutoReset(bool autoReset = true);
void setMaxRetries(int maxRetries);
void enableVersionCheck(bool enable = true);
void enableRollbackProtection(bool enable = true);void processMessage(const String& topic, const String& message);
bool performUpdate(const String& base64Data, const String& firmwareVersion);
void abortUpdate();bool isUpdateInProgress() const; // v1.2.0: single source of truth
bool isValidating() const;
bool isWriting() const;
OTAState getCurrentState(); // Full 9-state enum
int getProgress();
String getCurrentVersion();
String getDeviceID();
size_t getFreeOTASpace(); // v1.2.0: was commented out
OTAStatistics getStatistics(); // bytes, chunks, errors, avgSpeed
String getBootPartitionInfo();
void printDiagnostics();static bool checkMemory(size_t requiredBytes);
static size_t getFreeHeap();
static void logMemoryStatus();void onProgress(MQTTOTACallback callback); // (int pct, const String& ver)
void onError(MQTTOTAErrorCallback callback); // (const String& err, const String& ver)
void onSuccess(MQTTOTASuccessCallback callback); // (const String& ver)
void onStateChange(MQTTOTAStateCallback callback); // (uint8_t state)enum OTAState {
OTA_STATE_IDLE = 0,
OTA_STATE_RECEIVING = 1,
OTA_STATE_DECODING = 2,
OTA_STATE_VALIDATING = 3,
OTA_STATE_WRITING = 4,
OTA_STATE_COMPLETING = 5,
OTA_STATE_SUCCESS = 6,
OTA_STATE_ERROR = 7,
OTA_STATE_ABORTED = 8
};| Mode | Heap impact | Recommended for |
|---|---|---|
| Full (single message) | ~8 KB stack (StaticJson) + decoded size in RAM | Firmware < 100 KB |
| Chunked | ~4 KB stack (StaticJson) + 1 chunk in RAM | Any firmware size |
// Larger chunks = better throughput (up to MQTT_OTA_MAX_CHUNK_SIZE = 64 KB)
ota.setChunkSize(16384); // 16 KB chunks
// Reduce retries to fail faster on unstable links
ota.setMaxRetries(2);
// Track average speed
OTAStatistics s = ota.getStatistics();
Serial.printf("Speed: %.1f KB/s\n", s.avgSpeedBps / 1024.0f);ota.onError([](const String& error, const String& version) {
if (error.indexOf("timeout") != -1 || error.indexOf("sequence") != -1) {
Serial.println("Transfer interrupted — will retry on next OTA message");
// No manual restart needed; state is already IDLE after cleanup
}
});// Transport encryption
wifiClient.setCACert(rootCA);
// Firmware authentication (v1.2.0)
ota.setSecurityKey("production-secret-min-32-chars");
ota.requireSignature(true);bool canStartOTA() {
return (ESP.getFreeHeap() > 50000) &&
(WiFi.status() == WL_CONNECTED) &&
(!ota.isUpdateInProgress());
}void loop() {
mqttClient.loop();
ota.handle(); // Never skip this — non-blocking restart depends on it
}// In setup(), after begin(), log the full state
ota.printDiagnostics();
// Confirms partition, rollback status, HMAC key, and heapBackend ESP32 Device
│ │
│─── chunk 1/10 (partIndex=1) ────►│ esp_ota_begin()
│◄── ota/progress: 10% ────────────│ RECEIVING → DECODING → VALIDATING → WRITING
│ │
│─── chunk 2/10 ──────────────────►│ esp_ota_write()
│◄── ota/progress: 20% ────────────│ WRITING
│ ... (chunks 3–9) ... │
│─── chunk 10/10 ─────────────────►│ esp_ota_end()
│◄── ota/progress: 95% ────────────│ COMPLETING → VALIDATING (SHA-256) → SUCCESS
│◄── ota/success ──────────────────│ schedule restart (3 s timer)
│ │ ... handle() fires restart ...
│ │ ESP.restart()
Backend ESP32 Device
│ │
│─── chunk 3/10 ──────────────────►│ Out-of-sequence (expected 4)
│◄── ota/error: "Chunk out of │ esp_ota_abort()
│ sequence" ──────────────────│ State → IDLE, ready for retry
To use MQTTOTA in your project, you'll need an MQTT server to manage OTA updates. You can implement your own backend using our reference repository:
MQTT Broker for OTA Updates
- Repository: github.com/Ruben890/Mqtt-Broker
- Description: Complete backend for managing OTA updates via MQTT/MQTTS
- Features:
- Configurable MQTT server
- IoT device management
- Firmware update delivery
- OTA progress tracking
- Error handling and retry mechanisms
Steps to use the backend:
- Clone the backend repository
- Configure the MQTT broker according to your needs
- Implement the update delivery logic
- Connect your ESP32 devices to the broker
- Manage OTA updates from a centralized interface
Example workflow:
// From your backend
1. Prepare firmware in base64 format
2. Publish MQTT message to target device
3. Monitor progress via callbacks
4. Confirm successful completion
5. Log results in databaseThe backend provides a scalable architecture for managing multiple devices simultaneously, with support for mass updates and firmware version management.
Tested with:
- Mosquitto 2.0+
- EMQX 5.0+
- HiveMQ Cloud
- AWS IoT Core
- ESP-IDF MQTT client (used in
BasicOTAexample)
Licensed under the MIT License. See LICENSE for details.
Author: Jorge Gaspar Beltre Rivera
Project: MQTTOTA - For OTA Updates via MQTT/MQTTS
This project is developed independently. Even a small contribution helps me dedicate more time to development, testing, and releasing new features.
