feat: spool records to a file store with a CRC-16 at rest

A SolidSyslogBlockStore over a FileBlockDevice over the library's FatFs port,
replacing the Null store. The service task drains the ring into the store and
sends from there, so a failed send costs a retry rather than the record: the
audit trail survives an outage instead of ending at it.

  Flash      +11,584 B  (+3,956 on the previous stage)
  RAM         +9,128 B    (+1,436)
  Log stack     +672 B  (unchanged)
  Service       +960 B       (+64)

The log stack does not move: storing happens on the service task, and a task that
calls Log still knows nothing about what happens after it returns. The service
task's own high-water rises 64 bytes, which its existing allocation absorbs.

The static RAM is pool allocation, not buffers. The block size is file capacity —
nothing holds a block in memory, so the store costs its handles rather than its
capacity.

CRC-16 detects corruption, not tampering: it catches a truncated write or bit-rot,
and anyone who can edit a stored record can recompute it. It establishes that a
record came back the way it went in, which is the prerequisite for spooling at
all. Making stored records tamper-evident, and then unreadable, are later stages.

Three decisions come with the store — capacity, what happens when it fills, and
whether to be warned before that point. This device stores four blocks, one file
per block, syslog00.log upward on the volume it already mounts, and discards the
oldest when full.

SolidSyslog::FatFs is a header-configured upstream, so it is named in
SOLIDSYSLOG_PLATFORMS and linked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Author: DavidCozens

CMakeLists.txt

@@ -39,7 +39,7 @@ set(LWIP_CONTRIB_FREERTOS_DIR "${LWIP_DIR}/contrib/ports/freertos")
 # link target — only the header-configured packs below do.
 # https://docs.cososo.co.uk/solid-syslog/getting-started/#path-a--cmake-consumer
 # Pinned to a commit until there is a release tag to pin to.
-set(SOLIDSYSLOG_PLATFORMS "LwipRaw;StdAtomic;FreeRtos" CACHE STRING "" FORCE)
+set(SOLIDSYSLOG_PLATFORMS "LwipRaw;StdAtomic;FreeRtos;FatFs" CACHE STRING "" FORCE)
 
 include(FetchContent)
 FetchContent_Declare(SolidSyslog
@@ -167,7 +167,7 @@ target_include_directories(baseline PRIVATE
 # library, or context struct sizes diverge between consumer and library.
 target_compile_definitions(baseline PRIVATE MBEDTLS_USER_CONFIG_FILE=${MBEDTLS_USER_CONFIG_HEADER})
 
-target_link_libraries(baseline PRIVATE mbedtls mbedx509 mbedcrypto SolidSyslog SolidSyslog::LwipRaw SolidSyslog::FreeRtos)
+target_link_libraries(baseline PRIVATE mbedtls mbedx509 mbedcrypto SolidSyslog SolidSyslog::LwipRaw SolidSyslog::FreeRtos SolidSyslog::FatFs)
 
 target_link_options(baseline PRIVATE
     -mcpu=cortex-m3 -mthumb

README.md

@@ -10,42 +10,51 @@ It builds on a baseline that simulates the sort of device you might be adding th
 measures itself: see [docs/baseline.md](docs/baseline.md) for what the baseline is, how the
 figures are made, and how to run it.
 
-## This stage — Time quality
+## This stage — File store
 
-Add `SolidSyslogTimeQualitySd`, and give `MetaSd` an uptime source alongside its counter.
+Spool to a `SolidSyslogBlockStore` over a `SolidSyslogFileBlockDevice` over the library's FatFs
+port, replacing the Null store. The service task drains the ring into storage and sends from there,
+so a failed send costs a retry rather than the record: the audit trail survives an outage instead of
+ending at it.
 
 ```c
-struct SolidSyslogMetaSdConfig metaConfig = {
-    .Counter      = SolidSyslogStdAtomicCounter_Create(),
-    .GetSysUpTime = SolidSyslogFreeRtos_GetSysUpTime,   /* new */
+#define SYSLOG_STORE_PREFIX "syslog"
+#define SYSLOG_STORE_BLOCKS 4U
+
+struct SolidSyslogBlockStoreConfig storeConfig = {
+    .BlockDevice    = SolidSyslogFileBlockDevice_Create(SolidSyslogFatFsFile_Create(), SYSLOG_STORE_PREFIX, 0U),
+    .MaxBlocks      = SYSLOG_STORE_BLOCKS,
+    .DiscardPolicy  = SOLIDSYSLOG_DISCARD_POLICY_OLDEST,
+    .SecurityPolicy = SolidSyslogCrc16Policy_Create(),
 };
-sd[1] = SolidSyslogTimeQualitySd_Create(SyslogTimeQuality);
 ```
 
-```text
-... BOOT [meta sequenceId="1" sysUpTime="385"][timeQuality tzKnown="1" isSynced="0"] device started
-```
+Three decisions come with it: how much to store, which is capacity on the medium rather than RAM;
+what happens when it fills — discard oldest, discard newest, or halt; and whether to be warned
+before that point, via the capacity-threshold callback.
 
-Time quality states how far the clock can be trusted, which matters when comparing events from
-different devices.
+This device stores four blocks, one file per block, `syslog00.log` upward on the volume it already
+mounts, and discards the oldest when full.
 
-This device reads the host clock once at boot and then free-runs on the FreeRTOS tick, so `isSynced`
-is `0` and the callback writes no `syncAccuracy`. `tzKnown` is `1`; the device works in UTC
-throughout.
+The CRC-16 detects corruption, not tampering. It catches a truncated write or bit-rot; anyone who
+can edit a stored record can recompute it. It establishes that a record came back the way it went
+in, which is the prerequisite for spooling at all. Making stored records tamper-evident, and then
+unreadable, are later stages.
 
-`sysUpTime` accompanies the sequence number. After a reboot the sequence restarts at one, and an
-uptime near zero distinguishes that from a counter wrap.
+Storing happens on the service task, so a task that calls `SolidSyslog_Log` still knows nothing
+about what happens after it returns and its stack does not move. The RAM is pool allocation and
+handles rather than buffers — nothing holds a block in memory, so the store costs its handles
+rather than its capacity.
 
-The element lands before the store because store-and-forward breaks the assumption that a record
-reaches the collector shortly after it was raised. A record can arrive hours later, so the device
-states what its clock is worth first.
+`SolidSyslog::FatFs` is a header-configured upstream, so it is both named in
+`SOLIDSYSLOG_PLATFORMS` and linked.
 
-**When you need it.** If events from this device will be ordered against events from others, or if a
-record's timestamp will be relied on after a delay.
+**When you need it.** If losing the records raised during an outage is not acceptable, or if they
+must survive a reboot.
 
 <!-- STAGE-COST:START (generated by scripts/gen-cost-table.py — do not edit by hand) -->
 
-**Cost above baseline: Flash +7,628 B, RAM +7,692 B.**
+**Cost above baseline: Flash +11,584 B, RAM +9,128 B.**
 
 <!-- STAGE-COST:END -->
 
@@ -68,6 +77,7 @@ committed as [`run-report.md`](run-report.md), and rewritten by every stage.
 | Buffered | logging that returns immediately, with the send moved off the logging task | +6,804 | +7,488 |
 | TCP | records the network retransmits instead of dropping, and a send that fails when the collector is gone | +7,336 | +7,668 |
 | Time quality | a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap | +7,628 | +7,692 |
+| File store | records that survive a failed send, spooled to disk with a checksum at rest | +11,584 | +9,128 |
 
 *Deltas are bytes above the baseline, which is itself Flash 350,308 B, RAM 111,192 B.*
 

app/syslog/Syslog.c

@@ -9,18 +9,21 @@
 
 #include "Syslog.h"
 
+#include "SolidSyslogBlockStore.h"
 #include "SolidSyslogCircularBuffer.h"
 #include "SolidSyslogConfig.h"
+#include "SolidSyslogCrc16Policy.h"
 #include "SolidSyslogEndpoint.h"
 #include "SolidSyslogEndpointHost.h"
+#include "SolidSyslogFatFsFile.h"
+#include "SolidSyslogFileBlockDevice.h"
 #include "SolidSyslogFreeRtosMutex.h"
 #include "SolidSyslogFreeRtosSysUpTime.h"
 #include "SolidSyslogLwipRawAddress.h"
 #include "SolidSyslogLwipRawMarshal.h"
 #include "SolidSyslogLwipRawResolver.h"
 #include "SolidSyslogLwipRawTcpStream.h"
 #include "SolidSyslogMetaSd.h"
-#include "SolidSyslogNullStore.h"
 #include "SolidSyslogStdAtomicCounter.h"
 #include "SolidSyslogStreamSender.h"
 #include "SolidSyslogTimeQuality.h"
@@ -46,6 +49,10 @@
  * backlog the store is there to hold. */
 #define SYSLOG_BUFFER_RECORDS 8U
 
+/* One "<prefix>NN.log" per block, on the volume the device already mounts. */
+#define SYSLOG_STORE_PREFIX "syslog"
+#define SYSLOG_STORE_BLOCKS 4U
+
 static struct SolidSyslog* s_logger = NULL;
 static uint8_t s_ring[SOLIDSYSLOG_CIRCULAR_BUFFER_RING_BYTES(SYSLOG_BUFFER_RECORDS)];
 
@@ -115,12 +122,19 @@ void Syslog_Start(void)
     s_sd[0] = SolidSyslogMetaSd_Create(&metaConfig);
     s_sd[1] = SolidSyslogTimeQualitySd_Create(SyslogTimeQuality);
 
+    /* Oldest discarded when the ceiling is reached: a device that cannot reach its
+     * collector should keep the newest evidence, not stop logging. */
+    struct SolidSyslogBlockStoreConfig storeConfig = {
+        .BlockDevice = SolidSyslogFileBlockDevice_Create(SolidSyslogFatFsFile_Create(), SYSLOG_STORE_PREFIX, 0U),
+        .MaxBlocks = SYSLOG_STORE_BLOCKS,
+        .DiscardPolicy = SOLIDSYSLOG_DISCARD_POLICY_OLDEST,
+        .SecurityPolicy = SolidSyslogCrc16Policy_Create(),
+    };
+
     struct SolidSyslogConfig config = {
         .Buffer = SolidSyslogCircularBuffer_Create(SolidSyslogFreeRtosMutex_Create(), s_ring, sizeof(s_ring)),
         .Sender = sender,
-        /* No store-and-forward here. The Null object rather than NULL is how
-         * that is said out loud — NULL is reported as a fault. */
-        .Store = SolidSyslogNullStore_Get(),
+        .Store = SolidSyslogBlockStore_Create(&storeConfig),
         /* PROCID stays unset — a bare-metal image has no process. */
         .Clock = SyslogFields_Clock,
         .GetHostname = SyslogFields_Hostname,

measurements/file-store.csv

@@ -0,0 +1,13 @@
+# file-store figures (bytes) — captured by scripts/run.sh (CAPTURE=1).
+# The device reads measurements/Baseline.csv as its frozen baseline and reports current-minus-Baseline.
+flash_text,361260
+flash_data,632
+static_bss,119688
+heap_used,4440
+mbedtls_peak,21332
+mbedtls_free,11436
+lwip_mem_free,7576
+lwip_pbufs_free,14
+stack_log,792
+stack_service,1012
+stack_harness,2848

measurements/stages.tsv

@@ -19,3 +19,4 @@ sequence-id	Sequence numbers	every record numbered, so a gap in the sequence is
 buffered	Buffered	logging that returns immediately, with the send moved off the logging task
 tcp	TCP	records the network retransmits instead of dropping, and a send that fails when the collector is gone
 time-quality	Time quality	a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap
+file-store	File store	records that survive a failed send, spooled to disk with a checksum at rest

run-report.md

@@ -1,4 +1,4 @@
-# solid-syslog-example — run (time-quality)
+# solid-syslog-example — run (file-store)
 
 ## Device (self-measured)
 
@@ -10,16 +10,16 @@
 [device]   first record logged: yes
 [report] --- SolidSyslog cost above baseline (simulated existing application) ---
 [report] key,current,baseline,used_above_baseline
-[report] flash_text,357440,349992,7448
-[report] flash_data,496,316,180
-[report] static_bss,118388,110876,7512
+[report] flash_text,361260,349992,11268
+[report] flash_data,632,316,316
+[report] static_bss,119688,110876,8812
 [report] heap_used,4440,4440,0
-[report] mbedtls_peak,21316,21332,-16
-[report] mbedtls_free,11452,11436,16
+[report] mbedtls_peak,21300,21332,-32
+[report] mbedtls_free,11468,11436,32
 [report] lwip_mem_free,7576,7576,0
-[report] lwip_pbufs_free,14,14,0
+[report] lwip_pbufs_free,13,14,-1
 [report] stack_log,792,120,672
-[report] stack_service,948,52,896
+[report] stack_service,1012,52,960
 [report] stack_harness,2848,2840,8
 [report] --- end ---
 [device] ready
@@ -29,7 +29,7 @@
 
 ```text
    text	   data	    bss	    dec	    hex	filename
- 357432	    504	 118388	 476324	  744a4	/w/build/baseline-cross/baseline.elf
+ 361252	    640	 119688	 481580	  7592c	/w/build/baseline-cross/baseline.elf
 ```
 
 ## Listeners (proved before the device ran)
@@ -47,23 +47,23 @@
 ## Collector (syslog-ng) received
 
 ```text
-wire   <134>1 2026-08-15T22:00:10.850000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="385"][timeQuality tzKnown="1" isSynced="0"] device started
-parsed PRIORITY=134 TIMESTAMP=2026-08-15T22:00:10+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="385"][timeQuality tzKnown="1" isSynced="0"] MSG=device started
+wire   <134>1 2026-08-16T06:08:05.430000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="243"][timeQuality tzKnown="1" isSynced="0"] device started
+parsed PRIORITY=134 TIMESTAMP=2026-08-16T06:08:05+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="243"][timeQuality tzKnown="1" isSynced="0"] MSG=device started
 ```
 
-## Self-check (vs measurements/time-quality.csv)
+## Self-check (vs measurements/file-store.csv)
 
 ```text
-  OK    flash_text: 357440 (expected 357440, Δ0)
-  OK    flash_data: 496 (expected 496, Δ0)
-  OK    static_bss: 118388 (expected 118388, Δ0)
+  OK    flash_text: 361260 (expected 361260, Δ0)
+  OK    flash_data: 632 (expected 632, Δ0)
+  OK    static_bss: 119688 (expected 119688, Δ0)
   OK    heap_used: 4440 (expected 4440, Δ0)
-  OK    mbedtls_peak: 21316 (expected 21288, Δ28)
-  OK    mbedtls_free: 11452 (expected 11480, Δ28)
+  OK    mbedtls_peak: 21300 (expected 21332, Δ32)
+  OK    mbedtls_free: 11468 (expected 11436, Δ32)
   OK    lwip_mem_free: 7576 (expected 7576, Δ0)
-  OK    lwip_pbufs_free: 14 (expected 14, Δ0)
+  OK    lwip_pbufs_free: 13 (expected 14, Δ1)
   OK    stack_log: 792 (expected 792, Δ0)
-  OK    stack_service: 948 (expected 948, Δ0)
+  OK    stack_service: 1012 (expected 1012, Δ0)
   OK    stack_harness: 2848 (expected 2848, Δ0)
 ```