Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 28 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,51 +10,47 @@ It builds on a baseline that simulates the sort of device you might be adding th
measures itself: see [docs/baseline.md](docs/baseline.md) for what the baseline is, how the
figures are made, and how to run it.

## This stage — File store
## This stage — Origin

Spool to a `SolidSyslogBlockStore` over a `SolidSyslogFileBlockDevice` over the library's FatFs
port, replacing the Null store. The service task drains the ring into storage and sends from there,
so a failed send costs a retry rather than the record: the audit trail survives an outage instead of
ending at it.
Name the device in the record with `SolidSyslogOriginSd` — the software, its version, and the
enterprise number.

```c
#define SYSLOG_STORE_PREFIX "syslog"
#define SYSLOG_STORE_BLOCKS 4U

struct SolidSyslogBlockStoreConfig storeConfig = {
.BlockDevice = SolidSyslogFileBlockDevice_Create(SolidSyslogFatFsFile_Create(), SYSLOG_STORE_PREFIX, 0U),
.MaxBlocks = SYSLOG_STORE_BLOCKS,
.DiscardPolicy = SOLIDSYSLOG_DISCARD_POLICY_OLDEST,
.SecurityPolicy = SolidSyslogCrc16Policy_Create(),
#define SYSLOG_SOFTWARE "solid-syslog-example"
#define SYSLOG_SW_VERSION "0.1.0"

struct SolidSyslogOriginSdConfig originConfig = {
.Software = SYSLOG_SOFTWARE,
.SwVersion = SYSLOG_SW_VERSION,
.EnterpriseId = SYSLOG_ENTERPRISE_ID,
};
sd[2] = SolidSyslogOriginSd_Create(&originConfig);
Comment thread
coderabbitai[bot] marked this conversation as resolved.
```

Three decisions come with it: how much to store, which is capacity on the medium rather than RAM;
what happens when it fills — discard oldest, discard newest, or halt; and whether to be warned
before that point, via the capacity-threshold callback.
```text
... [origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] device started
```

This device stores four blocks, one file per block, `syslog00.log` upward on the volume it already
mounts, and discards the oldest when full.
This lands after the store rather than before it. While records went straight out, the answer to
"who sent this" was implied by the connection they arrived on. Once records can replay hours later
that is no longer so, and the record has to carry it.

The CRC-16 detects corruption, not tampering. It catches a truncated write or bit-rot; anyone who
can edit a stored record can recompute it. It establishes that a record came back the way it went
in, which is the prerequisite for spooling at all. Making stored records tamper-evident, and then
unreadable, are later stages.
The `ip` PARAM is left out here. The address the collector sees is still the address that reached
it; the next stage takes that assumption away.

Storing happens on the service task, so a task that calls `SolidSyslog_Log` still knows nothing
about what happens after it returns and its stack does not move. The RAM is pool allocation and
handles rather than buffers — nothing holds a block in memory, so the store costs its handles
rather than its capacity.
`SYSLOG_ENTERPRISE_ID` is defined in its own header rather than beside the element that carries it,
because the number identifies the vendor rather than the logger — anything else this product puts
its own name on wants the same one.

`SolidSyslog::FatFs` is a header-configured upstream, so it is both named in
`SOLIDSYSLOG_PLATFORMS` and linked.
> Enterprise number 32473 is reserved for documentation and testing by RFC 5612. A shipping product
> uses its own, registered with IANA.

**When you need it.** If losing the records raised during an outage is not acceptable, or if they
must survive a reboot.
**When you need it.** If records will be correlated across devices, replayed after a delay, or
relayed through anything.

<!-- STAGE-COST:START (generated by scripts/gen-cost-table.py — do not edit by hand) -->

**Cost above baseline: Flash +11,584 B, RAM +9,128 B.**
**Cost above baseline: Flash +11,988 B, RAM +9,172 B.**

<!-- STAGE-COST:END -->

Expand All @@ -78,6 +74,7 @@ committed as [`run-report.md`](run-report.md), and rewritten by every stage.
| TCP | records the network retransmits instead of dropping, and a send that fails when the collector is gone | +7,336 | +7,668 |
| Time quality | a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap | +7,628 | +7,692 |
| File store | records that survive a failed send, spooled to disk with a checksum at rest | +11,584 | +9,128 |
| Origin | the device named in the record itself, not inferred from the source address | +11,988 | +9,172 |

*Deltas are bytes above the baseline, which is itself Flash 350,308 B, RAM 111,192 B.*

Expand Down
16 changes: 15 additions & 1 deletion app/syslog/Syslog.c
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,12 @@
#include "SolidSyslogLwipRawResolver.h"
#include "SolidSyslogLwipRawTcpStream.h"
#include "SolidSyslogMetaSd.h"
#include "SolidSyslogOriginSd.h"
#include "SolidSyslogStdAtomicCounter.h"
#include "SolidSyslogStreamSender.h"
#include "SolidSyslogTimeQuality.h"
#include "SolidSyslogTimeQualitySd.h"
#include "SyslogEnterprise.h"
#include "SyslogFields.h"

#include "lwip/tcpip.h"
Expand All @@ -53,11 +55,14 @@
#define SYSLOG_STORE_PREFIX "syslog"
#define SYSLOG_STORE_BLOCKS 4U

#define SYSLOG_SOFTWARE "solid-syslog-example"
#define SYSLOG_SW_VERSION "0.1.0"

static struct SolidSyslog* s_logger = NULL;
static uint8_t s_ring[SOLIDSYSLOG_CIRCULAR_BUFFER_RING_BYTES(SYSLOG_BUFFER_RECORDS)];

/* The logger reads these on every record, so they outlive Syslog_Start. */
static struct SolidSyslogStructuredData* s_sd[2];
static struct SolidSyslogStructuredData* s_sd[3];

/* One reading at boot, then free-running on the tick — enough to stamp a record,
* not synchronisation. RFC 5424 section 7.1.3 forbids syncAccuracy alongside an
Expand Down Expand Up @@ -122,6 +127,15 @@ void Syslog_Start(void)
s_sd[0] = SolidSyslogMetaSd_Create(&metaConfig);
s_sd[1] = SolidSyslogTimeQualitySd_Create(SyslogTimeQuality);

/* No ip: the address the collector sees is the one that reached it, until a
* relay makes that untrue. */
struct SolidSyslogOriginSdConfig originConfig = {
.Software = SYSLOG_SOFTWARE,
.SwVersion = SYSLOG_SW_VERSION,
.EnterpriseId = SYSLOG_ENTERPRISE_ID,
};
s_sd[2] = SolidSyslogOriginSd_Create(&originConfig);

struct SolidSyslogBlockStoreConfig storeConfig = {
.BlockDevice = SolidSyslogFileBlockDevice_Create(SolidSyslogFatFsFile_Create(), SYSLOG_STORE_PREFIX, 0U),
.MaxBlocks = SYSLOG_STORE_BLOCKS,
Expand Down
12 changes: 12 additions & 0 deletions app/syslog/SyslogEnterprise.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/* This product's IANA Private Enterprise Number. It identifies the vendor, not
* the logger, so it lives on its own rather than beside any one element that
* carries it.
*
* 32473 is reserved for documentation (RFC 5612). Register your own at
* https://www.iana.org/assignments/enterprise-numbers/ */
#ifndef APP_SYSLOG_ENTERPRISE_H
#define APP_SYSLOG_ENTERPRISE_H

#define SYSLOG_ENTERPRISE_ID "32473"

#endif /* APP_SYSLOG_ENTERPRISE_H */
13 changes: 13 additions & 0 deletions measurements/origin.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# origin figures (bytes) — captured by scripts/run.sh (CAPTURE=1).
# The device reads measurements/Baseline.csv as its frozen baseline and reports current-minus-Baseline.
flash_text,361656
flash_data,640
static_bss,119724
heap_used,4440
mbedtls_peak,21332
mbedtls_free,11436
lwip_mem_free,7576
lwip_pbufs_free,13
stack_log,792
stack_service,1012
stack_harness,2848
1 change: 1 addition & 0 deletions measurements/stages.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ buffered Buffered logging that returns immediately, with the send moved off the
tcp TCP records the network retransmits instead of dropping, and a send that fails when the collector is gone
time-quality Time quality a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap
file-store File store records that survive a failed send, spooled to disk with a checksum at rest
origin Origin the device named in the record itself, not inferred from the source address
34 changes: 17 additions & 17 deletions run-report.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# solid-syslog-example — run (file-store)
# solid-syslog-example — run (origin)

## Device (self-measured)

Expand All @@ -10,14 +10,14 @@
[device] first record logged: yes
[report] --- SolidSyslog cost above baseline (simulated existing application) ---
[report] key,current,baseline,used_above_baseline
[report] flash_text,361260,349992,11268
[report] flash_data,632,316,316
[report] static_bss,119688,110876,8812
[report] flash_text,361656,349992,11664
[report] flash_data,640,316,324
[report] static_bss,119724,110876,8848
[report] heap_used,4440,4440,0
[report] mbedtls_peak,21332,21332,0
[report] mbedtls_free,11436,11436,0
[report] mbedtls_peak,21240,21332,-92
[report] mbedtls_free,11528,11436,92
[report] lwip_mem_free,7576,7576,0
[report] lwip_pbufs_free,14,14,0
[report] lwip_pbufs_free,13,14,-1
[report] stack_log,792,120,672
[report] stack_service,1012,52,960
[report] stack_harness,2848,2840,8
Expand All @@ -29,7 +29,7 @@

```text
text data bss dec hex filename
361252 640 119688 481580 7592c /w/build/baseline-cross/baseline.elf
361648 648 119724 482020 75ae4 /w/build/baseline-cross/baseline.elf
```

## Listeners (proved before the device ran)
Expand All @@ -47,21 +47,21 @@
## Collector (syslog-ng) received

```text
wire <134>1 2026-08-16T06:24:46.850000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="385"][timeQuality tzKnown="1" isSynced="0"] device started
parsed PRIORITY=134 TIMESTAMP=2026-08-16T06:24:46+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="385"][timeQuality tzKnown="1" isSynced="0"] MSG=device started
wire <134>1 2026-08-16T06:31:59.830000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="283"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] device started
parsed PRIORITY=134 TIMESTAMP=2026-08-16T06:31:59+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="283"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] MSG=device started
```

## Self-check (vs measurements/file-store.csv)
## Self-check (vs measurements/origin.csv)

```text
OK flash_text: 361260 (expected 361260, Δ0)
OK flash_data: 632 (expected 632, Δ0)
OK static_bss: 119688 (expected 119688, Δ0)
OK flash_text: 361656 (expected 361656, Δ0)
OK flash_data: 640 (expected 640, Δ0)
OK static_bss: 119724 (expected 119724, Δ0)
OK heap_used: 4440 (expected 4440, Δ0)
OK mbedtls_peak: 21332 (expected 21332, Δ0)
OK mbedtls_free: 11436 (expected 11436, Δ0)
OK mbedtls_peak: 21240 (expected 21332, Δ92)
OK mbedtls_free: 11528 (expected 11436, Δ92)
OK lwip_mem_free: 7576 (expected 7576, Δ0)
OK lwip_pbufs_free: 14 (expected 14, Δ0)
OK lwip_pbufs_free: 13 (expected 13, Δ0)
OK stack_log: 792 (expected 792, Δ0)
OK stack_service: 1012 (expected 1012, Δ0)
OK stack_harness: 2848 (expected 2848, Δ0)
Expand Down
Loading