Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 28 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,51 +10,47 @@ It builds on a baseline that simulates the sort of device you might be adding th
measures itself: see [docs/baseline.md](docs/baseline.md) for what the baseline is, how the
figures are made, and how to run it.

## This stage — File store
## This stage — Origin

Spool to a `SolidSyslogBlockStore` over a `SolidSyslogFileBlockDevice` over the library's FatFs
port, replacing the Null store. The service task drains the ring into storage and sends from there,
so a failed send costs a retry rather than the record: the audit trail survives an outage instead of
ending at it.
Name the device in the record with `SolidSyslogOriginSd` — the software, its version, and the
enterprise number.

```c
#define SYSLOG_STORE_PREFIX "syslog"
#define SYSLOG_STORE_BLOCKS 4U

struct SolidSyslogBlockStoreConfig storeConfig = {
.BlockDevice = SolidSyslogFileBlockDevice_Create(SolidSyslogFatFsFile_Create(), SYSLOG_STORE_PREFIX, 0U),
.MaxBlocks = SYSLOG_STORE_BLOCKS,
.DiscardPolicy = SOLIDSYSLOG_DISCARD_POLICY_OLDEST,
.SecurityPolicy = SolidSyslogCrc16Policy_Create(),
#define SYSLOG_SOFTWARE "solid-syslog-example"
#define SYSLOG_SW_VERSION "0.1.0"

struct SolidSyslogOriginSdConfig originConfig = {
.Software = SYSLOG_SOFTWARE,
.SwVersion = SYSLOG_SW_VERSION,
.EnterpriseId = SYSLOG_ENTERPRISE_ID,
};
sd[2] = SolidSyslogOriginSd_Create(&originConfig);
```

Three decisions come with it: how much to store, which is capacity on the medium rather than RAM;
what happens when it fills — discard oldest, discard newest, or halt; and whether to be warned
before that point, via the capacity-threshold callback.
```text
... [origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] device started
```

This device stores four blocks, one file per block, `syslog00.log` upward on the volume it already
mounts, and discards the oldest when full.
This lands after the store rather than before it. While records went straight out, the answer to
"who sent this" was implied by the connection they arrived on. Once records can replay hours later
that is no longer so, and the record has to carry it.

The CRC-16 detects corruption, not tampering. It catches a truncated write or bit-rot; anyone who
can edit a stored record can recompute it. It establishes that a record came back the way it went
in, which is the prerequisite for spooling at all. Making stored records tamper-evident, and then
unreadable, are later stages.
The `ip` PARAM is left out here. The address the collector sees is still the address that reached
it; the next stage takes that assumption away.

Storing happens on the service task, so a task that calls `SolidSyslog_Log` still knows nothing
about what happens after it returns and its stack does not move. The RAM is pool allocation and
handles rather than buffers — nothing holds a block in memory, so the store costs its handles
rather than its capacity.
`SYSLOG_ENTERPRISE_ID` is defined in its own header rather than beside the element that carries it,
because the number identifies the vendor rather than the logger — anything else this product puts
its own name on wants the same one.

`FatFs` joins `SOLIDSYSLOG_PLATFORMS`, and its sources compile against this device's own `ffconf.h`
like the rest of the application.
> Enterprise number 32473 is reserved for documentation and testing by RFC 5612. A shipping product
> uses its own, registered with IANA.

**When you need it.** If losing the records raised during an outage is not acceptable, or if they
must survive a reboot.
**When you need it.** If records will be correlated across devices, replayed after a delay, or
relayed through anything.

<!-- STAGE-COST:START (generated by scripts/gen-cost-table.py — do not edit by hand) -->

**Cost above baseline: Flash +11,596 B, RAM +9,124 B.**
**Cost above baseline: Flash +11,988 B, RAM +9,168 B.**

<!-- STAGE-COST:END -->

Expand All @@ -78,6 +74,7 @@ committed as [`run-report.md`](run-report.md), and rewritten by every stage.
| TCP | records the network retransmits instead of dropping, and a send that fails when the collector is gone | +7,348 | +7,664 |
| Time quality | a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap | +7,644 | +7,688 |
| File store | records that survive a failed send, spooled to disk with a checksum at rest | +11,596 | +9,124 |
| Origin | the device named in the record itself, not inferred from the source address | +11,988 | +9,168 |

*Deltas are bytes above the baseline, which is itself Flash 350,124 B, RAM 111,192 B.*

Expand Down
16 changes: 15 additions & 1 deletion app/syslog/Syslog.c
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,12 @@
#include "SolidSyslogLwipRawResolver.h"
#include "SolidSyslogLwipRawTcpStream.h"
#include "SolidSyslogMetaSd.h"
#include "SolidSyslogOriginSd.h"
#include "SolidSyslogStdAtomicCounter.h"
#include "SolidSyslogStreamSender.h"
#include "SolidSyslogTimeQuality.h"
#include "SolidSyslogTimeQualitySd.h"
#include "SyslogEnterprise.h"
#include "SyslogFields.h"

#include "lwip/tcpip.h"
Expand All @@ -53,11 +55,14 @@
#define SYSLOG_STORE_PREFIX "syslog"
#define SYSLOG_STORE_BLOCKS 4U

#define SYSLOG_SOFTWARE "solid-syslog-example"
#define SYSLOG_SW_VERSION "0.1.0"

static struct SolidSyslog* s_logger = NULL;
static uint8_t s_ring[SOLIDSYSLOG_CIRCULAR_BUFFER_RING_BYTES(SYSLOG_BUFFER_RECORDS)];

/* The logger reads these on every record, so they outlive Syslog_Start. */
static struct SolidSyslogStructuredData* s_sd[2];
static struct SolidSyslogStructuredData* s_sd[3];

/* One reading at boot, then free-running on the tick — enough to stamp a record,
* not synchronisation. RFC 5424 section 7.1.3 forbids syncAccuracy alongside an
Expand Down Expand Up @@ -122,6 +127,15 @@ void Syslog_Start(void)
s_sd[0] = SolidSyslogMetaSd_Create(&metaConfig);
s_sd[1] = SolidSyslogTimeQualitySd_Create(SyslogTimeQuality);

/* No ip: the address the collector sees is the one that reached it, until a
* relay makes that untrue. */
struct SolidSyslogOriginSdConfig originConfig = {
.Software = SYSLOG_SOFTWARE,
.SwVersion = SYSLOG_SW_VERSION,
.EnterpriseId = SYSLOG_ENTERPRISE_ID,
};
s_sd[2] = SolidSyslogOriginSd_Create(&originConfig);

struct SolidSyslogBlockStoreConfig storeConfig = {
.BlockDevice = SolidSyslogFileBlockDevice_Create(SolidSyslogFatFsFile_Create(), SYSLOG_STORE_PREFIX, 0U),
.MaxBlocks = SYSLOG_STORE_BLOCKS,
Expand Down
12 changes: 12 additions & 0 deletions app/syslog/SyslogEnterprise.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
/* This product's IANA Private Enterprise Number. It identifies the vendor, not
* the logger, so it lives on its own rather than beside any one element that
* carries it.
*
* 32473 is reserved for documentation (RFC 5612). Register your own at
* https://www.iana.org/assignments/enterprise-numbers/ */
#ifndef APP_SYSLOG_ENTERPRISE_H
#define APP_SYSLOG_ENTERPRISE_H

#define SYSLOG_ENTERPRISE_ID "32473"

#endif /* APP_SYSLOG_ENTERPRISE_H */
13 changes: 13 additions & 0 deletions measurements/origin.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# origin figures (bytes) — captured by scripts/run.sh (CAPTURE=1).
# The device reads measurements/Baseline.csv as its frozen baseline and reports current-minus-Baseline.
flash_text,361472
flash_data,640
static_bss,119720
heap_used,4440
mbedtls_peak,21256
mbedtls_free,11512
lwip_mem_free,7576
lwip_pbufs_free,13
stack_log,792
stack_service,1012
stack_harness,2848
1 change: 1 addition & 0 deletions measurements/stages.tsv
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ buffered Buffered logging that returns immediately, with the send moved off the
tcp TCP records the network retransmits instead of dropping, and a send that fails when the collector is gone
time-quality Time quality a timestamp the collector knows how far to trust, and an uptime that tells a reboot from a counter wrap
file-store File store records that survive a failed send, spooled to disk with a checksum at rest
origin Origin the device named in the record itself, not inferred from the source address
32 changes: 16 additions & 16 deletions run-report.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# solid-syslog-example — run (file-store)
# solid-syslog-example — run (origin)

## Device (self-measured)

Expand All @@ -10,12 +10,12 @@
[device] first record logged: yes
[report] --- SolidSyslog cost above baseline (simulated existing application) ---
[report] key,current,baseline,used_above_baseline
[report] flash_text,361088,349808,11280
[report] flash_data,632,316,316
[report] static_bss,119684,110876,8808
[report] flash_text,361472,349808,11664
[report] flash_data,640,316,324
[report] static_bss,119720,110876,8844
[report] heap_used,4440,4440,0
[report] mbedtls_peak,21324,21328,-4
[report] mbedtls_free,11444,11440,4
[report] mbedtls_peak,21340,21328,12
[report] mbedtls_free,11428,11440,-12
[report] lwip_mem_free,7576,7576,0
[report] lwip_pbufs_free,13,13,0
[report] stack_log,792,120,672
Expand All @@ -29,7 +29,7 @@

```text
text data bss dec hex filename
361080 640 119684 481404 7587c /w/build/baseline.elf
361464 648 119720 481832 75a28 /w/build/baseline.elf
```

## Listeners (proved before the device ran)
Expand All @@ -47,21 +47,21 @@
## Collector (syslog-ng) received

```text
wire <134>1 2026-08-16T19:35:52.310000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="231"][timeQuality tzKnown="1" isSynced="0"] device started
parsed PRIORITY=134 TIMESTAMP=2026-08-16T19:35:52+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="231"][timeQuality tzKnown="1" isSynced="0"] MSG=device started
wire <134>1 2026-08-16T19:38:12.360000Z 10.0.2.15 solid-syslog-example - BOOT [meta sequenceId="1" sysUpTime="236"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] device started
parsed PRIORITY=134 TIMESTAMP=2026-08-16T19:38:12+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA=[meta sequenceId="1" sysUpTime="236"][timeQuality tzKnown="1" isSynced="0"][origin software="solid-syslog-example" swVersion="0.1.0" enterpriseId="32473"] MSG=device started
```

## Self-check (vs measurements/file-store.csv)
## Self-check (vs measurements/origin.csv)

```text
OK flash_text: 361088 (expected 361088, Δ0)
OK flash_data: 632 (expected 632, Δ0)
OK static_bss: 119684 (expected 119684, Δ0)
OK flash_text: 361472 (expected 361472, Δ0)
OK flash_data: 640 (expected 640, Δ0)
OK static_bss: 119720 (expected 119720, Δ0)
OK heap_used: 4440 (expected 4440, Δ0)
OK mbedtls_peak: 21324 (expected 21332, Δ8)
OK mbedtls_free: 11444 (expected 11436, Δ8)
OK mbedtls_peak: 21340 (expected 21256, Δ84)
OK mbedtls_free: 11428 (expected 11512, Δ84)
OK lwip_mem_free: 7576 (expected 7576, Δ0)
OK lwip_pbufs_free: 13 (expected 14, Δ1)
OK lwip_pbufs_free: 13 (expected 13, Δ0)
OK stack_log: 792 (expected 792, Δ0)
OK stack_service: 1012 (expected 1012, Δ0)
OK stack_harness: 2848 (expected 2848, Δ0)
Expand Down