From 4ed519b2f794ad460aeb5f6e2b383585549cab78 Mon Sep 17 00:00:00 2001 From: David Cozens Date: Sun, 16 Aug 2026 20:19:51 +0100 Subject: [PATCH] feat: fill in the RFC 5424 header fields MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TIMESTAMP, HOSTNAME and APP-NAME come from what the device already has — the wall clock it acquired at boot, the address on its interface, and its own name. PROCID stays nil, because a bare-metal image has no process to identify. <134>1 2026-08-16T19:17:54.430000Z 10.0.2.15 solid-syslog-example - BOOT - device started Flash +5,108 B (+384 on the previous stage) RAM +1,908 B (unchanged) 384 bytes is what a record that says when it happened and which device it came from costs over one that says neither. That is the difference between a log line and evidence, and it is worth knowing the price of separately. Two things the adapters have to get right. The timestamp struct is zeroed before use, so a clock that cannot answer leaves Month == 0, fails the library's validation, and is emitted as the nil value rather than as a wrong time. And the hostname is read under the lwIP core lock, with ip4addr_ntoa_r rather than ip4addr_ntoa — the latter shares one static buffer across callers. RFC 5424 section 6.2.4 allows an address where a device has no resolvable name, which is this device exactly. Co-Authored-By: Claude Opus 5 (1M context) --- Makefile | 3 +- README.md | 57 +++++++++++----------------- app/syslog/Syslog.c | 11 +++--- app/syslog/SyslogFields.c | 68 ++++++++++++++++++++++++++++++++++ app/syslog/SyslogFields.h | 20 ++++++++++ measurements/header-fields.csv | 13 +++++++ measurements/stages.tsv | 1 + run-report.md | 26 ++++++------- 8 files changed, 145 insertions(+), 54 deletions(-) create mode 100644 app/syslog/SyslogFields.c create mode 100644 app/syslog/SyslogFields.h create mode 100644 measurements/header-fields.csv diff --git a/Makefile b/Makefile index a8ca5bd..a466d90 100644 --- a/Makefile +++ b/Makefile @@ -52,7 +52,8 @@ APP_SRCS := \ $(APP_DIR)/storage/diskio.c \ $(APP_DIR)/storage/SemihostingDisk.c \ $(APP_DIR)/syslog/Syslog.c \ - $(APP_DIR)/syslog/SyslogErrorHandler.c + $(APP_DIR)/syslog/SyslogErrorHandler.c \ + $(APP_DIR)/syslog/SyslogFields.c UPSTREAM_SRCS := \ $(FREERTOS_SRCS) \ diff --git a/README.md b/README.md index 802faf9..77c2185 100644 --- a/README.md +++ b/README.md @@ -10,57 +10,43 @@ It builds on a baseline that simulates the sort of device you might be adding th measures itself: see [docs/baseline.md](docs/baseline.md) for what the baseline is, how the figures are made, and how to run it. -## This stage — First record +## This stage — Header fields -The simplest configuration that sends a syslog message. A `SolidSyslogUdpSender` over lwIP's raw -API, with a `SolidSyslogPassthroughBuffer` in front of it, so `SolidSyslog_Log` formats the record -and hands it straight to the sender on the calling task — no queue, no background drain, nothing -to service. +The record so far carries no timestamp and no device name. Fill the RFC 5424 header fields from +what the device already has: the clock it acquired at boot, the address on its interface, and its +own name. ```c -struct SolidSyslogUdpSenderConfig senderConfig = { - .Resolver = SolidSyslogLwipRawResolver_Create(), - .Datagram = SolidSyslogLwipRawDatagram_Create(), - .Address = SolidSyslogLwipRawAddress_Create(), - .Endpoint = CollectorEndpoint, -}; -struct SolidSyslogSender* sender = SolidSyslogUdpSender_Create(&senderConfig); - struct SolidSyslogConfig config = { - .Buffer = SolidSyslogPassthroughBuffer_Create(sender), - .Sender = sender, - .Store = SolidSyslogNullStore_Get(), + /* ... */ + .Clock = SyslogFields_Clock, + .GetHostname = SyslogFields_Hostname, + .GetAppName = SyslogFields_AppName, }; ``` -What arrives is a valid RFC 5424 record any collector will parse: - ```text -<134>1 - - - - BOOT - device started +<134>1 2026-08-16T19:17:54.430000Z 10.0.2.15 solid-syslog-example - BOOT - device started ``` -Timestamp, hostname, app-name and process-id are the RFC's nil value. The record is valid without -them; filling them in is the next stage. The three bad-config reports from the previous stage are -gone, which is the other half of that stage's point. +PROCID stays nil, because a bare-metal image has no process to identify, and so does +STRUCTURED-DATA until the next stage. -**The record is built on the stack of whichever task calls `SolidSyslog_Log`.** Its size follows -`SOLIDSYSLOG_MAX_MESSAGE_SIZE`, so the logging task needs room for the record and the send beneath -it — here the task was at the RTOS floor and had to grow. It is sized generously for now and -tightened against measured high-water marks at the end. +Two things the adapters have to get right. The timestamp struct is zeroed before it is filled, so a +clock that cannot answer leaves `Month == 0`, fails the library's validation, and is emitted as the +nil value rather than as a wrong time. And the hostname is read under the lwIP core lock, with +`ip4addr_ntoa_r` rather than `ip4addr_ntoa` — the latter shares one static buffer across callers. -Two details are worth getting right. Every lwIP raw call has to happen on the thread that owns the -lwIP core; `lwipopts.h` sets `LWIP_TCPIP_CORE_LOCKING`, so taking the core lock in the caller's own -task is simpler than posting to the tcpip mailbox and is unconditionally synchronous, which the -marshal contract requires. And the collector address is a numeric literal, which keeps the resolver -numeric-only — no DNS, so no `LWIP_DNS` and no DNS resolver component compiled in. +Where a device has no resolvable name, RFC 5424 section 6.2.4 allows its address in the HOSTNAME +field instead, which is this device exactly. -**When you need it.** Every device needs this much. The question is whether UDP is enough: it drops -records silently, and anyone on the path can read them. If either matters, treat UDP as a stepping -stone to the TCP and TLS stages. +**When you need it.** As soon as more than one device reports to the collector, or a record's time +will be relied on. Everything the later stages add — a sequence number, the clock's quality, the +device's own identity — builds on these fields rather than replacing them. -**Cost above baseline: Flash +4,724 B, RAM +1,908 B.** +**Cost above baseline: Flash +5,108 B, RAM +1,908 B.** @@ -78,6 +64,7 @@ committed as [`run-report.md`](run-report.md), and rewritten by every stage. | Error handler | a fault inside the logger reaches the console instead of being silent | +412 | +8 | | Logger created | the logger object, reporting exactly what is still missing from it | +1,060 | +184 | | First record | a valid RFC 5424 record on the wire, over UDP | +4,724 | +1,908 | +| Header fields | a timestamped record naming the device, instead of nil values | +5,108 | +1,908 | *Deltas are bytes above the baseline, which is itself Flash 350,124 B, RAM 111,192 B.* diff --git a/app/syslog/Syslog.c b/app/syslog/Syslog.c index d4a4733..8bf232c 100644 --- a/app/syslog/Syslog.c +++ b/app/syslog/Syslog.c @@ -2,11 +2,7 @@ * * The smallest wiring that delivers: a UDP sender over lwIP, with a passthrough * buffer in front of it. Passthrough means Log sends inline on the calling task - * — no queue, no background drain, nothing to service. - * - * Timestamp, hostname, app-name and procid are left unset. RFC 5424 defines a - * NILVALUE for each, so a record carrying "-" for them is valid and a collector - * accepts it. */ + * — no queue, no background drain, nothing to service. */ #include "Syslog.h" @@ -20,6 +16,7 @@ #include "SolidSyslogNullStore.h" #include "SolidSyslogPassthroughBuffer.h" #include "SolidSyslogUdpSender.h" +#include "SyslogFields.h" #include "lwip/tcpip.h" @@ -79,6 +76,10 @@ void Syslog_Start(void) /* No store-and-forward here. The Null object rather than NULL is how * that is said out loud — NULL is reported as a fault. */ .Store = SolidSyslogNullStore_Get(), + /* PROCID stays unset — a bare-metal image has no process. */ + .Clock = SyslogFields_Clock, + .GetHostname = SyslogFields_Hostname, + .GetAppName = SyslogFields_AppName, }; s_logger = SolidSyslog_Create(&config); diff --git a/app/syslog/SyslogFields.c b/app/syslog/SyslogFields.c new file mode 100644 index 0000000..dd3b191 --- /dev/null +++ b/app/syslog/SyslogFields.c @@ -0,0 +1,68 @@ +/* See SyslogFields.h. */ + +#include "SyslogFields.h" + +#include "DeviceClock.h" + +#include "SolidSyslogHeaderField.h" +#include "SolidSyslogTimestamp.h" + +#include "lwip/ip4_addr.h" +#include "lwip/netif.h" +#include "lwip/tcpip.h" + +#include +#include +#include + +#define SYSLOG_APP_NAME "solid-syslog-example" + +void SyslogFields_Clock(struct SolidSyslogTimestamp* timestamp) +{ + struct tm utc; + uint32_t microseconds = 0U; + + /* Zeroed means "no usable time": Month == 0 fails the library's validation + * and the field is emitted as the RFC 5424 nil value. */ + (void) memset(timestamp, 0, sizeof(*timestamp)); + + if (DeviceClock_Now(&utc, µseconds)) + { + timestamp->Year = (uint16_t) (utc.tm_year + 1900); + timestamp->Month = (uint8_t) (utc.tm_mon + 1); + timestamp->Day = (uint8_t) utc.tm_mday; + timestamp->Hour = (uint8_t) utc.tm_hour; + timestamp->Minute = (uint8_t) utc.tm_min; + timestamp->Second = (uint8_t) utc.tm_sec; + timestamp->Microsecond = microseconds; + timestamp->UtcOffsetMinutes = 0; + } +} + +void SyslogFields_Hostname(struct SolidSyslogHeaderField* field, void* context) +{ + (void) context; + + char address[IP4ADDR_STRLEN_MAX] = {0}; + + /* netif state belongs to the lwIP core, so read and format under its lock. + * ip4addr_ntoa_r, not ip4addr_ntoa: the latter shares one static buffer. */ + LOCK_TCPIP_CORE(); + if (netif_default != NULL) + { + (void) ip4addr_ntoa_r(netif_ip4_addr(netif_default), address, (int) sizeof(address)); + } + UNLOCK_TCPIP_CORE(); + + if (address[0] != '\0') + { + SolidSyslogHeaderField_PrintUsAscii(field, address, strlen(address)); + } +} + +void SyslogFields_AppName(struct SolidSyslogHeaderField* field, void* context) +{ + (void) context; + + SolidSyslogHeaderField_PrintUsAscii(field, SYSLOG_APP_NAME, strlen(SYSLOG_APP_NAME)); +} diff --git a/app/syslog/SyslogFields.h b/app/syslog/SyslogFields.h new file mode 100644 index 0000000..d329392 --- /dev/null +++ b/app/syslog/SyslogFields.h @@ -0,0 +1,20 @@ +/* The RFC 5424 header fields this device supplies: adapters between what the + * device already has — a wall clock, an IP address, a name — and the shapes + * SolidSyslog asks for. */ +#ifndef SYSLOG_FIELDS_H +#define SYSLOG_FIELDS_H + +struct SolidSyslogTimestamp; +struct SolidSyslogHeaderField; + +/** SolidSyslogClockFunction. */ +void SyslogFields_Clock(struct SolidSyslogTimestamp* timestamp); + +/** HOSTNAME as the interface's IPv4 address — RFC 5424 section 6.2.4 allows an + * address where a device has no resolvable name. */ +void SyslogFields_Hostname(struct SolidSyslogHeaderField* field, void* context); + +/** APP-NAME, fixed for this firmware. */ +void SyslogFields_AppName(struct SolidSyslogHeaderField* field, void* context); + +#endif /* SYSLOG_FIELDS_H */ diff --git a/measurements/header-fields.csv b/measurements/header-fields.csv new file mode 100644 index 0000000..a45d747 --- /dev/null +++ b/measurements/header-fields.csv @@ -0,0 +1,13 @@ +# header-fields figures (bytes) — captured by scripts/run.sh (CAPTURE=1). +# The device reads measurements/Baseline.csv as its frozen baseline and reports current-minus-Baseline. +flash_text,354784 +flash_data,448 +static_bss,112652 +heap_used,4440 +mbedtls_peak,21288 +mbedtls_free,11480 +lwip_mem_free,7576 +lwip_pbufs_free,13 +stack_log,1024 +stack_service,52 +stack_harness,2848 diff --git a/measurements/stages.tsv b/measurements/stages.tsv index 4fbe98a..23aea7c 100644 --- a/measurements/stages.tsv +++ b/measurements/stages.tsv @@ -14,3 +14,4 @@ linked Linked the core library and lwIP raw-mode networking, linked but not yet error-handler Error handler a fault inside the logger reaches the console instead of being silent logger Logger created the logger object, reporting exactly what is still missing from it udp First record a valid RFC 5424 record on the wire, over UDP +header-fields Header fields a timestamped record naming the device, instead of nil values diff --git a/run-report.md b/run-report.md index e760e7e..fb79995 100644 --- a/run-report.md +++ b/run-report.md @@ -1,4 +1,4 @@ -# solid-syslog-example — run (udp) +# solid-syslog-example — run (header-fields) ## Device (self-measured) @@ -10,14 +10,14 @@ [device] first record logged: yes [report] --- SolidSyslog cost above baseline (simulated existing application) --- [report] key,current,baseline,used_above_baseline -[report] flash_text,354400,349808,4592 +[report] flash_text,354784,349808,4976 [report] flash_data,448,316,132 [report] static_bss,112652,110876,1776 [report] heap_used,4440,4440,0 -[report] mbedtls_peak,21328,21328,0 -[report] mbedtls_free,11440,11440,0 +[report] mbedtls_peak,21292,21328,-36 +[report] mbedtls_free,11476,11440,36 [report] lwip_mem_free,7576,7576,0 -[report] lwip_pbufs_free,13,13,0 +[report] lwip_pbufs_free,14,13,1 [report] stack_log,1024,120,904 [report] stack_service,52,52,0 [report] stack_harness,2848,2840,8 @@ -29,7 +29,7 @@ ```text text data bss dec hex filename - 354392 456 112652 467500 7222c /w/build/baseline.elf + 354776 456 112652 467884 723ac /w/build/baseline.elf ``` ## Listeners (proved before the device ran) @@ -47,21 +47,21 @@ ## Collector (syslog-ng) received ```text -wire <134>1 - - - - BOOT - device started -parsed PRIORITY=134 TIMESTAMP=2026-08-16T19:16:36+00:00 HOSTNAME=localhost APP_NAME= PROCID= MSGID=BOOT STRUCTURED_DATA= MSG=device started +wire <134>1 2026-08-16T19:19:27.850000Z 10.0.2.15 solid-syslog-example - BOOT - device started +parsed PRIORITY=134 TIMESTAMP=2026-08-16T19:19:27+00:00 HOSTNAME=10.0.2.15 APP_NAME=solid-syslog-example PROCID= MSGID=BOOT STRUCTURED_DATA= MSG=device started ``` -## Self-check (vs measurements/udp.csv) +## Self-check (vs measurements/header-fields.csv) ```text - OK flash_text: 354400 (expected 354400, Δ0) + OK flash_text: 354784 (expected 354784, Δ0) OK flash_data: 448 (expected 448, Δ0) OK static_bss: 112652 (expected 112652, Δ0) OK heap_used: 4440 (expected 4440, Δ0) - OK mbedtls_peak: 21328 (expected 21296, Δ32) - OK mbedtls_free: 11440 (expected 11472, Δ32) + OK mbedtls_peak: 21292 (expected 21288, Δ4) + OK mbedtls_free: 11476 (expected 11480, Δ4) OK lwip_mem_free: 7576 (expected 7576, Δ0) - OK lwip_pbufs_free: 13 (expected 13, Δ0) + OK lwip_pbufs_free: 14 (expected 13, Δ1) OK stack_log: 1024 (expected 1024, Δ0) OK stack_service: 52 (expected 52, Δ0) OK stack_harness: 2848 (expected 2848, Δ0)