Skip to content

Allow encoding into chunks#15

Merged
Teufelchen1 merged 6 commits into
Teufelchen1:mainfrom
chrysn-pull-requests:chunkable
Dec 9, 2025
Merged

Allow encoding into chunks#15
Teufelchen1 merged 6 commits into
Teufelchen1:mainfrom
chrysn-pull-requests:chunkable

Conversation

@chrysn

@chrysn chrysn commented Dec 5, 2025

Copy link
Copy Markdown
Contributor

Currently, if I have a 1200 byte CoAP buffer in a constrained device and want to send that to a UART (which is maybe buffered to a few bytes), I need to allocate a 2404-ish byte output buffer.

This PR adds a struct that can deal out pieces of data in chunks down to 2 bytes; it is a heavy refactoring of what used to be encode (which itself is now just driving that encoder through a single step).

It's a heavy PR in terms of LoC (+167-22 for a net +145, although 48 thereof is just tests), but I think it is useful for embedded applications. On the history, feel free to peruse if you feel like digging into history, but due to PR-internal refactoring, this is best reviewed as a single large diff.

The manually written generator is not preptty, but it allows writing
from a single maximal message size frame into a UART (eg. by buffering
in some reasonable buffer size of 16 bytes) without a double-sized
buffer.
Rather than keeping running totals and always indexing into slices, the
slices themselves are advanced, reducing the cognitive overhead of
tracking indices in many places.
Rather than anticipating how much space is needed, this lets the slip
encoder's general well-behavedness with empty buffers do its work.

A state is reduced by observing that none of the headers need escaping
anyway.
Rather than building a big state machine, this pulls out the common
pattern of having some slice of data that progress is made on.
Comment thread src/encode.rs
Comment thread src/encode.rs
Comment thread src/encode.rs
@Teufelchen1

Copy link
Copy Markdown
Owner

Thanks! This looks great!

Comment thread src/encode.rs
Comment on lines +152 to +153
/// Unlike many other write-style methods, this does *not* fill up the buffer to the last byte
/// except for the last block; this simplifies encoding escaped bytes.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it does fill the buffer up the buffer to the last byte? Not if the buffer is very big but if the buffer is smaller than the data, it will always fill the buffer to the limit.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's a large buffer, it fills it up to the last byte (and there's your existing test that covers this).

Repeated small buffers will eventually be reading up the whole message.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a corner case where if there is a large fixed-size buffer, and the application only presents that buffer to slipmux picemeal, and that buffer is odd-sized, and then in a maximal message, there's only the End byte missing, and the application only has a single byte to hadn to the library, then the "panics if <2 byte buffer" triggers even though technically things would fit.

But that'd make the usage condition more complex to describe and to test, and I don't see why anyone would do that: Either it's a fixed buffer, then it can be passed to encode_chunk as a whole, or it's some kind of streaming interface, where you get fixed buffers every time. (Or it's a streaming interface that gives you as much buffer as is free, and then it just takes a jiffy longer for a 2-long buffer to get free so that the function can be called properly again).

@Teufelchen1
Teufelchen1 merged commit d79d938 into Teufelchen1:main Dec 9, 2025
1 check passed
@chrysn
chrysn deleted the chunkable branch December 11, 2025 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants