feat(bluewave): export Blue Wave offline mail packets

[andy5995]

Refs #119

Are you interested in this? I can add import to it...

[andy5995]

This PR is now dependent on #799

[NuSkooler]

Nice! I'll start a review in a bit, ty!

[andy5995]

Nice! I'll start a review in a bit, ty!

So... this is definitely dependent on #799 ... When that's merged, I'll fill the gap in this to use the lifted module.

[NuSkooler]

I went through the packet format against the Cutting Edge structure kit rather than taking the offsets on faith, and it holds up: INF_HEADER 1230, INF_AREA_INFO 80, MIX_REC 14, FTI_REC 186, and every field you write lands on the correct byte. The AreaFlags/MessageFlags values all match the kit's INF_* / FTI_MSG* defines, ver = 3 is right (PACKET_LEVEL 3, and MultiMail gates oldstyle on ver < 3), and the from_to_len/subject_len you advertise line up with writeFixed reserving the NUL. The hard part is correct.

Comments inline. The msglength one is the interesting one, and it's a request for a comment rather than a change.

Two things too small for their own threads: areanum[6] only holds 5 digits since writeFixed reserves the NUL, so a pinned 6-digit area number would truncate (auto-numbering starts at 1, so it never bites in practice); and the init() comment says a datStream write failure is "unhandled ... thrown at the process" directly above the line that handles it.

[NuSkooler]

Worth recording why this is +1, because the two readers disagree and the kit is ambiguous.

The kit says "seek to this exact offset, then read msglength bytes ... to load the entire message text", and also that the leading space is "NOT to be considered part of the message". Those pull opposite ways, and implementations split on it:

BlueMail, bwave::getBody:
    if (c == 0 && kar == ' ') if (++c < msglen) kar = fgetc(datFile);
    -- the space counts.

MultiMail, bluewave::getblk:
    if (!count && (kar == ' ')) kar = fgetc(infile);
    -- no increment, the space is free.

So +1 is exactly right for BlueMail, and makes MultiMail read one byte past the text: it picks up the next message's leading space (or EOF, which it maps to ' '), i.e. one invisible trailing space. Going the other way would be exactly right for MultiMail and would drop the last character of every message in BlueMail. You picked the safer side and I'd keep it.

The reason to say so in the comment above: NoCarrierMail inherits MultiMail's version of this loop verbatim, so that's the reader where you'd notice the stray space -- and "fix" it straight into truncating every message for BlueMail users.

[NuSkooler]

These constants are the same numbers the writer uses, so the suite is checking the writer against itself: if an offset were wrong, every test in this file would still pass. Worth being clear-eyed about what that buys -- it locks in round-trip consistency and would catch a refactor that shifted a field, but nothing here would catch a struct that disagreed with the kit.

I checked the layout by hand against bluewave.h and it is correct, so there's nothing to fix. But since you maintain NoCarrierMail: did you open a generated packet in it? Saying so in the PR description is worth more than any assertion in this file, and a checked-in fixture packet with a "these exact bytes" test would keep it true.

[NuSkooler]

openMap frees the area tags, which is right, but it leaves the values unchecked -- and this PR introduces three of them (number, echotag, title). A mistyped echotag silently falls back to the derived tag, which changes what replies route by, and config validate won't say a word.

meta.js already handles this shape -- messageConferences.*.areas on line 255 uses the wildcard. Something like 'messageNetworks.bluewave.areas.*': { type: 'object', closedKeys: true } plus the three keys would close it.

messageNetworks.qwk.areas has the same hole with its conference key, so this is inherited rather than something you introduced -- but Blue Wave is where it starts to matter.

[NuSkooler]

One follow-up on the area flags: ENiGMA does already have the facility for this, so it's more actionable than I first thought.

[NuSkooler]

ENiGMA can tell these apart -- getMessageAreaByTag(areaTag).addressFlavor (Message.AddressFlavor: local | ftn | email | qwk | nntp | activitypub), already used this way in fse.js and new_scan.js.

That matters because the kit publishes an exact chart for how these two flags pair with NETWORK_TYPE:

INF_NET_FIDONET  0   Local     = ECHO=off, NETMAIL=off
                     EchoMail  = ECHO=on,  NETMAIL=off
                     NetMail   = ECHO=on,  NETMAIL=on
INF_NET_INTERNET 1   Local     = ECHO=off, NETMAIL=off
                     Newsgroup = ECHO=on,  NETMAIL=off
                     E-mail    = ECHO=on,  NETMAIL=on

Against that, entry.area ? Echo : NoPublic with a hardcoded network_type of 0 misses three ways: a purely local base (the common case) gets INF_ECHO, which means network mail; the private area gets neither ECHO nor NETMAIL, so it reads as Local rather than NetMail; and an email/nntp/activitypub area is announced as FidoNet.

network_type isn't cosmetic, which I'd assumed at first -- MultiMail branches on INF_NET_INTERNET in six places: it stops stripping soft CRs, pulls \001From:, \001Message-ID: and \001References: out of the body in endproc, and switches reply addressing between internet and FidoNet.

None of it bites while the packet is one-way, but it's the field that decides reply addressing, so it's worth getting right before .UPL import lands.

[andy5995]

Posted by Claude (Opus 5), an LLM made by Anthropic, at andy5995's direction.

@NuSkooler asked whether a generated packet had been opened in NoCarrierMail. It has now, and this is what it showed.

The packet was built by driving BlueWavePacketWriter from a script rather than through a session, with a stubbed StatLog and a config carrying three areas of different address flavors. Six messages across four areas, plus a fifth area declared with no mail. It was built from a tree that already carries the area-flag change requested in review, which is not pushed to this branch yet — so the .INF records below are the proposed behaviour, not what this PR currently produces.

In the reader

It opens. CP437 renders: box drawing, the ░▒▓█ run and é ü ñ all come through. Area grouping is right — the area holding three messages reports 1 of 3, private mail reports 1 of 1.

Below the reader

Two things the screen cannot show, read out of the packet directly.

The private message carries the flag. Its .FTI record is flags=0x0101, FTI_MSGLOCAL | FTI_MSGPRIVATE.

msglength spans the leading space and the text, and stops there. For every message, the byte at msgptr + msglength - 1 is the real final character of the body — message 6 ends on 0xA4, CP437 ñ, with no padding after it. That is the BlueMail-correct side of the ambiguity you described, and it means a reader using the MultiMail loop reads exactly one byte beyond: the next message's leading space, or EOF mapped to ' ' for the last message in the .DAT. One invisible trailing space, never a truncation. The comment in appendMessage() now records this so the next person does not "fix" it.

The area records:

Area	area_flags	network_type	Reads as
a local base	0x0021	0	Local
addressFlavor: ftn	0x0029	0	EchoMail
addressFlavor: email	0x0039	1	E-mail
declared, no mail	0x0021	0	Local
private_mail	0x00b9	0	NetMail

0x0021 is INF_SCANNING | INF_POST throughout; the rest is INF_ECHO, INF_NETMAIL and INF_NOPUBLIC per the chart in your comment.

The generator is a throwaway script rather than a checked-in fixture, so it does not answer the other half of that thread — a fixture packet with a byte-exact test would still be the thing that keeps the layout honest over time.

[andy5995]

[andy5995]

Posted by Claude (Opus 5), an LLM made by Anthropic, at andy5995's direction.

The branch is rebased onto master now that #799 has merged, so the diff here is Blue Wave alone. The caveat on my previous comment is spent: the .INF area records described there are what this branch produces, not a local-only preview.

Six commits sit on top of the original.

Answering the review

Area kinds from addressFlavor. areaKindFor() maps the flavor onto the chart you quoted: local is neither flag, ftn and qwk are INF_ECHO under INF_NET_FIDONET, nntp and activitypub are INF_ECHO under INF_NET_INTERNET, email adds INF_NETMAIL, and private mail is INF_ECHO | INF_NETMAIL | INF_NOPUBLIC. Five tests assert bytes 77 and 79 of the area record against that table.

msglength. Comment only, as you asked. It records that the kit contradicts itself, that BlueMail counts the leading space and MultiMail reads it for free, and that "fixing" the stray trailing space truncates every message for BlueMail readers.

The closed key set. One correction worth passing on: the wildcard form does not fire. I wrote 'messageNetworks.bluewave.areas.*': { type: 'object', closedKeys: true } with the three keys beneath it first, and validating a config carrying echotags: reported nothing at all. What works for an open map's values is the shape scannerTossers.ftn_bso.ticAreas already uses -- openMap: true with a value: { closedKeys: true, children: {...} } block. With that, the same config reports unknownKey and suggests echotag. Two tests cover it, and misc/config.schema.json is regenerated.

The fixture. test/fixtures/bluewave/ holds all four members from a fully pinned build, compared byte for byte, with BLUEWAVE_FIXTURE_UPDATE=1 to regenerate deliberately. The control run matters more than the assertion: moving title one byte inside the .INF area record fails the fixture while all twenty-one offset-based tests still pass. That is the gap you identified, and it is now covered.

Also in: a warning when a pinned area number exceeds five digits, since areanum is six bytes with the NUL reserved and a truncated number joins .MIX to the wrong area.

Beyond the review

The export is now reachable. Nothing in misc/menu_templates/ pointed at either offline-mail module, so a caller on a stock board could not invoke the export at all. B on the message base menu now opens it, and the stock luciano_blocktronics MSGMNU.ANS lists b blue wave export in the free slot beside c change area, using the same bright-key-and-cyan-label sequence as its neighbours. The SAUCE fileSize is updated to match.

This is scope andy5995 and I agreed to add rather than something the review asked for, and it leaves QWK as the one offline format with no way in. Reverting it is git revert on a single commit if you would rather it were separate, or left out entirely.

Still not done: reply packets, so uses_upl_file stays zero.