Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,39 @@
# Changelog

## [2.0.2] - unreleased

### Added

- **`parallel_files`** per folder module (1 to 16, default 1): that many
files of a batch in flight at once, each its own lidi transfer, spread
over the lidi ports. `lidi.max_clients` now defaults to what the modules
can have in flight (parallel files included) plus 2.
- **`dyode check`** compares the installed `dyode-in/out.service` with
config.yaml: a unit waiting for another network card than the config's
(it would never start), the wrong command for the side, a different
config path. It also suggests more lidi ports for the box's CPUs, and
warns when an explicit `max_clients` is below what the modules need.
- Reference systemd units in `packaging/systemd/reference/` (v1 and v2,
both sides), kept identical to the generator by a test.
- The guided setup suggests the lidi port count from the CPU count and asks
`parallel_files` on the folder module's Advanced screen.
- **`dyode setup`**, the guided setup DYODE (Python) 1.5 had, back in dialog
boxes (ncurses/whiptail look, drawn by the binary itself, so it works from
the offline bundle, over ssh and on serial consoles). Plain numbered
questions when there is no usable terminal, or with `--plain`. Esc goes
back on every screen; the result is checked by DYODE's own config loader
before it is written, old files are backed up, imported files keep every
option the wizard does not ask about. New screens for the lidi ports
(suggested from the CPU count) and rate cap, warnings for the diode card
(no link, default route, wrong address), and a key step. Every module
has an Advanced screen with all the options the loader accepts for it,
each with its default and validation.
- `install.sh` asks offline or online first (in the same dialog look, when
neither flag is given on a terminal), then runs the guided setup on every
interactive install (`--no-wizard` skips it, `--import-config FILE`
starts from a file), and writes the systemd unit from the config it
produced.

## [2.0.1] - 2026-10-26

### Added
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "dyode"
version = "2.0.1"
version = "2.0.2"
edition = "2024"
rust-version = "1.85"
license = "GPL-3.0-or-later"
Expand Down
22 changes: 16 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,20 @@ and run `sudo ./install.sh --offline --side in|out` from it: see
```bash
cargo build --release # -> target/release/dyode
sudo ./install.sh --side in # on the input box
sudo ./install.sh --side out # on the output box
dyode interfaces # NICs and MAC addresses, for config.yaml
sudo $EDITOR /etc/dyode/config.yaml # SAME file on both boxes
sudo dyode new-key -c /etc/dyode/config.yaml # on ONE box; copy dyode.key to the other
sudo dyode check -c /etc/dyode/config.yaml --side in
sudo ./install.sh --side out --import-config /media/usb/config.yaml # output box
```

On a terminal, `install.sh` first asks whether to install offline or
online, then runs the guided setup (`dyode setup`): dialog boxes for the
diode network card, addresses, lidi ports, modules (each with an Advanced
screen) and the shared key. It writes `/etc/dyode/config.yaml`, checked
by DYODE's own loader, and the systemd unit is generated from it. Set up
one box, then import its config.yaml on the other. `dyode setup` can be
run again at any time; `--no-wizard` skips it, and `dyode setup --plain`
asks plain questions instead of drawing dialogs. Then:

```bash
sudo dyode check -c /etc/dyode/config.yaml --side in # also compares the installed unit
sudo systemctl enable --now dyode-in # dyode-out on the output box
sudo ./install.sh --side out --firewall-only # output box, once config.yaml is final
```
Expand All @@ -65,8 +74,9 @@ lidi parameters must match.
|---|---|
| `dyode in` / `dyode out` | run the input / output side (v1, lidi) |
| `dyode serial-in` / `serial-out` | run DYODE v2 (Modbus over serial) |
| `dyode setup` | guided setup in dialog boxes (or `--plain` questions) |
| `dyode new-key` | create the shared message key |
| `dyode check` | validate config.yaml and the key, list endpoints |
| `dyode check` | validate config.yaml, the key and the installed unit; throughput hints |
| `dyode lidi-config` | print the lidi parameters derived from config.yaml |
| `dyode interfaces` | list NICs with MAC, state and IPv4 |
| `dyode systemd-unit --side in\|out` | print the systemd unit |
Expand Down
11 changes: 9 additions & 2 deletions config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,9 @@ dyode_out:
lidi:
ports: [5000] # UDP ports through the diode; one RaptorQ
# encode/decode thread per port. More ports =
# more throughput on multi-core boxes.
# more throughput on multi-core boxes: about one
# per CPU core minus one, up to 4, sized for the
# SMALLER box ('dyode check' suggests a list).
# mtu: 1500 # of the diode link; 9000 for jumbo frames
# block: 220000 # RaptorQ block size, bytes
# repair: 5 # repair packets, % of each block (lidi adds 2
Expand All @@ -39,7 +41,8 @@ lidi:
# transfer in flight with it. Raise on a lossy
# link.
# heartbeat: 10 # seconds; the output box warns when it stops
# max_clients: auto # simultaneous transfers (default: modules + 2)
# max_clients: auto # simultaneous transfers (default: one per module,
# parallel_files per folder module, plus 2)
# mode: mmsg # native | msg | mmsg (sendmmsg/recvmmsg)
# reset_timeout: 2 # receiver: decoder reset after this much silence
# abort_timeout: 120 # receiver: drop a transfer idle this long (0 = never)
Expand Down Expand Up @@ -77,6 +80,10 @@ modules:
# optional: settle: 2 seconds a file must be unchanged before sending
# optional: rescan: 30 seconds between full rescans
# optional: bitrate: 100 Mbit/s cap for this module (default: none)
# optional: parallel_files: 1
# files of a batch sent at once, each its own
# lidi transfer (1 to 16). Raise it with
# several lidi ports, for many files.
# optional: keep_sent_hours: 24
# sent files are moved to .dyode_sent and kept
# this long instead of being deleted, because a
Expand Down
81 changes: 76 additions & 5 deletions install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@
# fetched. The whole bundle is checked against
# SHA256SUMS before anything is changed.
# --online apt for OS packages; cargo may build the binary
# (default: offline when this folder is a bundle)
# (neither given: asked on an interactive terminal,
# else offline when this folder is a bundle)
# --with LIST also install file-sharing services for the received
# files: ssh, nfs, smb (comma-separated). Packages only:
# configure the shares yourself (never share *.incoming).
Expand All @@ -28,6 +29,13 @@
# --no-tuning do not install the kernel settings
# --firewall-only only (re)apply the firewall rules from config.yaml
# --tuning-only only (re)apply the kernel settings
# --no-wizard do not run the guided setup (dyode setup), which
# otherwise runs on every interactive install; an
# existing config.yaml is then kept, or one is made
# from config.example.yaml
# --wizard run it, and say so when the terminal cannot
# --import-config F start the guided setup from this config.yaml (e.g. the
# other box's)
# --no-enable write the systemd units but do not enable/start them
# --dry-run say what would be done, change nothing
#
Expand All @@ -42,6 +50,7 @@ ROOT="$(cd "$(dirname "$0")" && pwd)"
MODE="" WITH="" SKIP_OS=0
SIDE="" VARIANT="v1" BINARY="" PREFIX="/usr/local/bin" ETC="${DYODE_ETC_DIR:-/etc/dyode}"
NO_FIREWALL=0 NO_TUNING=0 FIREWALL_ONLY=0 TUNING_ONLY=0 NO_ENABLE=0 DRY_RUN=0
WIZARD=auto IMPORT_CFG=""
SYS_ETC="${DYODE_ETC:-/etc}"

say() { printf '%s\n' "$*"; }
Expand All @@ -62,6 +71,9 @@ while [ $# -gt 0 ]; do
--firewall-only) FIREWALL_ONLY=1; shift ;;
--tuning-only) TUNING_ONLY=1; shift ;;
--no-enable) NO_ENABLE=1; shift ;;
--wizard) WIZARD=yes; shift ;;
--no-wizard) WIZARD=no; shift ;;
--import-config) IMPORT_CFG="${2:-}"; WIZARD=yes; shift 2 ;;
--dry-run) DRY_RUN=1; shift ;;
--offline) MODE=offline; shift ;;
--online) MODE=online; shift ;;
Expand Down Expand Up @@ -109,8 +121,44 @@ SUPPORTED="Ubuntu 24.04 and 26.04, Debian 12 and 13 / Raspberry Pi OS bookworm a
is_bundle() { [ -f "$ROOT/SHA256SUMS" ] && [ -d "$ROOT/bin" ]; }
debs_present() { [ -n "$OS_ID" ] && ls "$DEB_DIR"/*.deb >/dev/null 2>&1; }

interactive() { [ -t 0 ] && [ -t 1 ]; }

# Offline or online, asked before anything is installed: in a dialog box
# by the dyode binary when one is at hand (the bundle's, a local build),
# else as a plain question.
ask_mode() {
local def=online bin="" c ans rc=0
is_bundle && def=offline
for c in "$BINARY" "$BUNDLE_BIN" "$ROOT/target/release/dyode" "$PREFIX/dyode"; do
if [ -n "$c" ] && [ -x "$c" ] && "$c" version >/dev/null 2>&1; then bin="$c"; break; fi
done
if [ -n "$bin" ]; then
ans="$(mktemp)"
# shellcheck disable=SC2046
"$bin" setup --ask-install-mode "$ans" --default "$def" $(is_bundle && echo --bundle) || rc=$?
MODE="$(cat "$ans" 2>/dev/null || true)"; rm -f "$ans"
[ "$rc" -eq 3 ] && die "installation cancelled; nothing was changed"
case "$MODE" in offline|online) return 0 ;; esac
fi
local hint="this folder is not an offline bundle"
is_bundle && hint="this folder is an offline bundle"
while :; do
printf 'Install offline (only this folder) or online (apt, cargo)? [%s] (%s): ' "$def" "$hint"
read -r ans || die "no answer; nothing was changed"
case "${ans:-$def}" in
offline|off|o|1) MODE=offline; return 0 ;;
online|on|n|2) MODE=online; return 0 ;;
q|quit) die "installation cancelled; nothing was changed" ;;
esac
done
}

if [ -z "$MODE" ]; then
if is_bundle; then MODE=offline; else MODE=online; fi
if interactive && [ "$FIREWALL_ONLY" -eq 0 ] && [ "$TUNING_ONLY" -eq 0 ]; then
ask_mode
elif is_bundle; then MODE=offline
else MODE=online
fi
fi

# -------------------------------------------------------- OS packages
Expand Down Expand Up @@ -346,12 +394,31 @@ say " $PREFIX/dyode ($("$DYODE" version 2>/dev/null || echo '?'))"

step "configuration in $ETC"
run mkdir -p "$ETC"
if [ -f "$ETC/config.yaml" ]; then
run_wizard() {
[ "$WIZARD" = no ] && return 1
if ! interactive; then
[ "$WIZARD" = yes ] && warn "--wizard needs an interactive terminal; skipping the guided setup"
return 1
fi
local args=(setup --side "$SIDE" --variant "$VARIANT" --etc "$ETC")
[ -n "$IMPORT_CFG" ] && args+=(--import-config "$IMPORT_CFG")
[ "$DRY_RUN" -eq 1 ] && args+=(--dry-run)
# The installer writes the systemd unit itself, from what setup wrote.
if DYODE_INSTALLER=1 "$DYODE" "${args[@]}"; then
return 0
fi
warn "guided setup left before writing; continuing without it"
return 1
}
if run_wizard; then
if [ "$DRY_RUN" -eq 1 ]; then say " guided setup done (dry run: nothing written)"; else say " $ETC/config.yaml written by the guided setup"; fi
elif [ -f "$ETC/config.yaml" ]; then
say " keeping $ETC/config.yaml"
else
run install -m 0640 "$ROOT/config.example.yaml" "$ETC/config.yaml"
say " wrote $ETC/config.yaml from config.example.yaml: EDIT IT (addresses, MACs, modules),"
say " then copy the SAME file to the other box. 'dyode interfaces' lists NICs and MACs."
say " then copy the SAME file to the other box. 'dyode interfaces' lists NICs and MACs,"
say " or run the guided setup: sudo dyode setup --side $SIDE"
fi
if [ "$VARIANT" = v1 ] && [ ! -f "$ETC/dyode.key" ]; then
say " no $ETC/dyode.key yet. On ONE box: sudo dyode new-key -c $ETC/config.yaml"
Expand All @@ -362,7 +429,11 @@ step "systemd"
case "$VARIANT/$SIDE" in v1/in) CMD=in ;; v1/out) CMD=out ;; v2/in) CMD=serial-in ;; v2/out) CMD=serial-out ;; esac
UNIT="$(mktemp)"
SERIAL_FLAG=""; [ "$VARIANT" = v2 ] && SERIAL_FLAG="--serial"
CFG_FOR_UNIT="$ETC/config.yaml"; [ -f "$CFG_FOR_UNIT" ] || CFG_FOR_UNIT="$ROOT/config.example.yaml"
CFG_FOR_UNIT="$ETC/config.yaml"
if [ ! -f "$CFG_FOR_UNIT" ]; then
CFG_FOR_UNIT="$ROOT/config.example.yaml"
[ "$VARIANT" = v1 ] && warn "the unit waits for the example config's NIC; run 'sudo dyode setup' or re-run this installer once config.yaml is right"
fi
# shellcheck disable=SC2086
"$DYODE" systemd-unit --side "$SIDE" $SERIAL_FLAG --workdir "$ETC" --binary "$PREFIX/dyode" -c "$CFG_FOR_UNIT" | sed '1d' > "$UNIT"
if [ "$DRY_RUN" -eq 1 ]; then
Expand Down
25 changes: 25 additions & 0 deletions packaging/systemd/reference/dyode-in-serial.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Reference only: install.sh and 'dyode setup' write the real
# /etc/systemd/system/dyode-in.service from your config.yaml (its diode
# NIC), with: dyode systemd-unit --side in --serial --workdir /etc/dyode --binary /usr/local/bin/dyode
# This one is for config.example.yaml (input NIC eth0, output NIC eth1).
[Unit]
Description=DYODE input (sending) side (serial-in)
After=network-online.target
Wants=network-online.target
# A diode may sit behind a network that takes long to come back: never give up.
StartLimitIntervalSec=0

[Service]
WorkingDirectory=/etc/dyode
ExecStart=/usr/local/bin/dyode serial-in --config /etc/dyode/config.yaml
Restart=always
RestartSec=5
StandardInput=null
# Creates /var/log/dyode-transfer with the right mode before start.
LogsDirectory=dyode-transfer
LogsDirectoryMode=0750
# root is needed for the static ARP entry.
User=root

[Install]
WantedBy=multi-user.target
30 changes: 30 additions & 0 deletions packaging/systemd/reference/dyode-in.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Reference only: install.sh and 'dyode setup' write the real
# /etc/systemd/system/dyode-in.service from your config.yaml (its diode
# NIC), with: dyode systemd-unit --side in --workdir /etc/dyode --binary /usr/local/bin/dyode
# This one is for config.example.yaml (input NIC eth0, output NIC eth1).
[Unit]
Description=DYODE input (sending) side (in)
After=network-online.target
Wants=network-online.target
# Stop and restart with the diode NIC, and do not start without it.
After=sys-subsystem-net-devices-eth0.device
BindsTo=sys-subsystem-net-devices-eth0.device
# A diode may sit behind a network that takes long to come back: never give up.
StartLimitIntervalSec=0

[Service]
WorkingDirectory=/etc/dyode
ExecStartPre=/bin/sh -c 'for i in $(seq 1 60); do ip -4 addr show dev eth0 2>/dev/null | grep -q "inet " && exit 0; sleep 1; done; echo "eth0 still has no IPv4 address after 60s" >&2; exit 1'
ExecStart=/usr/local/bin/dyode in --config /etc/dyode/config.yaml
Restart=always
RestartSec=5
StandardInput=null
# Creates /var/log/dyode-transfer with the right mode before start.
LogsDirectory=dyode-transfer
LogsDirectoryMode=0750
# root is needed for the static ARP entry.
User=root

[Install]
WantedBy=multi-user.target
WantedBy=sys-subsystem-net-devices-eth0.device
25 changes: 25 additions & 0 deletions packaging/systemd/reference/dyode-out-serial.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Reference only: install.sh and 'dyode setup' write the real
# /etc/systemd/system/dyode-out.service from your config.yaml (its diode
# NIC), with: dyode systemd-unit --side out --serial --workdir /etc/dyode --binary /usr/local/bin/dyode
# This one is for config.example.yaml (input NIC eth0, output NIC eth1).
[Unit]
Description=DYODE output (receiving) side (serial-out)
After=network-online.target
Wants=network-online.target
# A diode may sit behind a network that takes long to come back: never give up.
StartLimitIntervalSec=0

[Service]
WorkingDirectory=/etc/dyode
ExecStart=/usr/local/bin/dyode serial-out --config /etc/dyode/config.yaml
Restart=always
RestartSec=5
StandardInput=null
# Creates /var/log/dyode-transfer with the right mode before start.
LogsDirectory=dyode-transfer
LogsDirectoryMode=0750
# root is needed for binding the Modbus server to port 502.
User=root

[Install]
WantedBy=multi-user.target
30 changes: 30 additions & 0 deletions packaging/systemd/reference/dyode-out.service
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Reference only: install.sh and 'dyode setup' write the real
# /etc/systemd/system/dyode-out.service from your config.yaml (its diode
# NIC), with: dyode systemd-unit --side out --workdir /etc/dyode --binary /usr/local/bin/dyode
# This one is for config.example.yaml (input NIC eth0, output NIC eth1).
[Unit]
Description=DYODE output (receiving) side (out)
After=network-online.target
Wants=network-online.target
# Stop and restart with the diode NIC, and do not start without it.
After=sys-subsystem-net-devices-eth1.device
BindsTo=sys-subsystem-net-devices-eth1.device
# A diode may sit behind a network that takes long to come back: never give up.
StartLimitIntervalSec=0

[Service]
WorkingDirectory=/etc/dyode
ExecStartPre=/bin/sh -c 'for i in $(seq 1 60); do ip -4 addr show dev eth1 2>/dev/null | grep -q "inet " && exit 0; sleep 1; done; echo "eth1 still has no IPv4 address after 60s" >&2; exit 1'
ExecStart=/usr/local/bin/dyode out --config /etc/dyode/config.yaml
Restart=always
RestartSec=5
StandardInput=null
# Creates /var/log/dyode-transfer with the right mode before start.
LogsDirectory=dyode-transfer
LogsDirectoryMode=0750
# root is needed for binding the Modbus server to port 502.
User=root

[Install]
WantedBy=multi-user.target
WantedBy=sys-subsystem-net-devices-eth1.device
Loading
Loading