diff --git a/CHANGELOG.md b/CHANGELOG.md index 4c71d12dc..07747f595 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,51 +11,15 @@ layout) per [`STABILITY_POLICY.md`](STABILITY_POLICY.md). ## [Unreleased] -Bound for **0.6.1**, a patch release on the 0.6 line. This is an index: every -item below is (or will be) written up in full in the in-development -[`docs/release-notes/0.6.1-pre.md`](docs/release-notes/0.6.1-pre.md). - -### Fixed - -- A watch add the streaming server refused (over the token's watch quota, over the per-add rate limit, over a per-connection cap, without `stream:watch`, or malformed) was logged on the node and dropped, so a client could believe it was watching addresses it was not. The server now answers it on the `Watch` stream with a `WatchAddRejected` event (`watch_add_rejected` on WebSocket) naming the reason and the refused items, as the streaming docs promised; both SDKs surface it. `ResilientWatch` re-sends a rate-limited add once the limit allows, within its backoff budget, and stops re-registering other refused items on reconnect. A silent-payment add that only updates the labels of targets already watched no longer spends a rate-limit token. -- A block that reached the tip through the stored-tail drain, such as one that arrived before its parent, was connected without a chain event: `-blocknotify` did not run for it, it was not announced to peers, and Electrum, Esplora SSE, streaming and ZMQ `hashblock` subscribers never heard of it, so a wallet saw a confirmation in it only at the next block. The drain now reports each block it connects as every other connect at the tip does (#900). -- During initial sync, a block held by one slow peer is taken back and asked of another again. The network loop's periodic pass, which does that after 15 seconds, hardly ran while blocks were arriving: 3 or 4 times in two minutes of a fresh mainnet sync instead of every half second. The block the sync needed next could wait up to a minute on one peer while tens of thousands of later blocks sat downloaded behind it (#909). -- Initial sync no longer logs `Connector stuck waiting for block data` at WARN for nearly every block. The warning now fires only once the connector has waited 20 seconds on one block, longer than the scheduler takes to ask another peer for it, then once a minute, and reports how long it has waited (#904). -- An interrupted full `-reindex` now continues from the height it had flushed, on any start, instead of starting over: satd scans the block files again, checks the blocks it had replayed against them, and connects the rest, downloading nothing and writing no block file. 0.6.0 refused every start but `--reindex`, which then replayed from genesis, so on a mainnet node every stop, reboot or settings change during the days-long replay threw it away. A reindex that cannot be continued (one 0.6.0 started, a wipe that may not have finished, blocks that no longer match the files) runs again from genesis on any start (#906). -- A `-reindex` reads only the block headers when it scans the block files, at most 4 KiB per block, where it read every file whole. On mainnet that read the whole chain, about 700 GB, before the first block connected, and a reindex continued after a stop read it again on every start (#916). -- SIGTERM or SIGINT during `-reindex`, `-reindex-chainstate` or `-upgradechainstate` now stops the replay at once: satd abandons the block it is connecting, flushes what it had connected, keeps the rebuild marker and exits 0, within `-maxshutdownsecs`. There was no handler until startup finished, so the signal killed the replay mid-batch, or, with satd as PID 1 of its PID namespace (a container with no init), was dropped until the supervisor's SIGKILL. Anywhere else in startup a stop signal still ends satd at once, now as PID 1 too. A shutdown also logs each wait, for the block connector and the UTXO cache flush, before it starts (#907). -- A full `-reindex` no longer appends a second copy of the genesis block to the last block file. It indexes the genesis record `blk00000.dat` begins with, so the reindex writes nothing to the block files, which on a switch from Bitcoin Core are Core's (#931). -- StartOS package: the service log now shows satd's INFO lines (sync progress, a reindex's phases, the steps of a shutdown). StartOS starts every service with `RUST_LOG=warn`, which satd honors, so the log carried warnings only. -- `getrawaddrman` could report an address in a bucket Core's formulas would never give it. When two addresses derived the same slot, the second moved to the next free slot of the whole table, which from a bucket's last position is the next bucket's first. It now moves within its own bucket, as Core keeps every address in the bucket its formula gives; only a full bucket spills over. -- Signature operations are counted exactly as Bitcoin Core counts them, for the block sigop limit, mempool policy and block templates; the old counter scored an `OP_CHECKMULTISIG` after another signature opcode in P2SH and P2WSH scripts lower than Core. Parents admitted by `submitpackage` without a fee check carry their real sigop cost (#937). -- The Rust script engine treats 0x80, 0x81 and 0x83–0x86 as `OP_SUCCESSx` in tapscript, as Core does, instead of failing the leaf (`-consensus=rust`, `cpp-shadow`, `satd-chainstate-repair`) (#933). -- A block body that fails the context-free block checks no longer marks the block invalid, and an empty body is checked against the header's merkle root first (#936). -- On signet the signed coinbase commitment is rebuilt exactly as Core does; a missing block in a median-time-past window stops connecting with a storage error; malformed `nBits` count as zero work (#948). -- satd stops, as Core does, when it cannot read a coin from the chainstate, instead of treating the coin as spent and recording a valid block as invalid (#945). -- Header sync continues from the last header a peer sent, so a node on a competing best-header branch keeps syncing; the retarget start is found on the block's own branch, and targets above the proof-of-work limit are refused (#954). -- A block that fails to connect from storage is marked invalid instead of being retried by the IBD connector; the `headers` tip leaves the failed branch, and a chain with more work below it is activated at once (#957). -- A block passed to `submitblock` before its parent's block is now indexed as a header, as Core indexes it: it becomes the best header (`headers` counts it) and connects as soon as its parent does, where it stayed unconnected until another block arrived (#922). -- IBD no longer stalls with a full `-maxahead` window; re-sent blocks do not count against it, local write failures do not penalize the sender, and block downloads go only to peers that advertise block service (#953). -- A block that arrives before its parent is kept only if it was requested and within fixed bounds (4 per peer, 32 MB in all, 5 minutes) (#941). -- Core's P2P message size and count limits apply, block replies are queued about 1 MB per peer, a peer that takes no data for 20 minutes is disconnected, user agents are capped at 256 bytes and sanitized, and refused BIP 157 requests disconnect (#939). -- Automatic outbound connections take at most one peer per network group (/16, /32, or ASN with `-asmap`), as Core does; the extra outbound peers a node opens during initial sync (up to 64) are disconnected one every 10 seconds once it has caught up, until 8 remain (#894, #895). -- `noban` and manual peers are never punished for misbehaviour; local and onion-inbound peers are disconnected but not banned, so 127.0.0.1 is no longer banned; `getpeerinfo` reports onion-inbound peers as `onion` (#944). -- Address announcements are rate-limited per peer as in Core (`addr_processed`, `addr_rate_limited` in `getpeerinfo`); dials in flight count against the outbound limit; manual connections sit outside it; `setban` of an onion host disconnects it (#952). -- A full mempool checks an incoming transaction's scripts before evicting for it and never evicts its parent; eviction scales with what it removes; lowering `maxmempool` on reload trims at once (#942). -- `getmempoolentry`, `getrawmempool`, `getmempoolinfo`, `gettxout` and `sendrawtransaction` answer as Core does; a restart keeps entry times and loads a large mempool in linear time (#946). -- A restart re-admits the whole saved mempool: `mempool.dat` is written parents first, as Core writes it, and a file saved by 0.6.0 is reordered on load, where a child saved in the same second as its parent could be tried first and dropped with all its descendants (#949). -- RPC numbers use Core's formatting (16 significant digits, 8-decimal fee amounts); `submitblock` answers `duplicate-invalid` and throws -22/-25; `getblockfilter` uses Core's errors; a failed RPC password is answered after 250 ms (#955). -- Electrum scripthash status hashes the history in `get_history` order, which stops Electrum desktop reconnecting in a loop on addresses with pending transactions (#938). -- Esplora honours `?after_txid=` paging, answers 404/400 for out-of-range or misaligned block-txs pages, drops empty `witness` arrays, and `/fee-estimates` omits targets with no estimate (#943). -- Long Electrum and Esplora address-history reads run off the API runtime's workers, and `--electrumrequesttimeout` and `--esplorarequesttimeout` take effect (#947). -- The address, spend and filter indexes do not serve during or after an unfinished `backfillindex`; a reorg during a backfill no longer deletes live rows; reads during a flush wait for it (#958). -- A `backfillindex blockfilter` run no longer leaves a broken BIP 157 filter header chain. Rows the live connect path had not yet flushed for the same heights replaced the backfill's at the next flush, so `getblockfilter`, `getcfheaders` and `getcfcheckpt` served headers chained off zeros from that height on. On first start, 0.6.1 checks the stored header chain once on any datadir a filter backfill has written and rewrites the headers that differ. -- `bitcoin.conf` treats `#` anywhere as a comment start and refuses an `rpcpassword` containing `#`, as Core does; trace logs no longer include RPC headers or parameters; the Tor control cookie read is bounded (#940). +Nothing yet — 0.6.1 was cut on 2026-10-15. Add a one-line bullet here for every +user-facing change, and the full write-up to the next +`docs/release-notes/-pre.md`. ## Releases | Version | Date | Notes | |---|---|---| +| [0.6.1](docs/release-notes/0.6.1.md) | 2026-10-15 | Fixes for 0.6.0. An interrupted `-reindex` continues where it stopped on any start, a stop signal ends a rebuild cleanly (also as PID 1), and the reindex scan reads block headers only. Initial sync no longer waits on one slow peer or stalls with a full download window. Sigop counting, the tapscript `OP_SUCCESS` set, header sync, signet commitments and invalid-block handling match Bitcoin Core. Core's P2P message limits, address rate limits, one outbound peer per network group, and no bans for `noban`, manual, local or onion-inbound peers. Mempool eviction checks scripts first, a restart keeps the whole mempool, and mempool and other RPC answers match Core. Electrum status hashes follow history order, Esplora pages with `after_txid`, a refused streaming watch add is reported to the client, and the filter index repairs a header chain a 0.6.0 backfill broke. `rustls` 0.23.46 (RUSTSEC-2026-0346 to 0349, none affecting satd's configuration). No reindex; an `rpcpassword` containing `#` is now refused. | | [0.6.0](docs/release-notes/0.6.0.md) | 2026-10-05 | Solo mining, silent-payment sending, and half-size indexes. A **Stratum V1 and V2 solo-mining server** in the node (TLS for V1, Noise for V2, Job Declaration, per-miner statistics) lets a miner mine to its own address with no pool and no proxy, from block templates that now match Core's: the right difficulty at a retarget, the sigop limit, CPFP package selection, `-blockmaxweight`, and a validity check before a miner sees one. **Silent-payment sending (BIP 375)** over PSBT version 2, with `analyzepsbt` and `finalizepsbt` verifying every silent-payment output before it can be broadcast; the node stays keyless. Every index is keyed on transaction ordinals instead of txids, which **halves a fully indexed chainstate** (805 GB to 388 GB on mainnet). Faster block relay with BIP 152 high-bandwidth compact blocks and BIP 339 wtxid relay, sync, reorg and reindex robustness fixes, and another round of Core parity from Core's functional tests. Installable from Umbrel's community store or a signed appliance image for amd64 and arm64. **One `-reindex-chainstate` is required** (chainstate schema 3 to 7; app-store installs run it by themselves, pruned nodes must resync, and there is no downgrade). A configured `-rpcallowip` now also gates the TLS JSON-RPC listener, the HTTP listeners speak HTTP/1.1 only, and Esplora tokens that broadcast need `rpc:submit`. | | [0.5.2](docs/release-notes/0.5.2.md) | 2026-09-11 | Deployment and Bitcoin Core parity. Three new ways to run satd — a docker-compose **reference stack** with TLS everywhere and LND/CLN/RTL/Cashu/BTCPay overlays, a bootable **appliance image**, and **Umbrel and StartOS packages** — each installed and driven on a real server; `sat-cli`/`sat-tui` reach a TLS RPC listener; the container image ships `sat-tui` and a `HEALTHCHECK` and is **136 MB instead of 670 MB**. MCP is reachable by hostname (`-mcpallowedhost`, #734). The defects filed from the 0.5.1 functional-test review (#660–#673) and the ones Core's suite went on to find, closed in Core's direction: a mistyped RPC argument no longer discards the ones after it (#672), surplus arguments are refused (#688), `createrawtransaction`/`createpsbt` check the address network and parse amounts exactly (#689), mempool policy at Core parity — dust thresholds, feerate-diagram RBF, ephemeral dust, `submitpackage`/`testmempoolaccept` shapes (#660, #661, #665, #673), `getblocktemplate` proposal mode validates like `submitblock` (#663), more than a dozen RPC fields that were constants now come from the node (#667, #702), `-connect=0` no longer dials the address zero and a pinned node stops listening (#690), and two **security** fixes — `-whitelist` no longer reaches Tor or outbound peers (#701) and a proxied node no longer resolves peer hostnames locally (#668). An **IBD wedge** on fork-heavy networks, where a competing header could permanently replace the next block to fetch, is fixed and a damaged datadir heals on the first restart (#738). Drop-in binary upgrade, no reindex; `banlist.json` moves to Core's format, so delete it before any downgrade. | | [0.5.1](docs/release-notes/0.5.1.md) | 2026-09-04 | A Bitcoin Core compatibility release. satd now runs **Core's own functional test suite**, unmodified and pinned to a Core release, with all 264 test files inventoried as run-or-skip-with-a-reason; the set gates every pull request and found twelve defects in shipped behaviour — an RPC `Content-Type` Core-derived clients reject outright, a startup listener answering "no such method" where Core answers "warming up", a `bitcoin.conf` fee-rate value silently discarded, a panic on any unparseable `-bind` (and no IPv6 bind at all), mempool chain limits enforcing a limit Core v31 no longer has, and an oversized locator banning the peer where Core only disconnects. The mempool event broadcast now holds a whole block's confirmations — its 1024-slot ring was smaller than a single mainnet block, so every subscriber silently lost roughly 37% of each block's burst (#682). Silent payments reach wallets that exist today via `blockchain.tweaks.subscribe` on the Electrum server, plus `tweak_unspent_only` cut-through and spendable silent-payment outputs (#609). **Breaking:** `getindexinfo` is now Core's method — satd's richer view moved to `getsatdindexinfo`; `uacomment` is now validated and an unparseable `minrelaytxfee`/`dustrelayfee` now stops the node. Drop-in binary upgrade, no reindex. | @@ -68,4 +32,4 @@ item below is (or will be) written up in full in the in-development | [0.2.0](docs/release-notes/0.2.0.md) | 2026-05-27 | BIP 324 v2 transport, native TLS, client-side PSBT signing, Core CLI/config-compat gap closed, AssumeUTXO fast-start. **Breaking storage cleanup** — see notes. | | [0.1.0](docs/release-notes/0.1.0.md) | 2026-05-08 | First public release: mainnet-validated node, native Esplora/Electrum/cfilters, Core-compatible RPC/CLI, signed reproducible builds. | -[Unreleased]: https://github.com/epochbtc/satd/compare/v0.6.0...HEAD +[Unreleased]: https://github.com/epochbtc/satd/compare/v0.6.1...HEAD diff --git a/Cargo.lock b/Cargo.lock index 42a92c9b2..e5dff213c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1038,7 +1038,7 @@ dependencies = [ [[package]] name = "consensus" -version = "0.6.0" +version = "0.6.1" dependencies = [ "bitcoin", "bitcoinconsensus", @@ -1481,7 +1481,7 @@ dependencies = [ [[package]] name = "electrum-proto" -version = "0.6.0" +version = "0.6.1" dependencies = [ "async-trait", "bitcoin", @@ -1552,7 +1552,7 @@ dependencies = [ [[package]] name = "esplora-handlers" -version = "0.6.0" +version = "0.6.1" dependencies = [ "axum 0.8.9", "axum-test", @@ -2959,7 +2959,7 @@ dependencies = [ [[package]] name = "node" -version = "0.6.0" +version = "0.6.1" dependencies = [ "arc-swap", "async-trait", @@ -3014,7 +3014,7 @@ dependencies = [ [[package]] name = "node-filter-index" -version = "0.6.0" +version = "0.6.1" dependencies = [ "bitcoin", "serde", @@ -3023,7 +3023,7 @@ dependencies = [ [[package]] name = "node-index" -version = "0.6.0" +version = "0.6.1" dependencies = [ "bitcoin", "parking_lot", @@ -3034,7 +3034,7 @@ dependencies = [ [[package]] name = "node-sp-index" -version = "0.6.0" +version = "0.6.1" dependencies = [ "bitcoin", "hex", @@ -4464,7 +4464,7 @@ dependencies = [ [[package]] name = "sat-cli" -version = "0.6.0" +version = "0.6.1" dependencies = [ "base64 0.22.1", "bitcoin", @@ -4484,7 +4484,7 @@ dependencies = [ [[package]] name = "sat-tui" -version = "0.6.0" +version = "0.6.1" dependencies = [ "base64 0.22.1", "clap", @@ -4500,7 +4500,7 @@ dependencies = [ [[package]] name = "satd" -version = "0.6.0" +version = "0.6.1" dependencies = [ "axum 0.8.9", "base64 0.22.1", @@ -4550,7 +4550,7 @@ dependencies = [ [[package]] name = "satd-alert" -version = "0.6.0" +version = "0.6.1" dependencies = [ "hex", "hmac", @@ -4566,7 +4566,7 @@ dependencies = [ [[package]] name = "satd-auth" -version = "0.6.0" +version = "0.6.1" dependencies = [ "base64 0.22.1", "dashmap", @@ -4583,7 +4583,7 @@ dependencies = [ [[package]] name = "satd-events" -version = "0.6.0" +version = "0.6.1" dependencies = [ "async-trait", "axum 0.8.9", @@ -4612,7 +4612,7 @@ dependencies = [ [[package]] name = "satd-events-client" -version = "0.6.0" +version = "0.6.1" dependencies = [ "bitcoin", "prost", @@ -4629,7 +4629,7 @@ dependencies = [ [[package]] name = "satd-events-proto" -version = "0.6.0" +version = "0.6.1" dependencies = [ "prost", "protoc-bin-vendored", @@ -4639,7 +4639,7 @@ dependencies = [ [[package]] name = "satd-mcp" -version = "0.6.0" +version = "0.6.1" dependencies = [ "bitcoin", "bytes", @@ -4664,7 +4664,7 @@ dependencies = [ [[package]] name = "satd-policy" -version = "0.6.0" +version = "0.6.1" dependencies = [ "bitcoin", "proptest", @@ -4673,7 +4673,7 @@ dependencies = [ [[package]] name = "satd-psbt" -version = "0.6.0" +version = "0.6.1" dependencies = [ "base64 0.22.1", "bitcoin", @@ -5425,7 +5425,7 @@ checksum = "1f3ccbac311fea05f86f61904b462b55fb3df8837a366dfc601a0161d0532f20" [[package]] name = "tls-config" -version = "0.6.0" +version = "0.6.1" dependencies = [ "rcgen", "reqwest", diff --git a/Cargo.toml b/Cargo.toml index 06d2a24ff..84918caab 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -17,7 +17,7 @@ debug = "line-tables-only" [workspace.package] authors = ["Benjamen Keroack ", "20 Gauge Software, Inc.", "The satd developers"] -version = "0.6.0" +version = "0.6.1" edition = "2024" license = "MIT" repository = "https://github.com/epochbtc/satd" diff --git a/clients/go/version.go b/clients/go/version.go index be4803c44..155bac8c5 100644 --- a/clients/go/version.go +++ b/clients/go/version.go @@ -4,7 +4,7 @@ package satdevents // released with: the module is tagged clients/go/vX.Y.Z at the node's vX.Y.Z // commit, even when no Go code changed. TestVersionMatchesWorkspace pins it to // the workspace Cargo.toml, so a version bump that misses this file fails CI. -const Version = "0.6.0" +const Version = "0.6.1" // schemaVersion is the NodeEvent.schema_version this SDK decodes. It mirrors // SCHEMA_VERSION in the satd node; the two are bumped together. diff --git a/contrib/appliance/README.md b/contrib/appliance/README.md index 4f6747cf0..5b5f2f426 100644 --- a/contrib/appliance/README.md +++ b/contrib/appliance/README.md @@ -61,7 +61,7 @@ contrib/appliance/build-in-docker.sh --flavor core --out contrib/appliance/out # From a published, signed release instead. contrib/appliance/build-in-docker.sh --flavor desktop \ - --satd-source release --satd-version 0.6.0 --out out/ + --satd-source release --satd-version 0.6.1 --out out/ # With the tools already on the host. sudo contrib/appliance/build.sh --flavor core --out out/ diff --git a/contrib/warnet/Dockerfile b/contrib/warnet/Dockerfile index 01aaf1d4e..4fd28d8d0 100644 --- a/contrib/warnet/Dockerfile +++ b/contrib/warnet/Dockerfile @@ -12,7 +12,7 @@ # ghcr.io/epochbtc/satd for anything else. # # Warnet only renders image tags of the form X.Y[.Z][-suffix]; tag builds of -# this image `-warnet` (for example 0.6.0-warnet0). +# this image `-warnet` (for example 0.6.1-warnet0). ARG SATD_IMAGE=ghcr.io/epochbtc/satd:latest FROM ${SATD_IMAGE} diff --git a/contrib/warnet/README.md b/contrib/warnet/README.md index 4f954220e..547215452 100644 --- a/contrib/warnet/README.md +++ b/contrib/warnet/README.md @@ -37,8 +37,8 @@ Warnet renders only image tags shaped like `X.Y[.Z][-suffix]`. Tag the image `-warnet`: ```sh -docker build --build-arg SATD_IMAGE=ghcr.io/epochbtc/satd:0.6.0 \ - -t satd-warnet:0.6.0-warnet0 contrib/warnet +docker build --build-arg SATD_IMAGE=ghcr.io/epochbtc/satd:0.6.1 \ + -t satd-warnet:0.6.1-warnet0 contrib/warnet ``` `SATD_IMAGE` defaults to `ghcr.io/epochbtc/satd:latest`. Any satd image @@ -47,7 +47,7 @@ works, including one built from this repository with `docker build -t satd:dev . Load the image into the cluster. With [kind](https://kind.sigs.k8s.io/): ```sh -kind load docker-image satd-warnet:0.6.0-warnet0 --name +kind load docker-image satd-warnet:0.6.1-warnet0 --name ``` The examples set `pullPolicy: Never`, so a tank whose image is missing fails diff --git a/contrib/warnet/examples/all-satd/node-defaults.yaml b/contrib/warnet/examples/all-satd/node-defaults.yaml index d625baf42..1b28655ad 100644 --- a/contrib/warnet/examples/all-satd/node-defaults.yaml +++ b/contrib/warnet/examples/all-satd/node-defaults.yaml @@ -3,5 +3,5 @@ # missing image fail loudly instead of pulling something else. image: repository: satd-warnet - tag: "0.6.0-warnet0" + tag: "0.6.1-warnet0" pullPolicy: Never diff --git a/contrib/warnet/examples/core-satd/network.yaml b/contrib/warnet/examples/core-satd/network.yaml index cad59d86b..7f9d290d3 100644 --- a/contrib/warnet/examples/core-satd/network.yaml +++ b/contrib/warnet/examples/core-satd/network.yaml @@ -7,7 +7,7 @@ nodes: - name: tank-0001 image: repository: satd-warnet - tag: "0.6.0-warnet0" + tag: "0.6.1-warnet0" pullPolicy: Never addnode: - tank-0002 @@ -17,7 +17,7 @@ nodes: - name: tank-0003 image: repository: satd-warnet - tag: "0.6.0-warnet0" + tag: "0.6.1-warnet0" pullPolicy: Never addnode: - tank-0000 diff --git a/contrib/warnet/tests/render-confs.sh b/contrib/warnet/tests/render-confs.sh index 5467b6618..8847c2bdf 100755 --- a/contrib/warnet/tests/render-confs.sh +++ b/contrib/warnet/tests/render-confs.sh @@ -14,7 +14,7 @@ CHART="${WARNET_CHART:-$(python3 -c 'from warnet.constants import BITCOIN_CHART_ tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT render() { # [addnode] - printf 'image: {repository: satd-warnet, tag: "0.6.0-warnet0"}\n' > "$tmp/values.yaml" + printf 'image: {repository: satd-warnet, tag: "0.6.1-warnet0"}\n' > "$tmp/values.yaml" [ -z "${2:-}" ] || printf 'addnode: [%s]\n' "$2" >> "$tmp/values.yaml" helm template "$1" "$CHART" -f "$tmp/values.yaml" | python3 -c ' import sys, yaml diff --git a/docs/manual/src/electrum.md b/docs/manual/src/electrum.md index 4ab64c7c0..b05632647 100644 --- a/docs/manual/src/electrum.md +++ b/docs/manual/src/electrum.md @@ -249,7 +249,7 @@ Override it if you would rather not advertise the token, or need a different one for another client: ```ini -electrumservername=satd/0.6.0 +electrumservername=satd/0.6.1 ``` Keep an override from *beginning* with `electrs`. Several clients test that diff --git a/docs/manual/src/installing.md b/docs/manual/src/installing.md index 2c7d9aca8..7976742af 100644 --- a/docs/manual/src/installing.md +++ b/docs/manual/src/installing.md @@ -104,7 +104,7 @@ Images are attached to the [GitHub release](https://github.com/epochbtc/satd/rel alongside the tarballs, from 0.6.0 on, and signed with the same minisign key: ```sh -ver=0.6.0 +ver=0.6.1 flavor=core # or desktop arch=amd64 # or arm64, for Apple Silicon and arm64 servers base="https://github.com/epochbtc/satd/releases/download/v$ver" diff --git a/docs/release-notes/0.6.1-pre.md b/docs/release-notes/0.6.1-pre.md deleted file mode 100644 index cb3a3e4f3..000000000 --- a/docs/release-notes/0.6.1-pre.md +++ /dev/null @@ -1,682 +0,0 @@ -# satd 0.6.1 - -> Pre-release — in progress · [Changelog](../../CHANGELOG.md) - -satd 0.6.1 is a patch release on the 0.6 line, cut from the `release/0.6` -branch. It carries fixes for 0.6.0. - -This file accumulates entries as changes land; it is not yet a cut release. - -## Highlights - -- **A refused watch add is reported to the client.** The streaming server used - to drop a watch add it refused without telling the client; it now answers - with a `WatchAddRejected` event. See - [below](#a-refused-watch-add-is-reported-to-the-client). -- **An interrupted `-reindex` continues where it stopped.** A full reindex - that is stopped, killed or loses power continues on the next start, any - start, from the height it had flushed, instead of starting over. See - [below](#an-interrupted--reindex-continues-where-it-stopped). -- **A stop signal stops a rebuild cleanly.** SIGTERM or SIGINT during - `-reindex`, `-reindex-chainstate` or `-upgradechainstate` now stops the - replay at the last block it connected, flushed, and satd exits 0, including as PID 1 - of a container with no init. See - [below](#a-stop-signal-during-a-rebuild-stops-it-cleanly). - -- **Sigops are counted as Bitcoin Core counts them.** satd's counter scored - some P2SH and P2WSH scripts lower than Core for the block sigop limit, - mempool policy and block templates. See - [below](#sigops-are-counted-as-bitcoin-core-counts-them). -- **Header sync continues from a peer's last header.** A node whose best - header chain was on a competing branch could stop syncing headers from a - peer whose chain forked below it. See - [below](#header-sync-continues-from-a-peers-last-header). -- **Electrum desktop no longer reconnects in a loop.** The scripthash status - now hashes the history in the order `get_history` returns it, so an address - with a pending transaction shows it. See - [below](#electrum-status-hashes-follow-history-order). -- **A full mempool checks scripts before it evicts.** An incoming - transaction's scripts are checked before anything is evicted for it, and - eviction no longer scales with the size of the pool. See - [below](#a-full-mempool-checks-scripts-before-it-evicts). -- **Blocks that fail to connect from storage are marked invalid.** The IBD - connector no longer retries a failing stored block and restarts sync onto - it. See - [below](#blocks-that-fail-to-connect-from-storage-are-marked-invalid). -## Streaming Consumption API - -### A refused watch add is reported to the client - -The streaming docs said an over-quota watch add was rejected with -`RESOURCE_EXHAUSTED` on gRPC or `429` on WebSocket. The server sent neither: an -incremental `Add*` it refused was logged on the node and dropped, and the client -was never told. A wallet that hit its quota could believe it was watching an -address and miss a payment to it. - -The server now answers every incremental add it does not register with one -`WatchAddRejected` event on the `Watch` stream (`watch_add_rejected` on -WebSocket). It names the kind of add, the reason, the numbers behind it and the -refused items; none of those items is watched. An add that registers still gets -no event. - -| Reason | When | -|---|---| -| `QUOTA_EXCEEDED` | the items do not fit the token's watch quota; carries the units required, held and allowed | -| `RATE_LIMITED` | the token's per-add rate limit is spent; carries a retry-after hint | -| `CAP_EXCEEDED` | a per-connection cap: 16 silent-payment targets, 256 descriptors, or `streamwsmaxsubscriptions` | -| `PERMISSION_DENIED` | the token lacks `stream:watch` | -| `MALFORMED` | the message cannot be applied as a whole, such as an invalid descriptor | - -Both SDKs decode the event (Rust `Event::WatchAddRejected`, Go -`*WatchAddRejected`). `ResilientWatch` re-sends a rate-limited add itself once the -limit allows, within its backoff budget. Any other refusal, or a rate limit past -the budget, drops the refused items from its mirror, so a reconnect no longer -re-sends adds the server refused; a refused descriptor slide falls back to the -window the server kept. - -Two adds that grow nothing are no longer refused. A silent-payment add that only -updates the labels of targets already watched is applied without spending a -per-add rate token, and on WebSocket an add that only re-asserts items already -watched is no longer shed when the connection is at `streamwsmaxsubscriptions`. - -The event is an additive schema change (`NodeEvent` tag 32): an older SDK ignores -it, and an older server never sends it. - -## Consensus and validation - -### Sigops are counted as Bitcoin Core counts them - -satd now counts signature operations exactly as Bitcoin Core does: for the -block sigop limit, the mempool's per-transaction sigop cost, block templates, -`getblocktemplate`'s `sigops` field, and the policy rules that read -`tx.sigops_cost`. Earlier versions used rust-bitcoin's counter, which counted -some P2SH redeem scripts and P2WSH witness scripts lower than Core does: an -`OP_CHECKMULTISIG` that follows another signature opcode. - -Parents admitted through `submitpackage` without a fee check now carry their -real sigop cost and prevout data, so block templates count their sigops and -streaming spend matchers see their spends (#937). - -### The Rust script engine has Core's OP_SUCCESS set - -The Rust script engine now treats `OP_LEFT`, `OP_RIGHT`, `OP_INVERT`, -`OP_AND`, `OP_OR` and `OP_XOR` (0x80, 0x81, 0x83–0x86) as `OP_SUCCESSx` in -tapscript, as Bitcoin Core does. Before, it failed a tapscript leaf containing -one of them, where Core accepts the leaf. This affected `-consensus=rust`, -`-consensus=cpp-shadow` and `satd-chainstate-repair`, where the Rust engine's -verdict is used. The default `rust-shadow` mode decided with -libbitcoinconsensus and logged a shadow mismatch. - -The Rust engine now also runs the taproot rows of Core's script tests and -Core's taproot script-asset vectors, and exempts pay-to-anchor spends from the -upgradable-witness-program policy check as Core does (#933). - -### A block body that fails its own checks is not marked invalid - -Blocks that fail the context-free block checks (Core's `CheckBlock`: merkle -root, size, coinbase position, legacy sigop limit) are no longer marked -invalid, as in Bitcoin Core: such a body may simply not be the block its header -names. An empty block body is now checked against the header's merkle root -first. Before, a body that did not match its header could, in some arrival -orders, leave a valid block marked invalid until `reconsiderblock` was run. - -A block that extends the tip without its header having arrived first, and then -fails validation, is now recorded as invalid, so a second copy is refused -without being validated or stored again (#936). - -### Signet commitments, median-time-past and malformed nBits - -On signet, satd now rebuilds the signed coinbase commitment exactly as Bitcoin -Core does, so blocks whose witness commitment contains an empty -`OP_PUSHDATA1/2/4`, or ends in a push that does not parse, are judged as Core -judges them. - -If the block index is missing a block inside a median-time-past window, satd -now stops connecting with a storage error instead of computing the median from -fewer blocks, and does not mark the block invalid. The work of a block with -malformed `nBits` is now zero, as in Core (#948). - -## Chain, sync and reindex - -### Every block that reaches the tip is reported - -satd connects a block it already holds once its parent becomes the tip, through -a drain that runs every half second. Two things leave a block there: a block -that arrived before its parent, and the stored blocks an initial-sync batch can -leave behind when it ends. The drain connected the block and told nothing that -watches for new blocks: - -- `-blocknotify` did not run for it; -- it was not announced to peers; -- Electrum `blockchain.headers.subscribe` and scripthash subscribers, Esplora's - `/blocks/sse`, the streaming API and the ZMQ `hashblock` topic never saw it, - so a wallet learned of a confirmation in that block only at the next one. - -The drain now emits the same `BlockConnected` event as every other connect that -extends the tip, after the mempool has dropped the block's transactions (#900). -The connector used during initial sync still emits none, as before. - -### satd stops on a chainstate read error - -satd now stops, as Bitcoin Core does, when its chainstate database cannot read -a coin: an I/O error, a checksum mismatch, or a coin record that does not -decode. It logs `Error reading from database, shutting down`. Before, satd -treated such a coin as spent. If a valid block spent that coin, satd refused -the block and recorded it as invalid, and that record survived restarts. - -After such a stop, the next start replays the blocks after the last flush. If -the error comes back, check the disk and rebuild with `-reindex-chainstate` -(#945). - -### Header sync continues from a peer's last header - -After a full batch of headers, satd now asks the peer for more starting from -the last header that peer sent, as Bitcoin Core does. Before, a node whose -best header chain was on a competing branch could stop asking a peer whose -chain forked below it for headers, and so fail to sync to the chain with the -most work. - -The first block of a difficulty period is now found on the block's own -branch, so a block on a branch that forked before the period starts is judged -against that branch. Headers and blocks must also claim a target within the -network's proof-of-work limit, which on signet bounds the work behind each -header (#954). - -### Blocks that fail to connect from storage are marked invalid - -When a block failed validation while being connected from storage, during -initial sync or while catching up, satd did not record it as invalid. The IBD -connector retried the same block and restarted sync onto it until a better -chain appeared. satd now marks such a block invalid, as Bitcoin Core does, and -moves on. - -After any block fails to connect, the header tip `getblockchaininfo` reports -in `headers` now leaves that block's branch. A chain with more work made of -blocks below the failed one is now activated at once, instead of when the next -block arrives (#957). - -### A block submitted before its parent connects with it - -A block passed to `submitblock` before its parent's block was stored to wait -for the parent, but its header did not become the best header. When the -parent arrived and connected, nothing connected the child: it stayed -`valid-headers` in `getchaintips`, and `headers` in `getblockchaininfo` did -not count it, until another block or header arrived. satd now indexes the -header of a block it stores, as Bitcoin Core does for every block it accepts, -so the child is the best header as soon as it is stored and connects within -half a second of its parent. Peers announce headers before blocks, so P2P -sync was not affected; pools and test harnesses that submit blocks out of -order were (#922). - -## Initial block download - -### A block stuck with a slow peer is asked of another again - -satd downloads blocks from many peers at once and connects them in height -order. When the block it needs next has been with one peer for 15 seconds, the -network loop's periodic pass takes it back and asks another peer. That pass also -notices peers that have stopped delivering, and gives idle peers work. - -In 0.6.0 that pass hardly ran during initial sync. The loop takes arriving -blocks off its queue in batches, and a change in 0.6.0 let a full queue put the -pass off until the queue had room, which during initial sync it never has. The -block the sync needed next could wait up to a minute on one slow peer while tens -of thousands of later blocks sat downloaded behind it. - -The pass now runs every half second however busy the queue is, at most one -batch late (#909). We timed two-minute fresh mainnet syncs on a disk with slow -flushes (about 9 ms each), nine with 0.6.0 and six with this fix: - -- 0.6.0 never released a stuck block, and five of its nine runs waited more than - 50 seconds on a single block. -- With the fix the pass ran about every 0.7 seconds and released a stuck block - 15 to 25 times per run. No run waited more than 28 seconds on one block. - -A wait can still pass 15 seconds when the next peer asked is slow too. - -The rate at which a node stores blocks during initial sync is unchanged. That is -set by the disk, which this release does not change (#539). - -### The download window no longer stalls - -Initial sync no longer stalls when the `-maxahead` window is full while the -next block is not being downloaded. Blocks sent again for heights that are -already connected no longer count against the window, and a peer that is still -owed blocks only counts as active when it delivers blocks it was asked for. - -A local disk or database write failure while storing a downloaded block no -longer raises the sending peer's ban score. When following the tip, satd asks -one peer for each missing block instead of all of them. Block downloads, -including the AssumeUTXO background download, now go only to peers that -advertise block service, as in Bitcoin Core (#953). - -## P2P - -### Blocks that arrive before their parent are kept within bounds - -A block received before its parent is now kept only if the node asked for it, -and only within fixed bounds: valid proof of work, at most 4 MB, at most 4 -blocks per peer and 32 MB in all, for at most 5 minutes. Other such blocks are -dropped and the sender is asked for headers, which is how the node learns of -the missing parent. A peer's `ping` is no longer held while one of its blocks -waits for a parent (#941). - -### Core's message limits and a per-peer send budget - -satd now applies Bitcoin Core's limits to single P2P messages. A message over -4,000,000 bytes ends the connection, and an `inv` or `getdata` with more than -50,000 entries, `headers` with more than 2000 or `addr`/`addrv2` with more than -1000 counts as misbehaviour. - -Blocks a peer asks for with `getdata` are now served as the peer reads them, -with about 1 MB queued per peer, as with Core's default `-maxsendbuffer`. A -peer whose connection has taken no data for 20 minutes is disconnected, and -`disconnectnode` and bans now take effect on a peer that has stopped reading. -A peer's user agent is limited to 256 bytes and sanitized before it is logged -or reported in `getpeerinfo`. BIP 157 filter requests that Core refuses (an -unknown filter type, an unknown stop hash, a bad range, or any request while -filters are not served) now disconnect the peer instead of being ignored -(#939). - -### Misbehaving peers are handled as Core handles them - -Peers with the `noban` permission and manual connections (`addnode`, -`-addnode`, `-connect`) are never disconnected or banned for misbehaviour. A -peer on a local address, and every peer that connects through the node's Tor -hidden service, is disconnected but its address is not banned. - -Before, one such peer's misbehaviour could ban 127.0.0.1 for `-bantime` (24 -hours by default, kept across restarts), which refused every onion and local -P2P client without `noban` for that time. `getpeerinfo` now reports peers that -came in through the hidden service with `network: "onion"` (#944). - -### Address rate limits and outbound dial accounting - -satd now rate-limits the addresses each peer announces as Bitcoin Core does: -one to start, then one every 10 seconds. Peers with the `addr` permission and -answers to the node's own `getaddr` are exempt. `getpeerinfo` reports -`addr_processed` and `addr_rate_limited`. - -Outbound dials in flight now count against the automatic outbound limit, so a -reconnect pass no longer starts more dials than there are free slots. Manual -connections are outside that limit, as in Core: `addnode onetry` works on a -node with all its automatic peers connected (and answers null for a peer -already connected), added nodes are limited to 8 at a time and retried at -least once a minute, and bans stop automatic dials, onion hosts included, but -not manual ones. `setban` of an onion host now disconnects that peer (#952). - -### One outbound peer per network group, and 8 after initial sync - -Automatic outbound connections now follow Bitcoin Core's rule that no two -share a network group: the /16 for IPv4, the /32 for most IPv6, or the AS -number with `-asmap`. A manual, full-relay or block-relay-only peer, or a dial -in flight, holds its group; inbound peers, feelers and onion peers do not. -satd dialled learned addresses in the order it learned them, and addresses -learned together often sit in one group, so several outbound slots could go to -one operator's nodes (#894). - -During initial sync satd opens up to 64 outbound full-relay connections, to -download blocks from more peers at once. Nothing closed them afterwards, so a -node that had synced, rebuilt or caught up after downtime kept them until the -peers dropped them. Once the node has caught up it now disconnects the surplus -one peer every 10 seconds until 8 remain, choosing as Core's -`EvictExtraOutboundPeers` does: a peer that shares its network group with -another goes first, then the peer that least recently announced a new block, -and manual and block-relay-only peers, the first four outbound peers that showed -a chain as good as ours, the only peer on a network, a peer with blocks in -flight and a peer connected for under 30 seconds are kept (#895). - -## Mempool - -### A full mempool checks scripts before it evicts - -When the mempool is full, an incoming transaction's scripts are now checked -before anything is evicted to make room for it, and eviction never removes the -incoming transaction's own parent. Eviction now takes time in proportion to -what it removes rather than to the size of the pool, so admitting into a full -pool no longer holds the mempool lock for long. - -Lowering `maxmempool` with a config reload now trims the pool to the new size -at once; before, the node refused new transactions until the pool drained -below the new limit. The orphan pool's internal queue no longer grows with -every orphan ever received (#942). - -### Mempool RPCs match Bitcoin Core - -`getmempoolentry` and `getrawmempool true` report `vsize` rounded up, the -entry's real `height`, only direct parents in `depends`, `bip125-replaceable` -inherited from a signalling ancestor, and ancestor and descendant fees that -include `prioritisetransaction` deltas. `getmempoolinfo` gains `total_fee`, -reports `bytes` as total vsize and `usage` as estimated memory, and reports -`loaded: true` only once the saved mempool has finished loading. `gettxout` -now honours `include_mempool` (default true). `sendrawtransaction` refuses a -negative, unreadable or ≥ 1 BTC/kvB `maxfeerate` or `maxburnamount` instead of -silently dropping the limit, and answers `-27` for an already-confirmed -transaction without needing `-txindex`. - -A restart now keeps mempool entry times, so `-mempoolexpiry` keeps counting, -re-admits transactions that met the fee floor through -`prioritisetransaction`, and loads a large mempool in linear time. -Descendants of a transaction that conflicts with a new block now leave the -mempool with it, and confirmed transactions' priority deltas are cleared -(#946). - -### A restart keeps the whole mempool - -A clean restart could drop most of the saved mempool. satd re-admitted the -transactions in `mempool.dat` in order of entry time, which has one-second -resolution, and the file was written in no particular order, so a child that -entered in the same second as its parent was often tried first, refused for -missing inputs, and took every descendant behind it along. Each restart made -it worse, as the transactions it did re-admit then shared even fewer seconds. -satd now writes the file parents first, as Bitcoin Core does, and on load -puts every transaction after any parent in the same file, so a `mempool.dat` -saved by 0.6.0 also reloads in full. The file format is unchanged (#949). - -## Electrum and Esplora - -### Electrum status hashes follow history order - -The scripthash status satd announces, on `blockchain.scripthash.subscribe` and -in notifications, now hashes the history in the order `get_history` returns -it, as the Electrum protocol defines. Before, an address with both confirmed -and unconfirmed transactions got a status that never matched its history, so -Electrum desktop dropped the connection again and again and did not show the -pending transaction until it confirmed. - -`get_history` and `get_mempool` now list mempool transactions with height 0 -before those with -1, and order transactions within a block by their position -in it. A subscription no longer gets its own status pushed back at the next -block (#938). - -### Esplora paging, empty witnesses and fee estimates - -`/address/:addr/txs` and `/scripthash/:hash/txs` now honour mempool.space's -`?after_txid=` cursor, so explorers that page with "load more" get the next -page instead of the first one again; an unknown cursor answers 422 -`after_txid not found`. A script's mempool transactions are listed in the -order the node admitted them. `/block/:hash/txs/:start_index` answers 404 -`start index out of range` at or past the end of the block, and 400 for a -start that is not a multiple of 25, instead of an empty page. Legacy inputs no -longer carry an empty `witness` array. - -`/fee-estimates` now leaves out the targets the node has no estimate for, and -answers `{}` during initial sync or right after startup instead of reporting -1 sat/vB for every target (#943). - -### Long address histories no longer hold up other requests - -Electrum and Esplora requests that read a long address history now run on a -blocking thread pool instead of the API runtime's worker threads, so one slow -request no longer stalls other Electrum connections, Esplora, events gRPC or -`/metrics`. `--electrumrequesttimeout` now takes effect (a request past it is -answered with an error and its connection closed), and Esplora's address -routes answer 503 at `--esplorarequesttimeout`. - -Paging an address with `/txs/chain/:last_seen_txid` or `?after_txid=` reads -only the requested page instead of the whole history, and status -notifications for subscribed addresses are computed off the threads that -connect blocks, once per batch of queued blocks. satd now logs at startup when -fewer Electrum connections than `--electrummaxconns` can fill the server-wide -`--addrindexsubscriptions` registry; see the Electrum chapter's "Subscription -limits" in the Operator Manual (#947). - -## Indexes - -### Indexes do not serve during or after an unfinished backfill - -The address, spend and compact block filter indexes no longer serve while a -`backfillindex` run is in progress or after one stopped part-way, as the -silent-payment index already did, and `NODE_COMPACT_FILTERS` is withdrawn for -that time. Electrum and Esplora subscription statuses refused during a -backfill are pushed once it ends. - -A reorg during a backfill can no longer delete rows the active chain wrote, -which could drop an entry from an address history, report a spent output as -unspent, or break the filter header chain. A read during a chainstate flush no -longer misses the newest blocks' index rows; it waits for the flush. A block -whose outputs are all unspendable and which spends nothing no longer loses its -filter and height rows when it replaces another block in a reorg (#958). - -### A filter backfill keeps the filter header chain - -A `backfillindex blockfilter` run writes its rows straight to the database, -while the rows the live connect path writes for new blocks wait in the -chainstate cache until the next flush. Where both held a row for the same -height, the cached one won: reads served it and the flush wrote it over the -backfill's. Those cached rows were written before the rows below them -existed, so their filter headers chained off zeros. From the first such height -on, `getblockfilter`, `getcfheaders`, `getcfcheckpt` and BIP 157 peers got -filter headers that differ from Bitcoin Core's; the filters themselves were -correct. This hit the blocks a backfill's final catch-up rewrote that had -connected since the last cache flush, which on a syncing node is most of them, -and any blocks connected with the index on but not yet flushed when the -backfill started. - -A backfill's row now replaces the cached row for the same height. On the -first start of 0.6.1, a node with the filter index enabled on which a filter -backfill has ever run checks the stored header chain once, from genesis: -it recomputes each header from the stored filter and the header below it and -rewrites the ones that differ, logging the first height it rewrote. The check -reads every stored filter once (a few minutes on a mainnet node) and runs in -the background; filters are served meanwhile, and a backfill requested in that -time starts when it ends. - -## Observability - -### No warning for every block early in initial sync - -Early in initial sync, 0.6.0 logged `Connector stuck waiting for block data` at -WARN for nearly every block: 112 of the first 624 log lines on a fresh mainnet -sync, and 490 of 508 in a fresh Umbrel install. Blocks arrive out of order from -many peers and the connector takes them in height order, so it waits a second -or two on some block all through a healthy sync. The warning fired on the first -moment of every such wait. - -It now fires once the connector has waited 20 seconds on the same block, then -once a minute while the wait goes on, and it carries `waited_secs`, how long the -wait has lasted. That is longer than the scheduler takes to give up on a slow -peer and ask another for the block (15 seconds), so a wait that long means the -sync is held up and not merely handling a slow peer. The warning still carries -the scheduler's view of that block (#904). - -## Operator - -### An interrupted `-reindex` continues where it stopped - -0.6.0 refused to start after an interrupted full `-reindex` until it was -given `--reindex` again, and then replayed the chain from genesis. A mainnet -replay takes days, so on an app-store box every stop, reboot or settings -change during it threw the work away, and a package that passes `--reindex` -once, as StartOS's migration from Bitcoin Core does, left a node that would not -start at all. Start9's review of the StartOS package found it. - -Now any start that finds an unfinished full reindex continues it, whatever the -flags. satd scans the block files again and plans the chain to replay as -before, then checks each block the interrupted run had replayed against them: -its index entry must have the header the files hold, at the file position the -scan found it, at its planned height, with the chainwork its ancestors add up -to. It then connects the rest. Nothing is downloaded and no block file is -written. The replay flushes at least every 1000 blocks, so a kill costs at -most that much replay plus the scan, and a stop signal only the scan. - -A reindex that cannot be continued runs again from genesis, still on any -start: one whose wipe may not have finished, one satd 0.6.0 started (0.6.0 did -not record whether its wipe had finished), one at another chainstate schema, -and one whose replayed blocks no longer match the block files. In that last -case satd says so and exits without changing anything, and the next start -runs the reindex from genesis. An interrupted `-reindex-chainstate` is not -continued yet: its flags restart it from genesis, as in 0.6.0 (#508). - -### The `-reindex` scan reads block headers only - -A full `-reindex` first scans the block files and plans the chain to replay -from the block headers. It read each `blk*.dat` file whole to get them, so on -mainnet the scan read the entire chain, about 700 GB, before the first block -connected, and a reindex continued after a stop read it all again on every -start. - -The scan now reads each record's length and the block header after it, then -seeks to the next record. A block larger than 4 KiB costs one 4 KiB read; -smaller blocks share reads. The records it finds, and so the chain it plans, -are the same (#916). - -### A full `-reindex` writes nothing to the block files - -A full `-reindex` empties the database before it scans the block files, and -satd then wrote the genesis block again, to the end of the last block file, as -it does on a new datadir. Every reindex left one duplicate genesis record -behind, and on a node switched from Bitcoin Core it went into one of Core's -files. satd now indexes the genesis record that `blk00000.dat` begins with, -where satd and Core both write it, and writes genesis only when the files do -not begin with it (#931). - -### A stop signal during a rebuild stops it cleanly - -The block replays of `-reindex`, `-reindex-chainstate` and -`-upgradechainstate` run inside startup, and satd registered its SIGTERM and -SIGINT handlers only once startup had finished. Until then a stop signal had -its default effect. A process that is PID 1 of its PID namespace, as satd is -in a container started without an init, ignores such a signal, so `docker -stop` waited out its whole grace period and then killed the replay. Anywhere -else the signal killed the replay at once, part-way through a batch. Start9's -review of the StartOS package found it: on StartOS, a stop, reboot or settings -change during a reindex ran out the package's stop timeout and killed the -replay. - -satd now registers both signals as it starts. During a replay a stop request -is left to the replay, which checks for one between block files while it -scans them, and before and during each block it connects. It abandons the -block it is connecting, which writes nothing until it is complete, flushes -what it had connected, keeps the rebuild marker, and exits 0. On a slow disk a -single block's coin lookups can take seconds, so the stop does not wait for -them. If that takes longer -than `-maxshutdownsecs` (30 s by default), satd exits without waiting, and -loses what the replay connected since its last flush. Anywhere else in startup -a stop signal ends satd at once, as before, and now as PID 1 too. - -A shutdown after startup now also logs each wait before it starts: for the -block connector to finish the block in hand, and for the UTXO cache flush, -each with its share of `-maxshutdownsecs`. A stop that runs long says in its -last log line what it is waiting on. - -### The StartOS service log shows satd's progress - -StartOS starts every service with `RUST_LOG=warn,start_core=debug` in its -environment, and satd takes its log filter from `RUST_LOG`. Under the StartOS -package, satd's log therefore carried warnings only: no sync progress, none of -the reindex phases or stop messages above. The package now runs satd with -`RUST_LOG=info`, satd's default everywhere else. - -## Configuration - -### `bitcoin.conf` comments, trace logs and the Tor cookie - -satd now reads `bitcoin.conf` comments as Bitcoin Core does: a `#` anywhere on -a line starts a comment, so a line such as `txindex=1 # keep it` is no longer -misread, and an `rpcpassword` line containing `#` is refused at startup with -Core's error. sat-cli reads the file the same way. - -`-loglevel=trace` no longer writes RPC request headers (including the -`Authorization` credentials) or RPC parameters to the log. With Tor control -enabled, satd reads at most 33 bytes from the cookie file the control port -names and refuses anything that is not a regular file, so a misbehaving -control port cannot make startup hang or run out of memory (#940). - -## RPC - -### `getrawaddrman` keeps each address in its own bucket - -satd does not store addresses by slot. `getrawaddrman` reports a bucket and -position for each one, derived with Core's formulas. When two addresses derived -the same slot, the second moved to the next free slot of the whole table, and -from a bucket's last position that is the first position of the next bucket, one -Core's formulas would never give that address. For one /16 with 32 tried -addresses, about one key in a hundred put an address in another bucket, and some -of those reported the group in nine buckets, where Core's formula allows eight. - -The second address now takes the next free position in its own bucket, as Core -keeps every address in the bucket its formula gives. Only a full bucket sends it -on to the next one with room. Core's table never needs that, since it evicts -instead. - -### Number formatting, `submitblock` and `getblockfilter` match Core - -Difficulty, network hash rate and verification progress are now printed with -Core's 16 significant digits, and `getnetworkinfo` fee floors and PSBT fees as -8-decimal amounts. `submitblock` answers `duplicate-invalid` for a block -already marked invalid, fills in a missing coinbase witness reserved value as -Core does, and throws -22 for undecodable hex and -25 for internal errors -instead of returning them as reject reasons. `getblockfilter` uses Core's -error codes and messages, `getnetworkhashps -1` uses the 144-block interval on -regtest, and a failed RPC password is now logged and answered after a 250 ms -pause, as in Core (#955). - -## Upgrade notes - -- Drop-in upgrade from 0.6.0 for the changes so far: no reindex. The only - configuration change is for an `rpcpassword` containing `#` (below). -- A client that treated silence after a watch add as success should handle - `WatchAddRejected`; one built on a 0.6.0 SDK keeps working and ignores the - event. -- A full `-reindex` that 0.6.0 left unfinished runs again from genesis on the - first 0.6.1 start, with or without `--reindex`. Reindexes 0.6.1 starts are - continued. -- `bitcoin.conf`: a `#` anywhere on a line now starts a comment, as in Core. - An `rpcpassword` containing `#` is refused at startup; change the password, - or use `-rpcauth` (#940). -- satd now stops, as Core does, when it cannot read a coin from the - chainstate. A restart replays the blocks after the last flush; if the stop - recurs, check the disk and run `-reindex-chainstate` (#945). -- `gettxout` includes the mempool by default (`include_mempool=true`). - `getmempoolinfo.bytes` is total vsize, not serialized size - (`satd_mempool_bytes` and the `-maxmempool` budget still count serialized - bytes), and `loaded` is false until the saved mempool has finished loading. - `sendrawtransaction` refuses a negative, unreadable or ≥ 1 BTC/kvB - `maxfeerate` or `maxburnamount` instead of ignoring it (#946). -- `submitblock` throws error -22 for undecodable hex and -25 for an internal - error instead of returning a reject string. Difficulty, network hash rate - and verification progress are printed with 16 significant digits, and fee - fields in `getnetworkinfo` and PSBT RPCs with 8 decimals; clients that - compared these as strings may see different text (#955). -- Esplora `/fee-estimates` omits targets the node has no estimate for and - answers `{}` during initial sync. `/block/:hash/txs/:start_index` answers 404 - past the end of the block and 400 for a start that is not a multiple of 25 - (#943). -- The address, spend and compact block filter indexes do not serve during a - `backfillindex` run, or after one that stopped part-way until a re-run - completes; `NODE_COMPACT_FILTERS` is withdrawn meanwhile. A redundant run on - a complete index also stops serving until it completes (#958). -- A node with `-blockfilterindex` on which `backfillindex blockfilter` has - ever run checks its stored filter header chain once on its first 0.6.1 - start and rewrites the headers a 0.6.0 backfill left chained off zeros. No - action is needed; the log line `rewrote filter headers` names the first - height that was wrong. Until the check finishes, the old headers are served. -- Misbehaving `noban` and manual peers are never disconnected or banned. Peers - on local addresses and peers that come in through the onion service are - disconnected but not banned (#944). - ---- - -## About this release - -satd is free software under the [MIT License](../../LICENSE). - -- **Stability & compatibility** — Tier 1 surfaces (Bitcoin Core-compatible - JSON-RPC wire shape, CLI flag names/defaults, `bitcoin.conf` keys, on-disk - layout, Electrum/Esplora surfaces) are governed by - [`STABILITY_POLICY.md`](../../STABILITY_POLICY.md). satd follows - [semantic versioning](https://semver.org/spec/v2.0.0.html); while in the - `0.x` pre-1.0 line, deprecations may be accelerated (see the policy). -- **Security & disclosure** — report vulnerabilities per - [`SECURITY.md`](../../SECURITY.md). Consensus-affecting bugs are treated as - P0 with same-day acknowledgement. -- **Verifying this release** — tarballs are signed with minisign, container - images with cosign (keyless OIDC), and git tags with SSH signatures (no GPG). - The key matrix and verification steps are in - [`SECURITY.md`](../../SECURITY.md). -- **Intentional differences from Bitcoin Core** — catalogued in - [`CORE_DIFFERENCES.md`](../../CORE_DIFFERENCES.md). -- **Operating satd** — observability, the full flag matrix, tuning, and the - downstream packaging contract live in the - [Operator Manual](https://epochbtc.github.io/satd/). diff --git a/docs/release-notes/0.6.1.md b/docs/release-notes/0.6.1.md new file mode 100644 index 000000000..775e82dc4 --- /dev/null +++ b/docs/release-notes/0.6.1.md @@ -0,0 +1,276 @@ +# satd 0.6.1 + +> Released 2026-10-15 · git tag [`v0.6.1`](https://github.com/epochbtc/satd/releases/tag/v0.6.1) · [Changelog](../../CHANGELOG.md) + +satd 0.6.1 is a patch release on the 0.6 line, cut from the `release/0.6` +branch. It fixes bugs in 0.6.0: reindex and shutdown handling, initial sync +stalls, header and block validation that differed from Bitcoin Core, P2P +limits and peer handling, mempool behavior and RPC answers, Electrum and +Esplora, and the indexes. It is a drop-in upgrade from 0.6.0 with no reindex. +Read the [upgrade notes](#upgrade-notes) for the few changes an operator or +client can notice. + +## Highlights + +- **An interrupted `-reindex` continues where it stopped**, on any start, + instead of starting over from genesis (#906). +- **A stop signal stops a rebuild cleanly.** SIGTERM or SIGINT during a + reindex or chainstate upgrade flushes and exits 0, also as PID 1 in a + container (#907). +- **Initial sync no longer waits on one slow peer** for up to a minute, and + no longer stalls with a full download window (#909, #953). +- **Sigops, the tapscript `OP_SUCCESS` set, header sync and invalid-block + handling now match Bitcoin Core** (#937, #933, #954, #957). +- **A refused streaming watch add is reported to the client** with a + `WatchAddRejected` event (#899). +- **Electrum desktop no longer reconnects in a loop** on addresses with + pending transactions (#938). + +## Streaming Consumption API + +- A watch add the server refused (over the token's quota or rate limit, over + a per-connection cap, without `stream:watch`, or malformed) was logged on + the node and dropped, so a client could believe it was watching addresses + it was not. The server now answers with a `WatchAddRejected` event on the + `Watch` stream (`watch_add_rejected` on WebSocket) naming the reason and the + refused items; both SDKs decode it, and `ResilientWatch` retries a + rate-limited add and stops re-sending other refused items on reconnect + (#899). +- A silent-payment add that only changes labels no longer spends a rate-limit + token, and on WebSocket an add that only re-asserts watched items is no + longer shed at `streamwsmaxsubscriptions`. The event is an additive schema + change (`NodeEvent` tag 32): older SDKs ignore it (#899). + +## Consensus and validation + +- Signature operations are counted exactly as Bitcoin Core counts them, for + the block sigop limit, mempool policy and block templates. The old counter + scored an `OP_CHECKMULTISIG` after another signature opcode in P2SH and + P2WSH scripts lower than Core. Parents admitted by `submitpackage` without a + fee check now carry their real sigop cost (#937). +- The Rust script engine treats 0x80, 0x81 and 0x83–0x86 as `OP_SUCCESSx` in + tapscript, as Core does, instead of failing the leaf. This affected + `-consensus=rust`, `cpp-shadow` and `satd-chainstate-repair`; the default + `rust-shadow` mode decided with libbitcoinconsensus (#933). +- A block body that fails the context-free block checks no longer marks the + block invalid, since the body may not be the block its header names. An + empty body is checked against the header's merkle root first (#936). +- On signet the signed coinbase commitment is rebuilt exactly as Core does. A + block missing from a median-time-past window stops connecting with a storage + error instead of using fewer blocks, and malformed `nBits` count as zero + work (#948). + +## Chain, sync and reindex + +- A block that reached the tip through the stored-tail drain, such as one that + arrived before its parent, was connected without a chain event: + `-blocknotify` did not run, peers were not told, and Electrum, Esplora SSE, + streaming and ZMQ `hashblock` subscribers missed it. The drain now reports + each block it connects (#900). +- satd stops, as Core does, when it cannot read a coin from the chainstate, + instead of treating the coin as spent and recording a valid block as + invalid (#945). +- Header sync continues from the last header a peer sent, so a node whose best + header is on a competing branch keeps syncing. The retarget start is found + on the block's own branch, and targets above the proof-of-work limit are + refused (#954). +- A block that fails to connect from storage is marked invalid instead of + being retried by the IBD connector. The `headers` tip leaves the failed + branch, and a chain with more work below it is activated at once (#957). +- A block passed to `submitblock` before its parent is now indexed as a + header, as Core does: it becomes the best header and connects as soon as + its parent does, where it stayed unconnected until another block arrived + (#922). + +## Initial block download + +- The network loop's periodic pass, which takes a block back from a peer that + has held it for 15 seconds and asks another, hardly ran while blocks were + arriving. The block the sync needed next could wait up to a minute on one + slow peer. The pass now runs every half second (#909). +- Initial sync no longer stalls with a full `-maxahead` window. Re-sent blocks + do not count against it, a local write failure does not penalize the + sending peer, and block downloads go only to peers that advertise block + service (#953). + +## P2P + +- A block that arrives before its parent is kept only if it was requested and + within fixed bounds: 4 per peer, 32 MB in all, 5 minutes (#941). +- Core's limits on P2P message size and entry counts apply. Block replies are + queued about 1 MB per peer, a peer that takes no data for 20 minutes is + disconnected, user agents are capped at 256 bytes and sanitized, and BIP 157 + requests Core refuses disconnect the peer (#939). +- `noban` and manual peers are never disconnected or banned for misbehaviour. + Local and onion-inbound peers are disconnected but not banned, so 127.0.0.1 + is no longer banned. `getpeerinfo` reports onion-inbound peers as `onion` + (#944). +- Address announcements are rate-limited per peer as in Core, with + `addr_processed` and `addr_rate_limited` in `getpeerinfo`. Dials in flight + count against the outbound limit, manual connections sit outside it, and + `setban` of an onion host disconnects it (#952). +- Automatic outbound connections take at most one peer per network group + (/16, /32, or ASN with `-asmap`), as Core does. The extra outbound peers + opened during initial sync (up to 64) are disconnected one every 10 seconds + once the node has caught up, until 8 remain (#894, #895). + +## Mempool + +- A full mempool checks an incoming transaction's scripts before evicting + anything for it, and never evicts its parent. Eviction costs time in + proportion to what it removes, and lowering `maxmempool` on a config reload + trims the pool at once (#942). +- `getmempoolentry`, `getrawmempool`, `getmempoolinfo`, `gettxout` and + `sendrawtransaction` answer as Core does. A restart keeps entry times and + loads a large mempool in linear time (#946). +- A restart re-admits the whole saved mempool. `mempool.dat` is written + parents first, as Core writes it, and a file saved by 0.6.0 is reordered on + load; before, a child saved in the same second as its parent could be tried + first and dropped with its descendants (#949). + +## Electrum and Esplora + +- The Electrum scripthash status hashes the history in `get_history` order, as + the protocol defines. Electrum desktop no longer reconnects in a loop on + addresses with pending transactions (#938). +- Esplora honours mempool.space's `?after_txid=` paging, answers 404 or 400 + for out-of-range or misaligned block-txs pages, drops empty `witness` + arrays, and `/fee-estimates` omits targets with no estimate (#943). +- Long Electrum and Esplora address-history reads run off the API runtime's + worker threads, so one slow request no longer stalls the others, and + `--electrumrequesttimeout` and `--esplorarequesttimeout` take effect (#947). + +## Indexes + +- The address, spend and compact block filter indexes do not serve during or + after an unfinished `backfillindex`. A reorg during a backfill no longer + deletes rows the active chain wrote, and a read during a flush waits for it + (#958). +- A `backfillindex blockfilter` run no longer leaves a broken BIP 157 filter + header chain, where `getblockfilter`, `getcfheaders` and `getcfcheckpt` + served headers chained off zeros from some height on. The filters themselves + were correct. On first start, 0.6.1 checks and repairs the stored header + chain once (#963). + +## Operator + +- An interrupted full `-reindex` continues from the height it had flushed, on + any start, instead of starting over. satd scans the block files again, + checks the blocks it had replayed against them, and connects the rest, + downloading nothing. A reindex that cannot be continued runs again from + genesis on any start, where 0.6.0 refused every start but `--reindex` + (#906). +- The `-reindex` scan reads only block headers, at most 4 KiB per block, + instead of reading every block file whole (about 700 GB on mainnet) (#916). +- SIGTERM or SIGINT during `-reindex`, `-reindex-chainstate` or + `-upgradechainstate` stops the replay: satd flushes what it had connected, + keeps the rebuild marker and exits 0, within `-maxshutdownsecs`, also as + PID 1 of a container with no init. A shutdown logs each wait before it + starts (#907). +- A full `-reindex` no longer appends a second copy of the genesis block to + the last block file, which on a node switched from Bitcoin Core is one of + Core's files (#931). +- Initial sync logs `Connector stuck waiting for block data` only after a + 20-second wait on one block, then once a minute, with the time waited, + instead of at WARN for nearly every block (#904). +- StartOS package: the service log shows satd's INFO lines. StartOS + starts services with `RUST_LOG=warn`, so the log carried warnings only + (#930). + +## Configuration + +- `bitcoin.conf` treats `#` anywhere on a line as a comment start, as Core + does, and refuses an `rpcpassword` line containing `#`. Trace logs no longer + include RPC headers or parameters, and the Tor control cookie read is + bounded (#940). + +## RPC + +- `getrawaddrman` keeps each address in the bucket Core's formulas give it. + Two addresses that derived the same slot could put the second in the next + bucket (#923). +- RPC numbers use Core's formatting (16 significant digits, 8-decimal fee + amounts). `submitblock` answers `duplicate-invalid` and throws -22 or -25, + `getblockfilter` uses Core's errors, and a failed RPC password is answered + after 250 ms (#955). + +## Dependencies + +- `rustls` is updated to 0.23.46 for RUSTSEC-2026-0346, -0347, -0348 and + -0349. None of them applies to satd's configuration (#966). + +## Upgrade notes + +- Drop-in upgrade from 0.6.0: no reindex. The only configuration change is + for an `rpcpassword` containing `#` (below). +- A client that treated silence after a watch add as success should handle + `WatchAddRejected`; one built on a 0.6.0 SDK keeps working and ignores the + event. +- A full `-reindex` that 0.6.0 left unfinished runs again from genesis on the + first 0.6.1 start, with or without `--reindex`. Reindexes 0.6.1 starts are + continued. +- `bitcoin.conf`: a `#` anywhere on a line now starts a comment, as in Core. + An `rpcpassword` line containing `#`, including a trailing comment such as + `rpcpassword=pw # old`, is refused at startup; change the password or move + the comment, or use `-rpcauth` (#940). +- satd now stops, as Core does, when it cannot read a coin from the + chainstate. A restart replays the blocks after the last flush; if the stop + recurs, check the disk and run `-reindex-chainstate` (#945). +- `gettxout` includes the mempool by default (`include_mempool=true`). + `getmempoolinfo.bytes` is total vsize, not serialized size + (`satd_mempool_bytes` and the `-maxmempool` budget still count serialized + bytes), and `loaded` is false until the saved mempool has finished loading. + `sendrawtransaction` refuses a negative, unreadable or ≥ 1 BTC/kvB + `maxfeerate` or `maxburnamount` instead of ignoring it (#946). +- `submitblock` throws error -22 for undecodable hex and -25 for an internal + error instead of returning a reject string. Difficulty, network hash rate + and verification progress are printed with 16 significant digits, and fee + fields in `getnetworkinfo` and PSBT RPCs with 8 decimals; clients that + compared these as strings may see different text (#955). +- Esplora `/fee-estimates` omits targets the node has no estimate for and + answers `{}` during initial sync. `/block/:hash/txs/:start_index` answers 404 + past the end of the block and 400 for a start that is not a multiple of 25, + and an unknown `?after_txid=` cursor answers 422 (#943). +- `--electrumrequesttimeout` and `--esplorarequesttimeout` (30 seconds by + default) now take effect: an Electrum request past it gets an error and its + connection is closed, and an Esplora address route answers 503. Raise them + if clients read very long address histories (#947). +- The address, spend and compact block filter indexes do not serve during a + `backfillindex` run, or after one that stopped part-way until a re-run + completes; `NODE_COMPACT_FILTERS` is withdrawn meanwhile. A redundant run on + a complete index also stops serving until it completes (#958). +- A node with `-blockfilterindex` on which `backfillindex blockfilter` has + ever run checks its stored filter header chain once on its first 0.6.1 + start and rewrites the headers a 0.6.0 backfill left chained off zeros. The + check reads every stored filter once (a few minutes on mainnet) in the + background. No action is needed; the warning `rewrote filter headers` names + the first height that was wrong. Until the check finishes, the old headers + are served (#963). +- Misbehaving `noban` and manual peers are never disconnected or banned. Peers + on local addresses and peers that come in through the onion service are + disconnected but not banned (#944). + +--- + +## About this release + +satd is free software under the [MIT License](../../LICENSE). + +- **Stability & compatibility** — Tier 1 surfaces (Bitcoin Core-compatible + JSON-RPC wire shape, CLI flag names/defaults, `bitcoin.conf` keys, on-disk + layout, Electrum/Esplora surfaces) are governed by + [`STABILITY_POLICY.md`](../../STABILITY_POLICY.md). satd follows + [semantic versioning](https://semver.org/spec/v2.0.0.html); while in the + `0.x` pre-1.0 line, deprecations may be accelerated (see the policy). +- **Security & disclosure** — report vulnerabilities per + [`SECURITY.md`](../../SECURITY.md). Consensus-affecting bugs are treated as + P0 with same-day acknowledgement. +- **Verifying this release** — tarballs are signed with minisign, container + images with cosign (keyless OIDC), and git tags with SSH signatures (no GPG). + The key matrix and verification steps are in + [`SECURITY.md`](../../SECURITY.md). +- **Intentional differences from Bitcoin Core** — catalogued in + [`CORE_DIFFERENCES.md`](../../CORE_DIFFERENCES.md). +- **Operating satd** — observability, the full flag matrix, tuning, and the + downstream packaging contract live in the + [Operator Manual](https://epochbtc.github.io/satd/). diff --git a/docs/release-notes/README.md b/docs/release-notes/README.md index 1364df505..913f7c51f 100644 --- a/docs/release-notes/README.md +++ b/docs/release-notes/README.md @@ -16,6 +16,7 @@ The split is deliberate: | File | Release | Status | |---|---|---| +| [`0.6.1.md`](0.6.1.md) | 0.6.1 | Released 2026-10-15 | | [`0.6.0.md`](0.6.0.md) | 0.6.0 | Released 2026-10-05 | | [`0.5.2.md`](0.5.2.md) | 0.5.2 | Released 2026-09-11 | | [`0.5.1.md`](0.5.1.md) | 0.5.1 | Released 2026-09-04 | diff --git a/satd-events-client/Cargo.toml b/satd-events-client/Cargo.toml index 788ff2e00..e489d82c9 100644 --- a/satd-events-client/Cargo.toml +++ b/satd-events-client/Cargo.toml @@ -38,7 +38,7 @@ tls = ["dep:tokio-rustls", "tonic/tls", "tonic/tls-webpki-roots"] [dependencies] # Cargo uses `path` for the local build and `version` to resolve the published # dependency; keep this in lockstep with the workspace `version`. -satd-events-proto = { version = "=0.6.0", path = "../satd-events-proto" } +satd-events-proto = { version = "=0.6.1", path = "../satd-events-proto" } tonic = { workspace = true } prost = { workspace = true } # Only pulled by the `tls` feature, and only to install the `ring` rustls