fix(rich-markdown): demote links whose URL Telegram would mangle

CHANGELOG.md

@@ -1,6 +1,12 @@
 # Changelog
 
 
+## Unreleased
+
+### Miscellaneous
+
+- Regenerate changelog
+
 ## v0.11.1 - 2026-08-01
 
 ### Build

README.md

@@ -134,7 +134,8 @@ Member references in `members`/`admins` (group create) and in `members bulk-add`
 
 - **Dialect** (parsed server-side): `#`…`######` headings, tables with alignment, task lists, `>` quotes, fenced code with a language, `---` dividers, `~~strike~~`, `==marked==`, `||spoiler||`, footnotes, math, `<details>`, and media as a standalone block — by public HTTPS URL (`![](https://…jpg "caption")`, fetched by the server) or, **on the CLI only**, by local file (uploaded and referenced as `tg://photo?id=…` / `tg://video?id=…` / `tg://audio?id=…`). Headings, lists, tables, quotes, code, dividers, URL media and uploaded local media are verified over MTProto; the remaining constructs are documented for the Bot API twin of the same server feature and are unverified here.
 - **Wikilinks** (always on, no knob): Obsidian `[[target]]` / `[[target|alias]]` links are expanded to plain text before anything else runs — the alias wins when present, otherwise the target reads as Obsidian renders it (a leading `#`, a link into the current note, drops; every other `#` becomes ` > `; block references like `note#^blk` fall out of that same rule unchanged). Only the first `|` separates target from alias, so further pipes stay in the alias (`[[A|B|C]]` → `B|C`); an empty half falls back to the other (`[[Note|]]` → `Note`, `[[|Стас]]` → `Стас`); a link with both halves empty (`[[]]`, `[[|]]`) is not a link and ships verbatim rather than collapsing to nothing. `![[…]]` embeds are untouched (that syntax is media, resolved as described below) and so is anything inside inline code or a fenced code block. Unlike frontmatter stripping and local-media resolution, this is **not** CLI-only — a literal `[[…]]` reaching a Telegram reader is a defect whichever surface sent it, so it runs on the CLI, HTTP, and MCP alike. A link nested inside its own target/alias (`[[[[a]]]]`) expands too, up to a bounded nesting depth — an unrealistic depth beyond that ships the remainder verbatim rather than rescanning the document without limit. `--dry-run` reports the count as `rich_markdown_wikilinks`.
-- **Paragraph spacing** (on by default): the server renders neighbouring paragraphs tight against each other, so the markdown is rewritten before sending — a U+00A0-only spacer paragraph is inserted between two consecutive paragraphs, before every heading, and after every medium (a photo/video/audio block, or the `<tg-collage>`/`<tg-slideshow>` a run was grouped into), but never *before* media, never after a heading, never inside code/tables/lists/quotes/HTML blocks, and never next to a spacer the author already wrote. Turn it off with `--no-spaced-paragraphs` (HTTP/MCP: `spaced_paragraphs: false`) to keep the author's own spacing; the default also comes from `telegram.defaults.rich_markdown_spaced_paragraphs`. Note that this switches off *only* the spacer pass — media grouping and local-media rewriting are independent, so a truly byte-for-byte send also needs `--no-line-breaks`, `telegram.defaults.rich_markdown_grouping: none` (or `--media-group <index>=none` per run), an article with no local media, and no `[[wikilinks]]`, which are always expanded (no knob). Spacers count toward both limits below: if spacing would push the article past 500 blocks it is sent unspaced with the warning `spaced_paragraphs disabled: N blocks would exceed the 500-block limit`.
+- **Links with `&` in the URL** (always on, no knob): Telegram's own markdown parser HTML-escapes `&` (→ `&amp;`) and `'` (→ `&#39;`) inside a link destination, so `[Справочник](https://example.com/?action=view&handbook=235)` arrives pointing at `…&amp;handbook=235` — a URL the target server reads as a parameter named `amp;handbook`. No spelling of the link avoids it (`&amp;` in the source is escaped twice, `\&` becomes `&#092;&amp;`, autolinks and inline `<a href>` are escaped the same way), and the one form that survives is a **bare URL in the text** — the parser stores no link for it and the clients autodetect it. Such links are therefore rewritten to `text: url` (`[269 - AWRA](https://…&key=269)` → `269 - AWRA: https://…&key=269`); an empty anchor text, or one identical to the URL, collapses to the URL alone, and a link title is dropped (Telegram renders it nowhere). Links whose URL carries none of those characters are left as markdown links — `+`, `%20`, `#`, `~`, `|`, `_`, `*` and non-ASCII all survive the parser intact. `![…](…)` media is never demoted, nor is anything inside inline code or a fenced code block. Like wikilinks this runs on the CLI, HTTP and MCP alike — the parser doing the damage is Telegram's. `--dry-run` reports the count as `rich_markdown_unwrapped_links`.
+- **Paragraph spacing** (on by default): the server renders neighbouring paragraphs tight against each other, so the markdown is rewritten before sending — a U+00A0-only spacer paragraph is inserted between two consecutive paragraphs, before every heading, and after every medium (a photo/video/audio block, or the `<tg-collage>`/`<tg-slideshow>` a run was grouped into), but never *before* media, never after a heading, never inside code/tables/lists/quotes/HTML blocks, and never next to a spacer the author already wrote. Turn it off with `--no-spaced-paragraphs` (HTTP/MCP: `spaced_paragraphs: false`) to keep the author's own spacing; the default also comes from `telegram.defaults.rich_markdown_spaced_paragraphs`. Note that this switches off *only* the spacer pass — media grouping and local-media rewriting are independent, so a truly byte-for-byte send also needs `--no-line-breaks`, `telegram.defaults.rich_markdown_grouping: none` (or `--media-group <index>=none` per run), an article with no local media, no `[[wikilinks]]` (always expanded, no knob) and no link whose URL contains `&` or `'` (always demoted to a bare URL, no knob). Spacers count toward both limits below: if spacing would push the article past 500 blocks it is sent unspaced with the warning `spaced_paragraphs disabled: N blocks would exceed the 500-block limit`.
 - **Line breaks** (on by default): Telegram parses the markdown itself and, like CommonMark, folds a *single* newline inside a paragraph into a space — so an Obsidian note's
   ```
   Фотоальбом - https://…
@@ -147,7 +148,7 @@ Member references in `members`/`admins` (group create) and in `members bulk-add`
 - **Obsidian frontmatter** (**CLI-only**): a leading `---` … `---` YAML block is dropped when the file is read, next to the BOM strip and for the same reason — this dialect has no notion of frontmatter, so the article would otherwise open with a divider and a large heading reading `tags: [...] date: ...`. Only an exact `---` on the first line starts a block, only a matching `---` ends one, and the lines between them must read as YAML (a `key: value` entry first, then only entries, `- ` items, indented continuations or blank lines), so a note that merely begins with a horizontal rule keeps it — even when a later `---` divider would otherwise close the pair; a file that is *nothing but* frontmatter is reported as empty. HTTP/MCP take a markdown string an agent composed rather than a note file, so their input is passed through untouched.
 - **Limits**: 1..32 768 characters (validated locally after normalization, inclusive), ~500 blocks and 50 media attachments (counted locally and reported as warnings — Telegram is the authority); the server additionally caps nesting and table columns and reports its own errors (`RICH_MESSAGE_MARKDOWN_INVALID`, `RICH_MESSAGE_TEXT_TOO_LONG`, …).
 - **Exclusivity**: `--rich-markdown` is a targeted-send-only alternative to the message body — it cannot be combined with `--text`, `--file`, `--file-url` (HTTP/MCP: `text`, `file_urls`, `base64_files`) or with mass mode. `--spaced-paragraphs`/`--no-spaced-paragraphs`, `--line-breaks`/`--no-line-breaks`, `--rich-file`, `--vault-dir` and `--media-group` are errors without `--rich-markdown` (CLI exit 2; HTTP `spaced_paragraphs`/`line_breaks` without `rich_markdown` is a `422`).
-- Everything else is unchanged: entity resolution, the WRITE gate, `--operation-id` idempotency, topic/reply targeting (`--topic-id`/`--reply-to`), and scheduling (`--schedule-at`/`--delay`) all work. `--dry-run` reports the article as markers (`rich_markdown`, post-normalization `rich_markdown_chars`, `rich_markdown_blocks`, `rich_markdown_media`, `rich_markdown_wikilinks`, `rich_markdown_file`, `spaced_paragraphs`, `spaced`, `line_breaks`, `media_grouping`, `rich_markdown_groups`, `rich_files`) rather than echoing a 32k body — the listed files are never read. Normalization warnings go to stderr as `warning: ...` on a real send and ride the result JSON as `warnings`.
+- Everything else is unchanged: entity resolution, the WRITE gate, `--operation-id` idempotency, topic/reply targeting (`--topic-id`/`--reply-to`), and scheduling (`--schedule-at`/`--delay`) all work. `--dry-run` reports the article as markers (`rich_markdown`, post-normalization `rich_markdown_chars`, `rich_markdown_blocks`, `rich_markdown_media`, `rich_markdown_wikilinks`, `rich_markdown_unwrapped_links`, `rich_markdown_file`, `spaced_paragraphs`, `spaced`, `line_breaks`, `media_grouping`, `rich_markdown_groups`, `rich_files`) rather than echoing a 32k body — the listed files are never read. Normalization warnings go to stderr as `warning: ...` on a real send and ride the result JSON as `warnings`.
 - A chat that forbids media rejects the **whole** article: Telegram's `ChatSendMediaForbiddenError` (and its per-type siblings) becomes a `RichMediaForbidden` error naming the chat (HTTP `400`, CLI exit 2). There is no media-less fallback — an article's media is part of its body.
 - If Telegram accepts the request but the response carries no readable message id, the send is **not** marked failed — the operation goes to `needs_review` (the queue never auto-retries it), because the article may well have been delivered. Check the chat before `operations retry`; a blind re-send under a fresh key would duplicate it.
 - A failed rich send is **not** silently retried as plain text — it surfaces as the normal send error, and the caller decides. Requires `telethon >= 1.44` (layer 227), now the project's minimum pin; if an older Telethon is force-installed anyway, only the rich send fails, with an explicit version error (HTTP `500 {"error": "rich_message_unsupported"}`, CLI exit 1, MCP error message) — and the idempotency key is left free, so the same `--operation-id` sends normally once Telethon is upgraded.