Skip to content

fix(rich-markdown): demote links whose URL Telegram would mangle - #25

Merged
popstas merged 1 commit into
masterfrom
fix/rich-markdown-link-escaping
Aug 3, 2026
Merged

fix(rich-markdown): demote links whose URL Telegram would mangle#25
popstas merged 1 commit into
masterfrom
fix/rich-markdown-link-escaping

Conversation

@popstas

@popstas popstas commented Aug 3, 2026

Copy link
Copy Markdown
Owner

The bug

Telegram's own server-side markdown parser HTML-escapes & (→ &) and ' (→ ') inside a link destination. An ordinary query-string link therefore arrives pointing at ?action=view&handbook=235, which the target server reads as a parameter named amp;handbook. Nothing on our side rewrote it — one real note went out with 19 broken links.

What the spike proved

scripts/spike_rich_link_escaping.py (new) sends one article per mode to Saved Messages and reads it back through messages.getRichMessage, printing the TextUrl.url the server stored.

--mode escaping — no spelling of the link avoids the escaping:

source stored
[a](…&handbook=235) …&handbook=235
[a](…&handbook=235) …&handbook=235
[a](…\&handbook=235) …\&handbook=235
<https://…&handbook=235> …&amp;handbook=235
<a href="https://…&handbook=235"> …&amp;handbook=235
[a](<https://…>) no link at all
…%26handbook=235 unchanged, but the target parses it differently

--mode chars — only & and ' are mangled. +, %20, #, ~, |, _, * and non-ASCII survive untouched; <, >, " prevent a link from forming at all (a separate, pre-existing markdown-level issue, out of scope).

A bare URL in the text is the one form that works: the parser stores no link entity for it and the clients autodetect it.

The fix

unwrap_unsafe_links() in messages/rich_markdown.py rewrites [269 - AWRA](https://x/?a=1&k=2)269 - AWRA: https://x/?a=1&k=2.

  • Only links whose URL contains UNSAFE_LINK_URL_CHARS (&, ') — a working markdown link reads better than a bare URL, so clean links are untouched.
  • The anchor text is kept (in a list of handbook entries it is the only thing naming the target); an empty text, or one identical to the URL, collapses to the URL alone; a link title is dropped (Telegram renders it nowhere).
  • Media (![](…)) is excluded by the pattern's leading (?<!!); code spans are skipped by containment, not overlap — the rule the wikilink and media sweeps already use.
  • No knob and not CLI-only, like the wikilink pass: the parser doing the damage is Telegram's, not the vault's.
  • It can only shrink the text, so it stays out of the over-limit grew_by list.
  • Pass order is now strip → unwrap → group → space → split → count.
  • --dry-run reports rich_markdown_unwrapped_links.

Verification

  • pytest — 2275 passed (2240 + 35 new: 32 in tests/test_rich_markdown_links.py, plus CLI dry-run, HTTP and MCP surface tests)
  • ruff check src tests scripts — clean
  • Live: the real note re-sent to Saved Messages (msg 409106) and read back — 0 occurrences of &amp;, 18 intact &handbook=235&key= URLs, 0 TextUrl entities

Manual checks for a reviewer

  • messages send --rich-markdown <note-with-query-links> --dry-run reports a non-zero rich_markdown_unwrapped_links
  • Send such a note and click a link in the client — it must open the right page
  • Send a note whose links have no & and confirm they are still proper markdown links (blue anchor text, not a raw URL)
  • An article documenting the syntax inside backticks is unchanged

🤖 Generated with Claude Code

Telegram's own server-side markdown parser HTML-escapes `&` (-> `&amp;`)
and `'` (-> `&#39;`) inside a link destination, so an ordinary
query-string link arrives pointing at `?action=view&amp;handbook=235` —
a URL the target server reads as a parameter named `amp;handbook`. One
real note went out with 19 broken links.

`scripts/spike_rich_link_escaping.py` (new) proved there is no spelling
of the link that avoids it: `&amp;` in the source comes back
double-escaped, `\&` as `&#092;&amp;`, an autolink and an inline
`<a href>` are escaped exactly like a plain link, `%26` survives but
changes what the target parses, and a pointy-bracket destination makes
no link at all. Its `--mode chars` run drew the character list: only `&`
and `'` are mangled; `+`, `%20`, `#`, `~`, `|`, `_`, `*` and non-ASCII
survive untouched. A bare URL in the text is the one form that works —
the parser stores no link entity for it and the clients autodetect it.

So `unwrap_unsafe_links()` rewrites `[269 - AWRA](https://x/?a=1&k=2)`
to `269 - AWRA: https://x/?a=1&k=2`, keeping the anchor text because it
is usually the only thing naming the target. Links whose URL carries
neither character stay markdown links — they work and read better.
Media embeds are excluded by the pattern's leading `(?<!!)`, and code
spans are skipped by containment rather than overlap, the rule the
wikilink and media sweeps already use. Like the wikilink pass it has no
knob and is not CLI-only: the parser doing the damage is Telegram's.
It can only shrink the text, so it stays out of the over-limit
`grew_by` list.

Verified live (Saved Messages, msg 409106): the same note read back
through `messages.getRichMessage` now carries 0 occurrences of `&amp;`,
18 intact `&handbook=235&key=` URLs and 0 `TextUrl` entities.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@popstas
popstas merged commit 2efd790 into master Aug 3, 2026
2 checks passed
@popstas
popstas deleted the fix/rich-markdown-link-escaping branch August 3, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant