Skip to content

fix: resolve grammar from leading token of code fence info string - #54

Merged
veeso merged 1 commit into
veeso:mainfrom
lostbean:fix/split-language-token-before-grammar-lookup
Jun 7, 2026
Merged

fix: resolve grammar from leading token of code fence info string#54
veeso merged 1 commit into
veeso:mainfrom
lostbean:fix/split-language-token-before-grammar-lookup

Conversation

@lostbean

Copy link
Copy Markdown
Contributor

Problem

code.highlight looks up the entire code fence info string in the languages dictionary:

case dict.get(config.languages, language) {

When an author decorates a fence with a title or filename — a common convention — the info string no longer matches a bare language key, so highlighting silently falls back to unhighlighted text:

```cpp:main.ino      ← language is "cpp:main.ino", not "cpp" → Error(Nil)
```cpp main.ino      ← CommonMark: language is "cpp", rest is extra info → also missed

Repro

import blogatto/config/post/code

pub fn main() {
  let cfg = code.default()
  code.highlight(cfg, "cpp:main.ino", "int x = 1;")
  // before: Error(Nil) — block renders unhighlighted
  // after:  Ok([...highlighted spans...])
}

Fix

Match the full info string first (so explicitly registered language names — including any with unusual characters — keep resolving exactly as before), then fall back to the leading token: everything before the first space or :. An unknown leading token still returns Error(Nil), so behaviour for genuinely unsupported languages is unchanged.

The split is intentionally minimal — space and : only — matching CommonMark's "language is the text up to the first whitespace" rule plus the widespread lang:filename convention. No public API changes; the two new helpers are private.

Tests

Added to test/blogatto/config/post/code_test.gleam:

  • cpp:main.ino (colon suffix) highlights
  • cpp main.ino (space suffix) highlights
  • bare cpp still highlights (no regression)
  • brainfuck:hello.bf (unknown leading token) still returns Error(Nil)

I verified the two positive tests fail without the source change and pass with it. Full suite: 506 passed, no failures. gleam format --check src test is clean.

Downstream impact

In a blog built on Blogatto, this currently forces a workaround: re-tokenising code blocks ourselves with the trimmed language after Blogatto's lookup misses. With this fix the framework's own highlight wrapper handles decorated fences directly, letting that workaround be removed.

`code.highlight` looked up the full fence info string in the languages
dictionary, so a fence carrying a title or filename after the language
name failed to highlight. Authors commonly write fences like
`cpp:main.ino` or `cpp main.ino`, neither of which matches the bare
`cpp` key.

Match the full info string first so explicitly registered language
names keep resolving, then fall back to the leading token (everything
before the first space or `:`). Unknown leading tokens still return
Error(Nil).

Adds tests for the colon- and space-suffixed forms, the bare form, and
an unknown language with a suffix.
@cocogitto-bot

cocogitto-bot Bot commented May 29, 2026

Copy link
Copy Markdown

✔️ 760ba3e - Conventional commits check succeeded.

@veeso veeso left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

Nice contribution, I didn't even know about that syntax for code blocks.

Thank you

@veeso
veeso merged commit 1d6425f into veeso:main Jun 7, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants