[Bug] UTF-8 BOM prevents SKILL.md frontmatter from loading

[zfaustk]

Affected area

Coding agent and CLI

What happened?

On current main, a SKILL.md encoded as UTF-8 with an initial byte-order mark is skipped by the skill loader. The same content without the BOM loads normally.

The shared frontmatter parser normalizes newlines but leaves the initial U+FEFF in place. Because it then checks whether the content starts with ---, the BOM-prefixed file is treated as having no frontmatter. The skill loader consequently reports description is required and excludes the skill.

Steps to reproduce

Using the current parser:

const content = `---
name: demo
description: Test skill
---
Body`;

parseFrontmatter(content).frontmatter;
// { name: "demo", description: "Test skill" }

parseFrontmatter(`\uFEFF${content}`).frontmatter;
// {}

I also loaded two demo/SKILL.md files whose bytes differed only by the initial UTF-8 BOM:

plain: skills=["demo"], diagnostics=[]
BOM:   skills=[], diagnostics=["description is required"]

Expected behavior

An initial UTF-8 BOM should not change the parsed frontmatter or prevent the skill from loading.

Prime Agent version

main at 9f9501146e869466acaca66dac49cff857b7b4f9

Environment

macOS 26.3.2 arm64, Node.js 24.14.1

Additional context

Prompt templates use the same parser. In a separate check, a BOM-prefixed template also lost its frontmatter metadata and retained the frontmatter block as template content.

I can add a focused regression and narrow fix if maintainers confirm that BOM-prefixed Markdown resources should be supported.

[Hotragn]

Confirmed your diagnosis against main, and the same defect class reaches further than SKILL.md.

extractFrontmatter normalizes \r\n and \r but never strips a leading U+FEFF, so normalized.startsWith("---") is false and the file is treated as having no frontmatter:

frontmatter plain : {"name":"demo","description":"Test"}
frontmatter BOM   : {}

The JSON config loaders have the same gap, and there it is worse because JSON.parse throws outright on a BOM. settings-manager.ts:382, auth-storage.ts:622, and the model cache at model-registry.ts:929 all parse file content with no BOM handling. Writing a BOM into settings.json drops every setting:

plain    -> theme: light      errors: 0
with BOM -> theme: undefined  errors: 1
   SyntaxError: Unexpected token '', "{ "them"... is not valid JSON

So a BOM'd settings.json silently loses your configuration, and a BOM'd auth.json would be an unreadable credential store. That matters mostly on Windows, where Notepad and PowerShell redirection write UTF-8 with BOM by default — the same population already hitting the other Windows reports this week.

Worth noting the repo already has the helper: stripBom in packages/coding-agent/src/core/tools/edit-diff.ts, used by the edit tool so a model's oldText matches a BOM'd file. The frontmatter parser and the JSON loaders just don't reuse it.

If maintainers want this fixed as one change rather than only in the skill loader, the natural shape is stripping the BOM at each read boundary — extractFrontmatter plus the three JSON.parse sites — with cases for BOM'd frontmatter, BOM'd settings.json, and BOM'd auth.json. Prompt templates go through the same frontmatter parser, so they are covered by the first.

[RelaxJonh]

Confirmed on Windows 11 with Prime Agent 0.7.2 and Node.js 24.x.

The issue is particularly relevant on Windows because many Windows editors (Notepad, Visual Studio) historically saved UTF-8 files with a BOM by default. Users editing SKILL.md files in Notepad will silently produce BOM-prefixed files that Prime Agent silently ignores.

Root cause confirmed in packages/coding-agent/src/utils/frontmatter.ts:13:

const normalized = normalizeNewlines(content);
if (!normalized.startsWith("---")) {
    return { yamlString: null, body: normalized };
}

normalizeNewlines() only handles \r\n and \r — it does not strip the UTF-8 BOM (U+FEFF). When a BOM-prefixed file reaches extractFrontmatter(), the check normalized.startsWith("---") fails because the string actually starts with \uFEFF---.

This affects not just SKILL.md but any file parsed through parseFrontmatter() — prompt templates, extension configs, etc.

I can reproduce this on Windows with:

import { parseFrontmatter } from "./src/utils/frontmatter.ts";
const withBom = "\uFEFF---\nname: test\ndescription: Test\n---\nBody";
console.log(parseFrontmatter(withBom).frontmatter); // {} — empty

A one-line fix in extractFrontmatter() before the startsWith check would resolve this for all callers:

const normalized = normalizeNewlines(content).replace(/^\uFEFF/, "");

Happy to test a patch on Windows if maintainers want this addressed.

[Hotragn]

Upstream has fixed this, so there is a patch to port rather than a fix to design.

earendil-works/pi commit 1355cd36 ("fix(coding-agent): normalize UTF-8 BOMs in text inputs", closes earendil-works/pi#8337) adds a shared helper in packages/coding-agent/src/utils/text.ts:

export function splitBom(content: string): { bom: string; text: string } {
	return content.startsWith("\uFEFF") ? { bom: "\uFEFF", text: content.slice(1) } : { bom: "", text: content };
}
export function stripBom(content: string): string {
	return splitBom(content).text;
}

It then applies it at 16 read boundaries. The two that match this report are one line each:

• utils/frontmatter.ts: normalizeNewlines(stripBom(content))
• core/settings-manager.ts: JSON.parse(stripBom(content)) at both parse sites

They also covered themes, keybindings, manifests, model config, models store, package manager, trust manager, migrations, external editor, and CLI file input, and kept the BOM on edit operations so writes do not silently drop it. Total is +102/-32 across 20 files including a regression test.

On prime-agent main today the same two spots are still unprotected: utils/frontmatter.ts has no BOM strip, and core/settings-manager.ts parses at lines 389 and 575 without one. The line numbers differ from upstream but the code is the same.

Worth noting the fork already ships stripBom in core/tools/edit-diff.ts, so the frontmatter and settings fix can reuse what is there without adding the new util file.