feat(index): index .mdx files as markdown-twin content nodes#716
Merged
Conversation
Add .mdx to the indexable-extension set and treat it as a full markdown twin: parsed as markdown (frontmatter + prose + wikilinks), sub-unit split, editable via the node/edge write tools, and a valid edge source/target. A .mdx node's id RETAINS its extension (foo.mdx) so foo.md and foo.mdx never collide on the nodes.id primary key. Unify the three previously divergent id-derivation sites onto one rule in index.NodeIDForPath (strip only ".md", retain every other indexable kind) and add its inverse index.FilePathForID, replacing every hardcoded `sourceID + ".md"` reconstruction (edgewrite x5, doctor). FilePathForID treats a ".md"- suffixed id as a bare stem (a "*.md.md" file), since ".md" is the only stripped kind — so it is a true inverse and does not regress that file. Content routing needs no new branch: .mdx falls through the markdown default in ParseContentFile / NodeText / the sub-unit split. JSX tags and import lines are parsed as markdown (raw HTML / paragraph text); no MDX compiler. Tests: id round-trip incl. .md.md; md+mdx coexistence through the real reindex.Run pipeline; Create-accepts-.mdx; rename referrer-retarget to a moved .mdx node (the #687 analogue).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
.mdxfile support to Tusk. An.mdxfile is a full markdown twin: indexed, queryable, renderable, sub-unit-split, editable via the node/edge write tools, and a valid edge source/target — parsed as markdown (frontmatter + prose + wikilinks), exactly like.md.Why
MDX is markdown-with-JSX; vaults that mix
.mdand.mdxnotes were silently dropping every.mdxfile (the reindex walk only picked up.md/.html/.htm).How
.mdxis indexable — added toindex.indexableExts.foo.mdx) sofoo.mdandfoo.mdxnever collide on thenodes.idprimary key (Decision feat: add circular parent detection (Phase 1) #12)..mdstays the only bare-stem kind.reindex.nodeIDForPath,node.nodeIDForPath,parse.go) now delegate to a singleindex.NodeIDForPath(strip only.md, retain every other indexable kind). Its inverseindex.FilePathForIDreplaces every hardcodedsourceID + ".md"reconstruction (edgewrite.go×5,doctor.go).FilePathForIDtreats a.md-suffixed id as a bare stem (a*.md.mdfile) since.mdis the only stripped kind, so it's a true inverse..mdxfalls through the markdowndefaultinParseContentFile/NodeText/ the sub-unit split. JSX tags andimportlines parse as markdown (raw HTML / paragraph text). No MDX compiler — documented limitation.Verification
Automated: full suite (
go test ./...) +-raceon index/node/reindex green; gofmt/vet/golangci-lint clean. New tests:NodeIDForPath/FilePathForIDround-trip incl. the.md.mdcase; md+mdx coexistence through the realreindex.Runpipeline;Create-accepts-.mdx; rename referrer-retarget to a moved.mdxnode (the #687 analogue).Live e2e (built binary, scratch vault):
.mdxindexed as a first-classnote;foo.md+foo.mdxcoexist as distinct nodes[[wikilink]]materialized as an edge; full sub-unit splitnode get/query/node modify/edge add/node moveall work on the.mdxid; frontmatter rewrites preserve the MDX body (imports +<Callout>JSX)doctorclean; double-reindexconverges to0 indexed / 0 skippededge addon achangelog.md.mdnode writes to the correct fileReview
An adversarial multi-agent review of the diff surfaced one real (minor) regression —
FilePathForIDmis-reconstructing a*.md.mdfile — which is fixed here, plus the durability tests above.Out of scope
MDX/JSX compilation;
.markdownextension; per-component semantic extraction.