Skip to content

moresleep512/TypeMD

Repository files navigation

TypeMD

English | 简体中文

A WYSIWYG editor built to instantly open Markdown documents hundreds of MB—and even 1 GB—in size.

TypeMD is a Windows-first, disk-first Markdown editor built with Go, Wails, React, and TypeScript. It provides a continuous, page-free editing experience similar to Typora while keeping the complete document out of Go memory, JavaScript memory, and the DOM.

The Markdown file is always the single source of truth. TypeMD loads only a bounded slice near the viewport, stores structure and search indexes on disk, and progressively scans the remaining content in the background.

Note

TypeMD is currently an engineering preview / Windows RC candidate, not a finished stable release. The opening path has been tested with files up to 1 GiB; the remaining release gates are listed under Current limitations.

Why develop TypeMD?

I was fed up with Typora still becoming sluggish—and even freezing or crashing—when opening a Markdown file of approximately 170,000 Chinese characters on my 16 GB, Ryzen 9 8945HS Windows laptop. This was my actual experience on this machine, not a general claim about every Typora installation. That is why I built TypeMD: no matter how long a document is, writing should not feel heavy.

The project has one central rule:

Document size may increase the time spent on background tasks, but it must not increase foreground memory and mounted DOM without bound.

The bottleneck I actually want to solve

This project does not speculate about Typora's private implementation. It only addresses problems I could observe in my actual writing workflow:

Long-document problem I encountered TypeMD's approach
Opening a large file could block the entire writing surface Read and mount only a bounded first-screen slice; scan the remaining content in the background
More source could mean more memory and browser nodes Keep the body on disk and set explicit limits for Go caches, browser caches, and the DOM
A distant jump could require work proportional to the entire document Resolve byte/height anchors through disk indexes and request only a revisioned target slice
Recovery and search caches could accidentally become copies of the body Store only transactions and contentless inverted terms; the .md file is always the single source of truth

TypeMD's advantage is therefore not a different appearance, but moving file size out of the foreground editing hot path while continuing to pursue the quiet, continuous, near-final-layout writing experience that makes Typora most appealing.

Implemented capabilities

  • Continuous WYSIWYG Markdown editing without Word-style paper pagination.
  • Contextual Markdown markers: show the final result when inactive and reveal source markers when editing the corresponding element.
  • CommonMark/GFM, tables, task lists, footnotes, YAML, KaTeX, syntax highlighting, safe HTML, and local relative resources.
  • Disk-backed Piece Tree, append-only recovery journal, undo/redo, streaming save, and atomic replacement.
  • Streaming block index and SQLite FTS5 word/Trigram indexes that do not store the document body.
  • Two-level virtualization for semantic blocks and oversized visual-line fragments, with bounded DOM ownership.
  • Logical scrolling over a fixed 8,000,000 CSS px runway, height anchoring, two viewport-heights of preload above and below, and a visual buffer for distant jumps.
  • Multiple tabs, file list, outline, full-text search, themes, Chinese/English bilingual UI, file monitoring, recovery state, and external-change handling.

Real opening performance

The following data comes from actual release-build runs, not estimates. Every test file repeatedly contains YAML, headings, emphasis, lists, tasks, tables, quotes, footnotes, code fences, KaTeX, safe HTML, Chinese, emoji, combining characters, and RTL text; these are not plain-text best cases.

File size Process start to editable p50 / p95 Open request to editable p50 / p95 Kernel first slice p50 / p95 Peak process-tree working set p50 / p95
600 KiB 745.5 / 770.9 ms 160.9 / 167.5 ms 9.7 / 10.7 ms 537.6 / 538.8 MiB
10 MiB 755.4 / 760.1 ms 157.6 / 171.9 ms 10.5 / 11.2 ms 544.4 / 559.6 MiB
150 MiB 728.3 / 743.1 ms 159.4 / 163.7 ms 8.5 / 10.0 ms 544.4 / 551.4 MiB
700 MiB 726.1 / 733.0 ms 155.4 / 163.1 ms 9.3 / 9.4 ms 548.3 / 551.7 MiB
1 GiB 740.4 / 750.3 ms 150.4 / 162.6 ms 9.3 / 10.6 ms 551.1 / 555.7 MiB

Reference machine: Windows 10.0.26200, Ryzen 9 8945HS-class CPU, 16 logical processors, 16 GiB memory, and WebView2 150.0.4078.83. Each size starts five new TypeMD processes without warm-up and without forcibly clearing the Windows filesystem cache. p50/p95 uses nearest-rank, so p95 is the maximum of the five runs.

“Editable” means that the first bounded slice has been mounted, editable elements exist, and the browser has completed two consecutive animation frames. Full-document scanning and FTS indexing continue in the background and are not included in the opening time. The first screen for all five sizes mounts only 7,521 bytes of source and 128 render units.

The data is not presented as “everything is perfect”: the 1 GiB opening p95 is below the current 1.5-second target, but the combined TypeMD and WebView2 working set is still above the 400 MiB foreground target. Memory remains an open RC performance issue.

See the methodology and conclusions, all raw desktop samples, all raw kernel samples, and test-file hashes.

Overall architecture

Markdown file --> Base Source + Journal Piece Tree --> Go virtual-slice engine
                      |                                  |
                      +--> Streaming block scanner       +--> Wails adapter
                              |                                 |
                              +--> Disk block/height index      v
                              +--> Body-free FTS5          Browser virtualization core
                                                                  |
                                         +------------------------+--------------+
                                         v                                       v
                               Logical scroll + visual buffer              Bounded DOM pool
                                                                                 |
                                                                                 v
                                                                        WYSIWYG editor surface
                                                                                 |
                                                                                 v
                                                                     Optimistic source transactions

1. Disk document model

The original file is used as an immutable Base Source and opened for random reads; new text is written to an append-only Journal Source. A 64-bit Piece Tree assembles the logical file from ranges in those two Sources, so edits change only tree nodes and do not copy unchanged body text.

Tree nodes aggregate byte counts, newline counts, and Piece counts. Reads, inserts, deletes, snapshots, undo/redo, and save traversal therefore do not require a complete in-memory document. Saving traverses the snapshot of the selected revision, writes a temporary file next to the original, flushes it, and atomically replaces the original file.

2. Structure and search without caching the body

The block scanner reads bounded fragments and emits only metadata such as headings, paragraphs, lists, quotes, fences, tables, YAML, HTML, and stable checkpoints. Block metadata and estimated heights are written to a disk index; the source is released immediately after scanning.

Search uses modernc.org/sqlite FTS5 contentless word and Trigram tables and caches only inverted terms and fragment IDs. A hit is then verified by reading the candidate range from the Piece Tree, so the index never becomes a second authoritative copy of the Markdown body.

3. Two-level virtualization

The Go engine maps byte coordinates or logical heights to revisioned render slices. A single response has hard limits of 1 MiB and 512 blocks, while normal requests use smaller limits. The browser mounts only the current viewport and preload area, recycles distant units through the injected render pool, and enforces a hard DOM limit of 2,500 units.

For an oversized paragraph or continued semantic block, the renderer uses grapheme-safe visual lines and width measurement and does not write visual soft wraps into the source. The layout Worker can use OffscreenCanvas for font measurement while still processing only a bounded continued fragment each time.

Chromium never receives a full-document-height DOM. LogicalScroller maps the document onto an 8,000,000 px physical runway and rebases it near the edges. ResizeObserver updates actual heights while preserving the pixel position of the top visible stable key. Distant jumps such as search results retain a visual buffer until the target slice has completed two paint frames, reducing perceptible blank screens and layout jumps.

4. WYSIWYG transactions

React manages only tabs, sidebars, settings, and product state; it does not own the editable document tree. EditorSurface creates an imperative, bounded DOM and handles beforeinput, IME, selections, paste, commands, and optimistic edits.

All high-level operations are ultimately converted into Markdown source transactions. The browser first updates optimistically and sends batches with the expected revision; Go updates the Piece Tree, recovery journal, affected structure, and indexes. On a revision conflict, only the bounded range is resynchronized, without overwriting newer user input.

The current editing baseline deliberately uses a direct, Typora-like projection: clicking or focusing an already rendered block reveals only that block's Markdown source and syntax markers; moving the caret away immediately restores the rendered result. The rest of the document always remains formatted, and the entire document never switches to source mode. A frontend regression test locks in this behavior.

Module boundaries and current degree of decoupling

Module Responsibility Depends on desktop/UI? Current reuse method
internal/document/store Piece Tree, immutable snapshots, bounded reads No Used within the current Go module
internal/document/blocks Streaming Markdown block metadata and checkpoints No Used within the current Go module
internal/virtualization Stateless Source/Index → Slice orchestration No Used within the current Go module
internal/search, internal/indexing Body-free FTS and index generations No Wails dependency Used within the current Go module
frontend/src/editor/virtualization Scheduling, logical scrolling, measurement, buffering, DOM pool No Wails, React, or Markdown dependency Imported from source within the repository
frontend/src/editor/rendering Concrete Markdown DOM renderer Browser only Replaceable adapter
frontend/src/editor/core Input, selection, commands, source transactions Uses Bridge contracts TypeMD composition layer
internal/app, frontend/src/bridge Wails lifecycle and versioned DTOs Yes Desktop shell only

The kernel is already highly decoupled, but it is not yet a publicly released SDK. Go's internal rule prevents other repositories from importing these packages directly, and the TypeScript core has not yet been published as an npm package. External reuse currently requires moving or vendoring the source; claiming that it is already a directly installable standalone module would be inaccurate.

Using the Go core within this repository

file, _ := os.Open(path)
info, _ := file.Stat()

tree := store.New(file, info.Size())
snapshot := tree.Snapshot()

var metas []blocks.Meta
_ = blocks.Scan(ctx, snapshot, snapshot.Len(), blocks.Options{}, func(meta blocks.Meta) error {
    metas = append(metas, meta)
    return nil
})

The virtualization Engine is stateless. It only requires a bounded Range Source and block/height index that implement its small interfaces:

engine := virtualization.NewEngine()
slice, err := engine.Render(source, index, virtualization.State{
    Revision: revision,
    ByteLength: byteLength,
}, virtualization.Request{
    Anchor: "height",
    StartHeight: logicalY,
    MaxBytes: 512 << 10,
    MaxBlocks: 128,
})

Using the browser core within this repository

const controller = new VirtualDocumentController(host, {
  initialItems,
  initialLogicalHeight,
  loader: request => bridge.loadSlice(request),
  renderer: {
    key: item => item.id,
    create: item => renderUnit(item),
  },
  maximumUnits: 2_500,
})

The Loader decides where slices come from, and the Renderer decides what one unit looks like; the Controller exclusively owns request scheduling, stale response rejection, scrolling, measurement, anchoring, buffering, and recycling.

Future kernel extraction

After the public contracts stabilize, the generic Go document/store/blocks/virtualization packages and the browser virtualization package will be extracted into a separate repository. At that point, the Go internal boundary will be removed, the TypeScript package will be published, host-neutral examples will be added, and semantic versioning will be adopted. Wails, TypeMD product state, the Markdown feature UI, and desktop dialogs will remain in this repository.

Technology stack

Layer Technology
Desktop Go 1.26, Wails 2.13, Windows WebView2
Frontend React 19, TypeScript, Vite 7
Local parsing Lezer Markdown/Common
Storage and search Piece Tree, modernc.org/sqlite FTS5
File integrity fsnotify, BLAKE3, CRC32C recovery frames
Markdown capabilities DOMPurify, Turndown, KaTeX, highlight.js
Layout Intl.Segmenter, Canvas/OffscreenCanvas measurement
Tests Go testing, Vitest, and the Playwright/axe toolchain prepared for RC

Build and development

Requirements:

  • Windows 10/11 and WebView2 Runtime.
  • Go 1.26.
  • Node.js 24–26; always invoke npm.cmd on Windows.
  • Wails CLI 2.13.
  • NSIS / makensis is required only when generating the installer.
go test ./...

Set-Location frontend
npm.cmd ci
npm.cmd test
npm.cmd run build
Set-Location ..

wails dev
wails build -clean
wails build -clean -nsis

The release executable is located at build/bin/typemd.exe. To regenerate the rich Markdown corpus and reports, see the performance benchmark guide.

Current limitations

  • Windows 10/11 and UTF-8/UTF-8 BOM are the initial validated targets.
  • Opening performance has been tested up to 1 GiB, but input, prolonged scrolling, search, save, installer upgrade/uninstall, accessibility, and the extreme 1 GiB single-block case are not yet all complete.
  • The combined foreground memory of TypeMD and WebView2 currently does not meet the 400 MiB target.
  • Visual line breaking is already grapheme-safe and supports common Unicode line-breaking opportunities; complete conformance and special behavior for additional writing systems are still being improved.
  • PDF/Word export, real-time collaboration, plugins, cloud sync, and mobile are outside the scope of v1.

For more details, see the architecture description, architecture decisions, and the Windows RC checklist.

Roadmap / TODO

  • Continue refining caret placement, Chinese IME, table editing, cross-block selection, and other interaction details so that daily input consistently approaches a native editing experience.
  • Continue hardening distant fast scrolling, document-end anchoring, delayed image/font layout, and oversized single-block edge cases.
  • Reduce the combined TypeMD and WebView2 working set while preserving bounded opening latency.
  • Extract the generic Go Piece Tree, block scanning, indexing, and virtual-slice kernel into an independent public repository with stable APIs, examples, and semantic versioning.
  • Package the browser virtualization Controller as a host-independent library.
  • Complete the Windows RC accessibility, installer lifecycle, crash recovery, and fault-injection gates, then validate macOS and Linux.

Contact

If you want to contact me: moresleep512@outlook.com.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages