Khmer Segmenter Web Demo

A static browser demo for Khmer word segmentation and experimental spelling assistance. The segmentation and typo-candidate engine run in a Web Worker through the Rust library compiled to WebAssembly.

The flat light/dark color palette follows the Typsastra visual theme: neutral paper surfaces, restrained forest accents, lime highlights, and no gradients. The Typsastra wordmark and favicon link to the canonical assets in the Typsastra repository.

Features

• live Viterbi segmentation while typing;
• a frequency-ranked autocomplete dropdown anchored to the typing caret and updated independently of the slower full-document analysis pass;
• one spelling-error treatment for both unknown spans and probable typos;
• correction suggestions for every invalid span, with prefix-ranked dictionary fallbacks when conservative fuzzy matching finds no close edit;
• whole-span Khmer corrections shown after hovering over an underlined word, with right-click available as an explicit alternative;
• one-click replacement without sending text to a server;
• segmented view and result download;
• selectable typing, document, and high-recall spellcheck profiles;
• selectable lexical or visual COENG DA/TA spelling policy;
• safe ZWSP word-break insertion and runtime loading of custom KDIC v2 packs.

Spelling correction is experimental. Suggestions are deterministic dictionary candidates, not context-aware predictions, and must be reviewed before use. The bundled model uses supplemental short entries only to improve segmentation; spellcheck, correction suggestions, and autocomplete remain restricted to the curated authoritative vocabulary. The local v0.2 demo pack combines RAC with the reviewed geography, science, economics, law, and MPTC terminology layers. It also includes the 126-entry pilot community layer and corpus-informed segmentation costs. Community entries can preserve names, borrowed words, and newer terms as segments, but they remain invalid for spellcheck and autocomplete unless a curated application dictionary accepts them. Autocomplete and full-document analysis use separate workers so a new-word completion request is not blocked by segmentation or spellchecking. The completion worker is warmed during initialization so the first typed prefix does not pay the WebAssembly cold-start cost. The demo uses precision-first diagnostics: it inspects OOV regions only, limits automatic edits to a low Khmer-weighted distance, and requires at least 0.80 confidence. It calls the shared typing profile used by Python, native Rust, and WASM. Applications can use document for an explicit full-document check; the high-recall profile remains experimental and is intended for evaluation and dictionary curation rather than live underlines. Use the arrow keys to move through the dropdown and Tab or Enter to accept the highlighted result. Hover briefly over a red-underlined word to preview its corrections, or right-click it to keep the correction panel open explicitly.

Khmer editor behavior

• Backspace immediately after a COENG + consonant subscript deletes the pair as one editing unit.
• Inserting COENG before existing Khmer text adds a temporary ZWNJ placeholder, preventing accidental attachment to the following consonant.
• Typing a consonant immediately before or after that placeholder consumes the ZWNJ and creates the intended subscript.
• ZWSP and ZWNJ remain real Unicode characters in the text but are displayed as small circles—purple for ZWSP and amber for ZWNJ—so users can navigate around them deliberately. Hovering a marker reveals its Unicode name.
• Clipboard paste removes every ZWSP before inserting text while preserving ZWNJ and visible Khmer content.

Run locally

WebAssembly modules must be served over HTTP rather than opened with a file:// URL.

On Windows, double-click start_demo.cmd. It starts a local server, opens the correct page automatically, and requires no package installation. Keep its terminal window open while using the demo.

Alternatively, start the server manually:

python -m http.server 8000

Then open http://localhost:8000.

Run the checked-in WASM smoke test with:

node test/wasm_smoke.mjs
node test/autocomplete_browser_smoke.mjs

Rebuild the Rust WebAssembly package

From the khmer_segmenter/port/rust directory:

wasm-pack build --target web \
  --out-dir ../../../khmer_segment_webui_demo/wasm \
  . --no-default-features --features wasm

Build the compact KDIC from the main repository's prepared local data:

python scripts/build_dictionary_kdict.py \
  --output ../khmer_segment_webui_demo/data/khmer_dictionary.kdict

The checked-in local demo currently uses the optional v0.2 inclusive pilot. First overlay the locally reviewed official lexicons onto the released RAC pack. Each command uses the previous output as its base so source provenance and lexical policy remain embedded:

khmer-segment data compile dataset/lexicon/generated/geography-lexicon.klex.json \
  --base port/rust/data/khmer_dictionary.kdict --output build/web-official-1.kdict
khmer-segment data compile dataset/lexicon/generated/science-lexicon.klex.json \
  --base build/web-official-1.kdict --output build/web-official-2.kdict
khmer-segment data compile dataset/lexicon/generated/economic-lexicon.klex.json \
  --base build/web-official-2.kdict --output build/web-official-3.kdict
khmer-segment data compile dataset/lexicon/generated/law-lexicon.klex.json \
  --base build/web-official-3.kdict --output build/web-official-4.kdict
khmer-segment data compile dataset/lexicon/generated/mptc-lexicon.klex.json \
  --base build/web-official-4.kdict --output build/web-official-5.kdict

Then add the reviewed community segmentation layer and corpus-informed costs:

python scripts/blend_corpus_frequencies.py \
  --base-kdict build/web-official-5.kdict \
  --community-kdict community/panhapich_khmer_text_corpus.kdict \
  --corpus-frequencies dataset/community/Panhapich-khmer-text-corpus/known_frequencies.json \
  --corpus-weight 0.2 \
  --output-klex dataset/community/Panhapich-khmer-text-corpus/web_inclusive.klex.json \
  --output-kdict ../khmer_segment_webui_demo/data/khmer_dictionary.kdict

BUILD_INFO.json pins the source commit, corpus revision, configuration, and SHA-256 hashes of the generated WASM and KDIC artifacts.

Data attribution

The compiled dictionary derives from Seanghay Hay's Khmer Dictionary 44k and is redistributed for non-commercial use with attribution, as confirmed by the dataset author. Frequency weights are generated by the Khmer Segmenter project from segmentation output. See the main project's data documentation before redistributing a build. The demo's specific notice is in DATA_LICENSE.md.