Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ stopgap/vsix/
# analyzer-templates checkout, which has no license to commit it under.
studio/public/analyzers/

# The try page's analyzers as it fetches them -- written by
# studio/scripts/copy-analyzers.mjs at build time from a VisualText/analyzers
# release. Build output, and a second copy of another repository besides.
studio/public/try/

__pycache__/
.venv/

Expand Down
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ see the parse tree — with no install, no C++ toolchain, and no VS Code.
|---|---|---|
| **1. Stopgap** | `openvscode-server` container with the real `dehilster.nlp` extension baked in | **in this repo** — see [stopgap/](stopgap/) |
| **2. Studio** | Purpose-built web app: Monaco front end + an API server running the engine | **live** — in [studio/](studio/): the editor, the run server, analyzers opened from GitHub and committed back, deployed at studio.visualtext.org/studio/ for invited people |
| **2b. Try page** | A read-only page at `/try/`: run VisualText's own analyzers on your own text, with no install and no account | **in this repo** — see [docs/TRY-PAGE.md](docs/TRY-PAGE.md) |
| **Shared views** | `@visualtext/analyzer-views`: the sequence, knowledge base, output files, parse trees, code, log and values as web components, for the studio and any other page that shows an analyzer | **released** — see [analyzer-views/](analyzer-views/) |
| **3. Client-side** | Emscripten/WASM build of the engine for zero-server demos | speculative |

Expand Down Expand Up @@ -94,6 +95,30 @@ Deploying beside the phase-1 editor, at studio.visualtext.org/studio/ — behind
password, or with GitHub sign-in for invited people:
[studio/deploy/INSTALL.md](studio/deploy/INSTALL.md).

## Trying an analyzer, without an account (`/try/`)

The same build serves a second page at `/try/`: pick one of VisualText's own analyzers, type
text, press **Run**, and see the passes, the knowledge base, what the run wrote and the parse
trees — with each pass's own tree and matched rules when **Log files** is on. It is drawn with
the same `@visualtext/analyzer-views` elements as the editor, and loads no Monaco.

It is read-only. The analyzers are copied in when the site is built, from a pinned
[`VisualText/analyzers`](https://github.com/VisualText/analyzers) release, and the only thing
a visitor supplies is the text. That is what lets this page be opened to people who are not
invited while the editor is not — **the visitor never supplies code**. It is also a promise
the page has to keep: accepting a pasted grammar would put it back behind the sandbox that
[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md#the-run-server-is-not-a-sandbox) describes as not
built.

```bash
cd studio
VISUALTEXT_ANALYZERS=../../analyzers npm run analyzers # write the catalog
npm run dev # http://localhost:5173/try/
```

[docs/TRY-PAGE.md](docs/TRY-PAGE.md) has which analyzers it offers and why, which ones are
left out and why, how the catalog is built, and what to watch for when changing it.

## Quick start (phase 1)

Requires Docker and a checkout of [`vscode-nlp`](https://github.com/VisualText/vscode-nlp)
Expand Down
30 changes: 30 additions & 0 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,36 @@ With sign-in, the invited list — not a sign-up — is what decides who gets th
`NLP_STUDIO_REQUIRE_SIGN_IN=1` makes the server refuse to start rather than run open when
the proxy asks for no password.

#### Trying an analyzer without that sandbox (implemented)

`/studio/try/` (`studio/try/`, `studio/src/try/`) lets anyone who reaches the page run one of
VisualText's analyzers on their own text. It does not need the per-run sandbox above,
because it does not run the visitor's code: the analyzers are copied in at build time by
`studio/scripts/copy-analyzers.mjs` from a pinned `VisualText/analyzers` release — the same
`analyzers.zip` VisualText downloads as its examples — and the page offers no way to edit
the NLP++. The text is the only thing a visitor supplies, and it is data the rules read, not
rules the engine runs.

That is the whole of the argument, so the two halves of it have to hold:

- **The page must stay read-only.** The moment it accepts a `.nlp` file, a pasted grammar or
an analyzer named by URL, the code being run is a stranger's again and everything in *The
run server is not a sandbox* applies. `/api/run` itself still takes arbitrary `files` — the
restraint is in the page, not the server — so opening this to the internet means rate
limits in front of `/api/run` and, for anything more than these analyzers, the sandbox.
- **The grammars must be ones we would run ourselves.** They are picked by name in
`copy-analyzers.mjs`, not discovered, and each one is confirmed to run before it is listed.

Which analyzers it offers and why, which are left out and why, and what to watch for when
changing it: [TRY-PAGE.md](TRY-PAGE.md).

What it shows is the extension's own views, through `@visualtext/analyzer-views`: the pass
sequence, the knowledge base (`.dict` and `.kbb`, never `.kb`), what the run wrote, the parse
trees, and with **Log files** on — the engine's `-DEV` — each pass's own tree and the text
marked with what that pass's rules matched. The marks are made in the browser by
`ruleMatches()` over *the text that ran*, which the page keeps beside the result rather than
reading back out of the box, so editing the text after a run cannot silently move them.

### Keeping work (implemented)

Edits are saved in the browser as they are typed (`studio/src/drafts.ts`). Each analyzer has
Expand Down
Loading
Loading