Implement tryAs

CHANGELOG.md

@@ -9,6 +9,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 
+- `tryAs` checked cast: `<value> tryAs <type>` validates the value against the
+  type at runtime and pushes a `Maybe` — `just value` when it conforms, `none`
+  when it does not. Unlike `as` (a static-only checker hint), `tryAs` is meant
+  for data crossing a trust boundary such as `parseJson` output: one check at
+  the boundary and everything downstream is precisely typed. Compose with the
+  existing `?` unwrap for a die-loud form, e.g.
+  `"pkgs.json" parseJson tryAs Manifest ? :packages? (:name?) map`.
+  A `tryAs` whose source type could never be the target is a type error.
+  Validation depth is limited to 1024 nested levels; exceeding it (a cyclic
+  value, a cyclic `type` declaration, or absurdly deep data) fails the script
+  with a clear error rather than producing `none`.
+
 - CLI completions for `cargo`: subcommands (including installed third-party
   ones via `cargo --list`), per-subcommand options, and dynamic values for
   `--target`, `--features`, `-p`/`--package`, `--bin`/`--example`/`--test`/`--bench`

ai/json_boundary_casts.md

@@ -0,0 +1,118 @@
+# Checked casts at the JSON boundary (`tryAs`)
+
+Status: implemented on branch `try-as` (2026-07-09). Doc pattern + `tryAs`
+both landed. One finding from implementation: `parseJson` maps every JSON
+number to `MShellFloat` (Go's default unmarshal), so `tryAs {str: int}` on
+JSON counts is always `none` — validate JSON numbers as `float`. Whether
+`parseJson` should produce `int` for integral numbers is an open follow-up.
+
+## Problem
+
+Drilling into parsed JSON under the type checker is painful.
+Extracting `.packages[].name` took four nested `match` blocks,
+because `parseJson` returns the wide union `list|dict|numeric|str|bool|null`
+and every union member must be handled at every level.
+Each `match dict:` arm proves a fact, uses it once, and throws the proof away —
+boilerplate scales with nesting depth.
+
+## What already works (and the doc gap)
+
+The static side of the fix already exists:
+
+```mshell
+type Manifest = {packages: [{name: str}]}
+
+"pkgs.json" parseJson as Manifest :packages? (:name?) map
+```
+
+This type-checks and runs today. `type_system.inc.html` even recommends the
+pattern ("prefer adding a named type and an `as` assertion near the boundary"),
+but `doc/mshell.md` — the doc agents actually read — never shows
+`parseJson ... as T`. **Cheapest immediate action: document this pattern in
+`doc/mshell.md`.**
+
+## The gap: `as` is static-only
+
+`as` is a checker hint with no runtime work (`Evaluator.go`, `MShellAsCast`
+case). When the JSON doesn't match the assertion, the failure surfaces late
+and badly: `{"packages": [{"name": 42}]}` through the cast above dies with
+`Cannot get length of a Float.` at a downstream call site, not at the boundary.
+
+## Design: `tryAs` — parse, don't validate
+
+One new checked cast that returns proof as a more precise type:
+
+```mshell
+"pkgs.json" parseJson tryAs Manifest      # ( -- Maybe[Manifest]), recoverable
+"pkgs.json" parseJson tryAs Manifest ?    # unwrap-or-die, composes with existing ?
+```
+
+- Runtime: structural walk of the value against the reified type expression.
+  JSON values are only dicts/lists/scalars, so the validator is a small
+  recursive function and structural checking is complete in this domain.
+- Success: `just` wrapping the value, statically typed `T`.
+  Downstream `:field?` / `map` need no further matching.
+- Failure: `none`.
+
+### Naming rationale
+
+- `as?` rejected: `?` consistently means *unwrap* a Maybe (`?`, `:field?`),
+  so a Maybe-*returning* `as?` reads backwards.
+- `as!` rejected: `!` means *store to variable* (`name!`).
+- `tryAs` follows the getter convention: base word returns Maybe,
+  existing `?` unwraps. Die-loud form is derived (`tryAs T ?`), not a
+  second primitive.
+
+### Division of labor with static `as`
+
+Both stay; the dividing line is trust boundaries:
+
+- `as` — value originates inside typed code: empty/ambiguous literals
+  (`[] as [str]`), narrowing inferred unions, branding. Checker can see
+  everything; runtime check would verify the statically obvious. Free.
+- `tryAs` — value crosses in from outside (parseJson, process output,
+  spreadsheets). Only the runtime can know the shape; pay one walk,
+  get a `Maybe[T]` proof.
+
+Possible follow-on: once `tryAs` exists, a hint diagnostic when bare `as`
+narrows an external-data union (e.g. the parseJson result) to a shape —
+"did you mean tryAs?" — while leaving literal-hint uses untouched.
+
+## Deferred (deliberately)
+
+- **Path-precise error messages.** `tryAs ... ?` failing as a bare `none`
+  loses ".packages[3].name: expected str, got float". A future Result type
+  with an error string is the planned home for this; the validator computes
+  the message internally anyway, so a Result-returning variant just exposes
+  it. The "no message on bad unwrap" problem is bigger than JSON and is not
+  being solved here.
+- **`dig` with typed default** (`json ['packages' 0 'name'] "unknown" dig`,
+  signature `(json [str|int] a -- a)`, default's type = result type via
+  existing generics). Nice for one-off drills without declaring a type.
+  If built: no `'*'` wildcard segments (they change the result shape and
+  break `a -- a`); `tryAs` + `map` owns extract-many. Conflates missing
+  with the sentinel value.
+- **Recursive types** (`type Json = int | str | [Json]`) don't work today —
+  the name isn't in scope inside its own body. A first-class "any JSON"
+  type would need checker work; with checked casts at the boundary it's
+  rarely needed.
+
+## Implementation notes
+
+Structure (after review feedback on exhaustiveness): there is no separate
+validator file. `TypeExpr.go` defines a `TypeExpression` interface
+(`MShellParseItem` + `validateObj`) that every type-expression node must
+implement, and the parser productions return it — so a new node kind
+without runtime validation does not compile. Built-in named types (bytes,
+null, path, datetime, Grid, GridView, GridRow) live in one
+`builtinNamedTypes` table consulted by BOTH the checker's resolveTypeExpr
+and the runtime validator, so a type cannot be half-added; a name missing
+from the table does not exist statically either. `Maybe` (parametric) and
+`none` (constructor, not a type) are deliberately special-cased outside
+the table.
+
+- Type declarations are currently erased at runtime (`MShellTypeDecl` is
+  static-only). `tryAs` needs type expressions reified into the evaluator
+  so the validator can walk them.
+- The shape language already covers everything JSON needs: shapes, optional
+  fields (`name?: T`), unions, `null` vs `none`, `Maybe`.

ai/tryas_review.md

@@ -0,0 +1,119 @@
+# tryAs review — outstanding items
+
+Critical review of the `try-as` branch (`7dce9f5 Implement tryAs`), 2026-07-09.
+The unbounded-recursion crashes found in this review are already **fixed** on the
+branch (depth budget of 1024 in `validateObj`, see `maxTryAsValidateDepth` in
+`mshell/TypeExpr.go`, with regression tests in `tests/fail/tryas_cyclic_type.msh`,
+`tests/fail/tryas_cyclic_value.msh`, `tests/fail/tryas_depth_exceeded.msh`, and
+`tests/success/tryas_depth.msh`). Everything below is still open and needs a
+decision.
+
+## 1. The recursive-type contradiction (design decision needed)
+
+The runtime and the checker disagree about whether recursive named types exist.
+
+- Runtime: `tryAs Name` resolves names late through `state.typeDefs`, so
+  `type Tree = {value: int, kids?: [Tree]}` validates finite values correctly.
+  Verified: `type A = [B]` / `type B = [A]` works at runtime.
+- Checker: `CheckProgram` pre-pass 1 resolves each `type` body in file order
+  *before* declaring the name, so any self- or forward-reference is
+  "unknown type" — recursive types are statically unrepresentable.
+
+Consequence: recursive types (the natural way to type JSON trees, which is the
+flagship `tryAs` use case) work only when you *don't* pass `--check-types`.
+The two halves should agree. Options:
+
+- **(a)** Register all type names before resolving bodies so the checker admits
+  recursion. The arena/TypeId side needs a story for the cycle (a `TidNamed`
+  indirection or iterative resolution). Runtime is already safe now that the
+  depth budget exists. This is the option that makes the flagship use case work.
+- **(b)** Make the runtime reject what the checker rejects: resolve/validate the
+  type AST eagerly at `type` declaration time and fail on unknown names. Loses
+  recursive types entirely.
+
+## 2. JSON integer footgun passes the checker silently (high)
+
+`parseJson` produces a `float` for every JSON number, so any `int` inside a
+`tryAs` target is **always-none** against JSON data — and `--check-types` says
+nothing. Verified:
+
+```mshell
+"[1, 2]" parseJson tryAs [int]    # always none; checker silent
+```
+
+The docs warn about this (good), but the branch already added a checker error
+for statically-impossible casts (`5 tryAs str`); this is the same bug class and
+the one users will actually hit, since `{count: int}` is what everyone writes
+first. Options:
+
+- Checker warning when the cast source is the JSON output union and the target
+  contains `int` anywhere.
+- Accept integral floats for `int` in `validateObj` — rejected in the branch's
+  own comments because validation deliberately mirrors `match` type-arm
+  semantics; changing it here without changing `match` creates a new
+  inconsistency.
+- Make `parseJson` produce `int` for integral numbers — biggest change, fixes
+  the root cause, affects existing scripts that assume float.
+
+## 3. Wildcard shapes validate nothing at runtime (medium)
+
+`TypeShapeExpr.validateObj` never reads `a.Wildcard`. Verified inconsistency:
+
+```mshell
+{"a": 1, "b": "x"} tryAs {a: float, *: float}   # just  — wildcard ignored
+{"a": 1, "b": "x"} tryAs {*: float}             # none  — dict form checks values
+```
+
+For static `as` dropping the wildcard was harmless (width subtyping already
+allows extra keys — comment at the `TypeShapeExpr` declaration in
+`mshell/TypeExpr.go`), but `tryAs` makes runtime claims: a user who writes
+`*: float` is asking for enforcement they silently don't get. Options:
+
+- Validate non-field keys against the wildcard type in
+  `TypeShapeExpr.validateObj` (my preference: it's what the syntax says).
+- Reject wildcard shapes as `tryAs` targets with a clear error (fail closed).
+
+Either is better than the current silent half-validation.
+
+## 4. Quotation targets are half-open (medium)
+
+`tryAs (int -- int)` checks only that the value is a quotation
+(`TypeQuoteExpr.validateObj`) — signatures are not verified, so
+`("hi" wl) tryAs (int -- int)` produces `just`, and the checker afterward
+trusts `Maybe[(int -- int)]` for a value with a different signature. That is a
+static-soundness hole shaped like a checked cast. It is documented, but
+consider failing closed instead: reject quote types in `tryAs` targets with an
+error ("quotation signatures cannot be verified at runtime"), rather than
+issuing a certificate the runtime can't back. If kind-only checking is kept,
+it stays a documented sharp edge.
+
+## 5. Union arm errors are value-dependent (low)
+
+`1 tryAs int | Bogus` never reports the unknown type `Bogus` (first arm
+matches, short-circuit); a value that matches no arm hits the error instead.
+So whether a malformed type expression is diagnosed depends on the data. A
+one-time eager walk of the type AST at cast time (validating all names/arity)
+would make the errors deterministic. Cheap to do now; confusing bug reports
+later if skipped.
+
+## 6. Notes, no action required
+
+- `validateObj`'s `case Maybe:` (value, non-pointer) branch is dead code — the
+  codebase only ever pushes `*Maybe`. Harmless.
+- The `just` result retains the original mutable object; a post-validation
+  mutation through an alias (`append` on a still-reachable inner list) can
+  invalidate the certified type. Inherent to the language's aliasing
+  semantics, not specific to this branch; maybe worth one sentence in the
+  docs next to "parse, don't validate".
+- Lexer edges verified good: `tryas`, `tryAsX`, `tryA`, `trying` all stay
+  LITERAL; bare `tryAs` and `Maybe`-without-argument produce clean parse
+  errors.
+- `builtinNamedTypes` table (single source for checker resolution + runtime
+  check) and the `TypeExpression` interface forcing `validateObj` on every
+  node kind are both good structure — keep that pattern for future node kinds.
+- Test-coverage suggestions once decisions above land: wildcard-shape
+  behavior (whichever way it goes), a quotation-target case, nested
+  `Maybe[Maybe[T]]`, and a `tryAs` inside a quotation in `tests/success/`
+  (works today — verified `(tryAs int ...) map` — but only top-level uses are
+  covered, and quotations dispatch through the separate `evaluateItems` path
+  in `mshell/Evaluator.go`).

doc/mshell.md

@@ -564,6 +564,34 @@ type Row = [Cell]
 { "name": "Ada", "age": 36 } as Person :age? 1 +
 ```
 
+`as` is a static-only assertion: a hint to the checker with no runtime work.
+Use it when the value originates inside typed code and the checker just needs help — typing an empty literal (`[] as [str]`), narrowing an inferred union, or tagging a literal into a named type.
+
+`tryAs` is the checked cast for data crossing a trust boundary (parseJson output, process output, spreadsheet rows).
+It validates the value against the type at runtime and pushes a `Maybe`: `just value` when it conforms, `none` when it does not.
+This is the "parse, don't validate" pattern: one check at the boundary, and everything downstream is precisely typed with no match boilerplate.
+Compose with the existing `?` unwrap for the die-loud form.
+
+```mshell
+type Manifest = {packages: [{name: str}]}
+
+# Recoverable: handle bad input with one match at the boundary.
+"pkgs.json" parseJson tryAs Manifest match
+    just m : @m :packages? (:name?) map,
+    none   : [],
+end
+
+# Die-loud: unwrap immediately when malformed input should stop the script.
+"pkgs.json" parseJson tryAs Manifest ? :packages? (:name?) map
+```
+
+A `tryAs` whose source type can never be the target (e.g. `5 tryAs str`) is flagged by the checker, since it would always produce `none`.
+Shape validation allows extra keys, requires all non-optional fields, and validates optional fields only when present.
+A `none` conforms to `Maybe[T]` for every `T`; quotation types are checked by kind only (signatures are not verified at runtime).
+Named types must be declared before the cast runs.
+Note that `parseJson` produces a `float` for every JSON number, so validate JSON numbers as `float`, not `int`.
+Validation depth is limited to 1024 nested levels; exceeding it (a cyclic value, a cyclic `type` declaration, or absurdly deep data) fails the script with a clear error rather than producing `none`.
+
 Dictionary types are split into homogeneous dictionaries and shapes.
 A homogeneous dictionary is for dynamic keys where every value has the same type.
 In a type expression, write `{str: int}`.
@@ -1060,7 +1088,7 @@ end wl # Output: 11
 - `gridValues`: Extract Grid or GridView cell values as row-major lists, without a header row and without coercing cell types. (`Grid|GridView -- [[a]]`)
 - `toCsvCell`: Escape a single CSV cell. If the value contains `,`, `"`, or a newline, wraps the value in double quotes and doubles any embedded quotes; otherwise returns the input unchanged. (`str -- str`)
 - `toCsv`: Serialize a list of rows to a CSV string. Each cell is escaped with `toCsvCell`, cells are joined with `,`, and rows are joined with `\n`. (`[[str]] -- str`)
-- `parseJson`: Parse JSON from a string, binary, or file path into mshell objects. JSON `null` becomes the `null` type (distinct from `none`). (`path|str|binary -- list|dict|numeric|str|bool|null`)
+- `parseJson`: Parse JSON from a string, binary, or file path into mshell objects. JSON `null` becomes the `null` type (distinct from `none`); every JSON number becomes a `float`. Narrow the result with `tryAs` (checked, boundary data) or `as` (static hint). (`path|str|binary -- list|dict|numeric|str|bool|null`)
 - `parseExcel`: Parse an `.xlsx` (OOXML) spreadsheet into a list of sheets in workbook (tab) order. Each sheet is a dict with a `name` key (the worksheet name), a `data` key holding a rectangular list of rows (list of lists), a `hidden` key (bool; `true` for hidden or veryHidden sheets), and a `visibility` key (`"visible"`, `"hidden"`, or `"veryHidden"`). Cell values are typed: numbers become floats (dates appear as Excel serial floats), strings become strings (shared, inline, and formula-string results all resolved), booleans become booleans, error cells (e.g. `#DIV/0!`) become `none`, and empty/padding cells are the empty string. Chartsheets are skipped; hidden worksheets are included. Dates are returned as raw Excel serial floats; apply `fromOleDate` at the call site to convert. `parseExcel` assumes the default 1900-based date system, which matches `fromOleDate`'s OLE epoch (1899-12-30). Workbooks saved with the 1904 date system (`<workbookPr date1904="true"/>`, seen on some files originally authored on older Mac Excel or with the "Use 1904 date system" option enabled) have serials offset by 1462 days; on those files, add 1462 to each serial before calling `fromOleDate`, e.g. `@wb :0: :data? :3: :0: 1462 + fromOleDate`. (`path|binary -- list`)
 - `seq`: Generate a list of integers, starting from 0. Exclusive end to integer on stack. `2 seq` produces `[0 1]`. `(int -- [int])`
 - `repeat`: Create a list containing the provided value repeated `n` times. `(a int -- [a])`