Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions jac/jaclang/cli/docs/internals/interop.md
Original file line number Diff line number Diff line change
Expand Up @@ -545,6 +545,8 @@ Everything crossing a marshalled boundary is **JSON** (for `cl↔sv` and the
carry graph metadata (`_jac_type`, `_jac_id`, `_jac_archetype`), and
custom objects are tagged with `__type__` -- the exact shape the client's
`__from_wire` and the server's `_deserialize_wire_args` read back.
A walker serialises to `{}` in `api_mode`: its `has` fields are the request
the caller already holds, and the `reports` list beside it is its output.
- **Envelope** -- the outer HTTP body is a `TransportResponse`:

```json
Expand Down
10 changes: 5 additions & 5 deletions jac/jaclang/cli/docs/reference/plugins/jac-scale-http.md
Original file line number Diff line number Diff line change
Expand Up @@ -1628,7 +1628,7 @@ A plain import bridges the boundary in two flavors depending on where the import

In the sv-to-sv flavor, `order_service.jac` doing `import from inventory_service { check_stock }` -- with `inventory_service` in the routes table -- does not load `inventory_service` into the consumer's process. Calling `check_stock(sku)` issues a `POST /function/check_stock` against the inventory service's URL and returns the result. The same source runs unchanged whether `inventory_service` is a separate microservice, a sibling process started by the same `jac run` command, or (when the routes entry is removed) a normal in-process import.

Both `def:pub` functions and `walker:pub` archetypes can cross the boundary. Function imports POST to `/function/<name>` and return the function's value. Walker imports POST to `/walker/<name>` and return the rehydrated walker instance with its `has` fields populated and `reports` attached, so call sites read the result the same way they would after a local spawn. See [Walker Imports](#walker-imports) for the wire shape and ergonomics.
Both `def:pub` functions and `walker:pub` archetypes can cross the boundary. Function imports POST to `/function/<name>` and return the function's value. Walker imports POST to `/walker/<name>` and return a walker instance carrying the fields you passed (provider literal defaults for the rest) and the `reports` the provider produced, so call sites read the result the same way they would after a local spawn. See [Walker Imports](#walker-imports) for the wire shape and ergonomics.

For a step-by-step walkthrough that covers project setup, running both services, and watching the round-trip, see the [Microservices tutorial](../../tutorials/production/microservices.md). The rest of this section is a reference for the discovery rules, wire contract, and plugin override surface.

Expand Down Expand Up @@ -1693,10 +1693,10 @@ What happens when the consumer evaluates `Greet(name=self.who)`:

1. The stub class collects the keyword arguments into a JSON dict (boundary-typed values are serialized via `_to_wire` first).
2. The runtime POSTs that dict to `/walker/Greet` on the resolved provider URL using the same dispatch chain as function calls (test client → registry → `JAC_SV_<MOD>_URL` → automatic spawn).
3. The provider spawns and runs the walker, then returns a `TransportResponse` envelope whose `data.result` is the executed walker as a dict and whose `data.reports` is the list of values it emitted via `report`.
4. The consumer rehydrates `data.result` into an instance of the local stub class, attaches `data.reports` as the instance's `reports` attribute, and returns it.
3. The provider spawns and runs the walker, then returns a `TransportResponse` envelope whose `data.result` is `{}` (a walker on the wire is its reports) and whose `data.reports` is the list of values it emitted via `report`.
4. The consumer builds an instance of the local stub class from the arguments it sent, filling omitted fields with the provider's literal defaults, attaches `data.reports` as the instance's `reports` attribute, and returns it. Field state the provider's walk mutated stays on the provider.

The result is a normal walker instance on the consumer: `rg.name`, `rg.reports[0]`, and `isinstance(rg, Greet)` all work. Boundary-typed values inside the walker's `has` fields and inside the `reports` list are unwrapped recursively, so a walker that emits an `obj` type comes back as that type, not as a raw dict.
The result is a normal walker instance on the consumer: `rg.name`, `rg.reports[0]`, and `isinstance(rg, Greet)` all work. Boundary-typed values inside the walker's `has` fields are rebuilt as their stub types, so a walker that takes an `obj` argument carries it back as that type, not as a raw dict.

A few notes:

Expand Down Expand Up @@ -1832,7 +1832,7 @@ Two parallel hooks let a plugin own the wire-level transport for sv-to-sv calls:

Plugins typically override both with the same auth-forwarding, tracing, retry, and circuit-breaker policy. The jac-scale plugin does exactly that: walker calls share the per-provider circuit breaker with function calls (both express provider liveness, so a tripped breaker should protect either kind), forward the inbound `Authorization` header, propagate `X-Trace-Id` across the hop, retry transport-level failures with exponential backoff, and respect the per-service `rpc_timeout` config.

Overrides for `sv_walker_call` must end by returning the rehydrated walker instance: call `stub_cls._from_wire(envelope.data.result)` and attach `envelope.data.reports` to the resulting instance's `reports` attribute. The default implementation is a useful reference and reusing `_unwrap_sv_envelope` / `_hydrate_walker_envelope` from the jac-scale source keeps error semantics consistent with the function path.
Overrides for `sv_walker_call` must end by returning the hydrated walker instance: hand the decoded response to `sv_client.hydrate_walker_envelope(data, label, args, stub_cls)`, which builds the instance from the caller's arguments plus the provider's `reports` (and returns the raw `data` payload when no stub class is given). Function overrides finish with `sv_client.function_result(data, label)`. Both raise the same `sv-to-sv ... failed` error on a non-ok envelope, so error semantics stay consistent across the two paths.

## CLI Commands

Expand Down
4 changes: 2 additions & 2 deletions jac/jaclang/cli/docs/tutorials/production/microservices.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ Both error and success cases survive the boundary intact. The `_jac_type` metada

### Walker Imports

`def:pub` is one of two shapes that can cross the service boundary; the other is `walker:pub`. A walker imported from a routes-table service becomes a remote spawn: the consumer-side stub class accepts the walker's `has` fields as keyword arguments, fires off a `POST /walker/<name>` over the wire, and returns the executed walker with its fields and `reports` populated -- the same shape you'd get from a local spawn.
`def:pub` is one of two shapes that can cross the service boundary; the other is `walker:pub`. A walker imported from a routes-table service becomes a remote spawn: the consumer-side stub class accepts the walker's `has` fields as keyword arguments, fires off a `POST /walker/<name>` over the wire, and returns a walker instance that carries the fields you passed, the provider's literal defaults for any you left out, and the `reports` the provider produced. Field state the provider's walk mutates stays on the provider; `report` is the only channel that crosses the wire.

Add a walker to `math_service.jac`:

Expand Down Expand Up @@ -228,7 +228,7 @@ curl -X POST http://localhost:8002/walker/TriggerGreet \
```

```json
{"ok":true,"type":"response","data":{"result":{"_jac_type":"TriggerGreet","_jac_id":"...","_jac_archetype":"walker","reports":[],"who":"world"},"reports":["hello, world"]},"error":null,"meta":{"extra":{"http_status":200}}}
{"ok":true,"type":"response","data":{"result":{},"reports":["hello, world"]},"error":null,"meta":{"extra":{"http_status":200}}}
```

The provider log shows the cross-service hop: `POST /walker/Greet 200`. The consumer's `Greet(name=self.who)` call site reads exactly like a local construction; the compiler swaps it for an HTTP spawn at compile time.
Expand Down
6 changes: 3 additions & 3 deletions jac/jaclang/cli/skills/jac-sv-endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,11 +82,11 @@ Every response is wrapped in a standard envelope:

```json
{"ok": true, "type": "response",
"data": {"result": <return value or executed walker>, "reports": [<report values>]},
"data": {"result": <return value, or {} for a walker spawn>, "reports": [<report values>]},
"error": null, "meta": {"extra": {"http_status": 200}}}
```

Errors flip `ok` to `false` and fill `error: {code, message}` (e.g. `UNAUTHORIZED` + `http_status: 401`). Returned archetypes carry `_jac_type` / `_jac_id` / `_jac_archetype` keys - wire bookkeeping that lets the jac client rehydrate real typed instances; raw REST consumers should read fields and ignore them.
Errors flip `ok` to `false` and fill `error: {code, message}` (e.g. `UNAUTHORIZED` + `http_status: 401`). Returned archetypes carry `_jac_type` / `_jac_id` / `_jac_archetype` keys - wire bookkeeping that lets the jac client rehydrate real typed instances; raw REST consumers should read fields and ignore them. A walker spawn answers with an empty `result`: its `has` fields are the request, `report` is the only thing it sends back, so read `data.reports`.

## @restspec - custom methods and paths

Expand Down Expand Up @@ -132,7 +132,7 @@ S3 backends and `get_url` presigning: `jac-sv-deploy`.
- Mark an endpoint `async def:pub` when its body uses `await` (external API calls, LLM endpoints), so the result is awaited rather than handed back as an unresolved coroutine.
- Give every endpoint an explicit return type - **the return type IS the wire format**. Use typed objs/nodes for domain data (the client gets dot access: `items[0].title`); an ad-hoc `dict` is fine for a one-off payload (`{"liked": True, "likes": ...}`).
- **JSON-shaped `dict` returns: don't chase the warning pair.** A bare `-> dict` draws W1036 (add type args); `-> dict[str, any]` swaps it for the noisier W1037 (explicit any disables checking). Where a heterogeneous dict is genuinely the contract, keep bare `dict` - W1036 is informational.
- **`_jac_id` is volatile** - the runtime assigns a fresh one to the walker instance and to every freshly-constructed report obj on every response (persistent node jids are stable). Strip it before hashing, caching, or diffing responses.
- **`_jac_id` is volatile** - the runtime assigns a fresh one to every freshly-constructed report obj on every response (persistent node jids are stable). Strip it before hashing, caching, or diffing responses.
- Mixed visibility in one module is normal design: an anonymous `walker:pub` (public directory, trending) sits next to authenticated plain walkers.
- Walker spawns take **keyword** arguments mapped to `has` fields (`{"title": ...}` in the body); function calls take the declared parameters. Don't pass nodes by reference across the wire - pass `jid(node)` strings.
- **404/405 on a new endpoint = its name is not in the entry module's import.** Client-side import self-registration is unreliable per-name (jac#7695): adding a `def:pub` to a module `main.jac` already imports still 405s until the new name is added there too. Name every endpoint in the entry import. Full rule: `jac-fullstack-patterns`.
Expand Down
12 changes: 6 additions & 6 deletions jac/jaclang/cli/skills/jac-sv-streaming.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
name: jac-sv-streaming
description: Streaming endpoints - SSE (server-sent events), `def:pub ... -> Generator`, `report stream()`, progress updates, live feeds, token-by-token output, sv-to-sv stream pass-through, consuming a stream in the browser with fetch + getReader. Load when an endpoint must deliver results incrementally instead of one response. Pair with `jac-sv-endpoints`, `jac-sv-microservices`.
description: Streaming endpoints - SSE (server-sent events), `def:pub ... -> Generator`, `return stream()`, progress updates, live feeds, token-by-token output, sv-to-sv stream pass-through, consuming a stream in the browser with fetch + getReader. Load when an endpoint must deliver results incrementally instead of one response. Pair with `jac-sv-endpoints`, `jac-sv-microservices`.
---

A function endpoint streams by returning a `Generator`: build a nested generator and `report` it - the ONE place a `def` uses `report` (everywhere else only walkers report). Each `yield` leaves the server as one SSE frame the moment it happens:
A function endpoint streams by returning a `Generator`: build a nested generator and `return` it. Only walkers `report`; a `def` that reports is a compile error (E1135). Each `yield` leaves the server as one SSE frame the moment it happens:

```jac
import time;
Expand All @@ -16,7 +16,7 @@ def:pub narrate(n: int) -> Generator {
yield f"chunk {i}";
}
}
report stream(); # a def that reports - streaming's one exception
return stream(); # the generator is the response
}
```

Expand All @@ -37,7 +37,7 @@ def:pub story() -> Generator {
yield str(chunk); # ...frame out; nothing is buffered
}
}
report stream();
return stream();
}
```

Expand Down Expand Up @@ -86,8 +86,8 @@ import from guestbook { story } # in main.jac, top level (server context)

## Pitfalls

- **`report stream();`, not `return stream();`** - and the outer endpoint's return type must be `Generator`, or the result is serialized as one ordinary response.
- **`return stream();` with the endpoint typed `-> Generator`** - otherwise the result is serialized as one ordinary response. A `def` that reports is a compile error (E1135); a streaming walker still does `report stream();`.
- **`data:` payloads are JSON-encoded** - `data: "chunk 0"` with quotes; `JSON.parse(line[6:])`, not the raw slice.
- Chunks may coalesce or split at arbitrary byte boundaries - always buffer and split on the blank-line separator, keeping the last partial frame for the next read.
- 404/405 on the stream URL = nothing registers it: no client-side stub reference AND no entry-module import (the registration rule above).
- Iterating without re-yielding (e.g. `list(narrate(n))`) collapses the stream into one buffered response - the gateway must itself report a generator.
- Iterating without re-yielding (e.g. `list(narrate(n))`) collapses the stream into one buffered response - the gateway must itself return a generator.
Original file line number Diff line number Diff line change
Expand Up @@ -6127,6 +6127,7 @@ impl JcirGenPass._gen_py_sv_walker_stub(
lines: list[str] = [];
lines.append('class ' + local_name + ':');
lines.append(' __jac_fields__ = ' + fields_repr);
lines.append(' __jac_field_defaults__ = ' + repr(binding.field_defaults));
lines.append(' __jac_walker_name__ = ' + "'" + walker_name + "'");
lines.append(' __jac_provider_module__ = ' + "'" + module_name + "'");
lines.append(' __jac_boundary_types__ = ' + boundary_map);
Expand Down
10 changes: 10 additions & 0 deletions jac/jaclang/compiler/driver/impl/compiler.impl.jac
Original file line number Diff line number Diff line change
Expand Up @@ -965,6 +965,15 @@ impl JacCompiler.stamp_boundary_facts(
}
}

def collect_walker_names(nd: uni.UniNode, out: list[str]) {
if isinstance(nd, uni.Archetype) and nd.sym_name and nd.arch_kind == 'walker' {
out.append(nd.sym_name);
}
for k in nd.kid {
collect_walker_names(k, out);
}
}

def imported_names(imp: uni.Import) -> set[str] {
names: set[str] = set();
for item in imp.items {
Expand Down Expand Up @@ -992,6 +1001,7 @@ impl JacCompiler.stamp_boundary_facts(
try {
collect_pub_walkers(src, walkers);
} except Exception {}
collect_walker_names(src, facts.sv_walker_names);
for name in wanted {
wk_decl = walkers.get(name);
if wk_decl is not None {
Expand Down
1 change: 1 addition & 0 deletions jac/jaclang/compiler/frontend/codeinfo.jac
Original file line number Diff line number Diff line change
Expand Up @@ -623,6 +623,7 @@ obj InteropBinding {
ret_type: str = 'int',
param_types: list[str] = [],
param_names: list[str] = [],
field_defaults: dict[str, any] = field(default_factory=`dict),
ast_node: any = None,
source_module: (str | None) = None,
kind: str = 'function';
Expand Down
22 changes: 22 additions & 0 deletions jac/jaclang/compiler/frontend/diagnostics.jac
Original file line number Diff line number Diff line change
Expand Up @@ -872,6 +872,17 @@ glob E0001 = _err("E0001", Category.SYNTAX, "Expected '{expected}', got '{got}'"
" not checked here, since another toolchain resolves it.",
blocks_codegen=True
),
E1135 = _err(
"E1135",
Category.SEMANTIC,
"`report` is only valid inside a walker, node, or edge",
help="`report` appends to the reports of the walker whose walk is running,"
" so it belongs to the archetypes that take part in a walk: a walker, or"
" a node or edge the walker visits, including their methods and impls."
" A function answers with `return`; a streaming function returns its"
" generator; spawn a walker from a function if reported values should"
" reach the caller."
),
E1301 = _err(
"E1301",
Category.SEMANTIC,
Expand Down Expand Up @@ -1961,6 +1972,17 @@ glob E0001 = _err("E0001", Category.SYNTAX, "Expected '{expected}', got '{got}'"
"Client binding for server endpoint '{name}' is async: it hands back a Promise where the server function hands back '{ret}'",
help="A server endpoint reachable from client code is bound to a client-side forwarder that calls it over HTTP, so the forwarder is necessarily async. That is transparent where the result is awaited or ignored (a command handler, an event listener) and wrong where a plain value is required (a sort comparator, a reducer). Await the call, or drop ':pub' so the function is placed client-side and stays synchronous."
),
W6010 = _warn(
"W6010",
Category.CODEGEN,
"Endpoint '{name}' returns walker '{ret}', which is empty on the wire",
help="A walker serialises to {{}} in a response: its `has` fields are the"
" request and `report` is its only output. A function that returns one"
" hands its consumers an empty object, and the reports the walker made"
" are only visible to raw callers in `data.reports`. Return the walker's"
" `reports` (or a plain obj built from them), or let the caller spawn the"
" walker itself."
),
W6008 = _warn(
"W6008",
Category.CODEGEN,
Expand Down
1 change: 1 addition & 0 deletions jac/jaclang/compiler/frontend/unitree.jac
Original file line number Diff line number Diff line change
Expand Up @@ -825,6 +825,7 @@ obj ImportBoundaryFacts {
client_native_edge: bool = False,
sv_abilities: dict[(str, Ability)] = {},
sv_walkers: dict[(str, Archetype)] = {},
sv_walker_names: list[str] = [],
na_target_decls: dict[(str, Ability)] = {},
na_unmappable_targets: list[str] = [],
native_absorb_path: (str | None) = None,
Expand Down
2 changes: 1 addition & 1 deletion jac/jaclang/compiler/passes/boundary_analysis_pass.jac
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ walker BoundaryAnalysisPass(UniPass) {
def _register_client_bridged_import(nd: Import) -> None;
def _extract_walker_has_fields(
walker_nd: (Archetype | None)
) -> tuple[list[str], list[str]];
) -> tuple[list[str], list[str], dict[str, any]];

def _find_archetype(nd: UniNode, target: str) -> (Archetype | Enum | None);
def _parse_imported_archetype(
Expand Down
Loading
Loading