From a472b188848ad7be080c9eecd8a1fc1eaa4d69af Mon Sep 17 00:00:00 2001 From: EHotwagner Date: Tue, 21 Jul 2026 12:34:47 +0200 Subject: [PATCH] docs(M3-Net): add acquisition, quick-start exchange, and a docs/ guide Bring FS.GG.Net's README to the M2 consumer standard: a product-page shape (title/one-liner, what-it-can-do, Acquire, Quick start, Go deeper, Where this sits) with build/test/internals below the fold. Acquire names the nuget.org entry package (FS.GG.Net.Core) and the .WebSocket/.Server/.Protobuf/.Grpc/.Elmish set, credential-free (ADR-0039). Quick start is a copy-pasteable code-first gRPC exchange with a visible result. Add docs/getting-started.md: install + package map, and the two samples/* exchanges (code-first gRPC in-process, SC2 WebSocket handshake) promoted into a first-class walkthrough. Closes #10. FSGG-Worker: merlin-4263 --- README.md | 122 +++++++++++++++----- docs/getting-started.md | 239 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 331 insertions(+), 30 deletions(-) create mode 100644 docs/getting-started.md diff --git a/README.md b/README.md index d832b59..a6bcc93 100644 --- a/README.md +++ b/README.md @@ -1,17 +1,89 @@ # FS.GG.Net -A render-independent, domain-neutral **transport** component for the FS-GG platform — a bottom-layer sibling to `FS.GG.Game` and `FS.GG.Audio` that *reaches up to nothing*. It carries protobuf messages over a WebSocket or a gRPC channel, and knows no `.proto` and no game types: the message schemas live in the applications that consume it. +A render-independent, domain-neutral transport component for .NET apps that need to carry protobuf messages over a WebSocket or a gRPC channel. -Decision record: **[ADR-0052](https://github.com/FS-GG/.github/blob/main/docs/adr/0052-onboard-fs-gg-net-transport-component.md)**. +## What it can do + +- Carry **protobuf request/response exchanges** over a WebSocket, with a pluggable correlation strategy (single-in-flight lockstep, or id-multiplexed). +- Drive a **raw-protobuf-over-WebSocket** protocol such as StarCraft II's `sc2api` — with an id-echo desync guard that turns a lost or misordered reply into an explicit error instead of a stale result. +- Stand up **code-first gRPC** services and clients from plain F# records and an interface — no `.proto` to author or vendor. +- Serve the same typed exchange from the **server side** (a Kestrel WebSocket acceptor), reusing one symmetric `ITransport` seam on both ends. +- Model connection lifecycle uniformly as a single `ConnectionState`, and drive it from **Elmish** via a `Cmd`/`Sub` idiom. + +## Acquire + +Every `FS.GG.*` package is **public on [nuget.org](https://www.nuget.org) and +restores with no credential** ([ADR-0039][adr-0039]). Add the entry package: + +```sh +dotnet add package FS.GG.Net.Core +``` + +`FS.GG.Net.Core` is the seams; add the pieces your transport needs alongside it — +`FS.GG.Net.WebSocket`, `FS.GG.Net.WebSocket.Server`, `FS.GG.Net.Protobuf`, +`FS.GG.Net.Grpc`, `FS.GG.Net.Elmish`. The full package map is in +[the packages reference](docs/getting-started.md#the-packages). + +## Quick start + +A code-first gRPC exchange — no `.proto`. Define the contract as F# records plus an +interface, then compose the client from three pieces (`FS.GG.Net.Grpc` opens the +channel, `FS.GG.Net.Protobuf` registers the records, protobuf-net.Grpc types the proxy): + +```fsharp +open System +open System.ServiceModel +open System.Threading.Tasks +open ProtoBuf +open ProtoBuf.Grpc +open ProtoBuf.Grpc.Client +open FS.GG.Net.Grpc +open FS.GG.Net.Protobuf + +// The wire contract — plain F# records + a service interface, no .proto. +[] +type GreetRequest = { [] Name: string } +[] +type GreetReply = { [] Message: string } + +[] +type IGreeter = + [] + abstract SayHello: request: GreetRequest * context: CallContext -> ValueTask + +// Register the records once, open a channel, get a typed client, call it. +Registration.records [ typeof; typeof ] +let channel = GrpcTransport.connect (Uri "http://localhost:5000") +let greeter = channel.CreateGrpcService() + +let reply = greeter.SayHello({ Name = "World" }, CallContext.Default).Result +printfn "SayHello -> %s" reply.Message +printfn "channel state via FS.GG.Net.Grpc: %A" (GrpcTransport.stateOf channel) +``` -## Why it exists +Against the matching server you should see: -Two first uses share **protobuf serialization** but not transport: +``` +SayHello -> Hello, World! +channel state via FS.GG.Net.Grpc: Connected +``` -| Use case | Transport | Correlation | Package path | -|---|---|---|---| -| Custom **StarCraft II** client ↔ SC2 headless server | raw protobuf over **WebSocket** (`ws://…/sc2api`) | `Sequential (Some idEcho)` — single in-flight, id-verified (`Request`/`Response` field 97) | `Core` + `WebSocket` + `Protobuf` | -| Custom **Beyond All Reason** client ↔ owned gRPC proxy | **gRPC** | native (grpc-dotnet) | `Core` + `Grpc` | +The complete, runnable server + client (and the SC2-over-WebSocket exchange) are the +two walkthroughs in [the getting-started guide](docs/getting-started.md). + +## Go deeper + +- [`docs/getting-started.md`](docs/getting-started.md) — install, then run either exchange (code-first gRPC in-process, or the SC2 WebSocket handshake) end to end. +- Sibling bottom-layer components: [FS.GG.Game](https://github.com/FS-GG/FS.GG.Game) and [FS.GG.Audio](https://github.com/FS-GG/FS.GG.Audio). + +## Where this sits + +FS.GG.Net is a bottom-layer transport sibling to `FS.GG.Game` and `FS.GG.Audio` that reaches up to nothing — it knows no `.proto` and no game types. See the +[platform vocabulary (ADR-0020)](https://github.com/FS-GG/.github/blob/main/docs/adr/0020-platform-workspace-component-vocabulary.md) +and [`docs/architecture.md`](https://github.com/FS-GG/.github/blob/main/docs/architecture.md) +for how the whole platform fits together. + +--- ## The two-tier seam @@ -20,34 +92,24 @@ The abstraction is two tiers, not "WebSocket vs gRPC as sibling transports": - **`ITransport`** — an ordered duplex channel of *complete application messages*. WebSocket implements it (fragment reassembly is internal). gRPC does **not** — it owns its own framing and sits above this tier. - **`IMessageChannel<'Req,'Resp>`** — a typed protobuf request/response exchange with a pluggable `Correlation` strategy. The piece with no off-the-shelf .NET equivalent, and the SC2 substrate. -The core unifies only the **`ConnectionState` lifecycle** and (via `FS.GG.Net.Elmish`) the **`Cmd`/`Sub` idiom** — deliberately small, so the version can stay stable. +The core unifies only the **`ConnectionState` lifecycle** and (via `FS.GG.Net.Elmish`) the **`Cmd`/`Sub` idiom** — deliberately small, so the version can stay stable. The per-package responsibilities are the table in [the getting-started guide](docs/getting-started.md#the-packages). -## Packages (coherent set on `$(FsGgNetVersion)`) - -| Package | Responsibility | -|---|---| -| **FS.GG.Net.Core** | Pure, BCL-only seams: `ConnectionState`, `ITransport`, `IMessageCodec`, `IdEcho`, `Correlation`, `IMessageChannel`, and a working `Sequential` correlator. | -| **FS.GG.Net.WebSocket** | `ITransport` over `ClientWebSocket` (client `connectAsync` + `ofSocket` for any open socket) — fragment reassembly, pooled buffers, initial connect-retry. The SC2 substrate. | -| **FS.GG.Net.WebSocket.Server** | Server-side transport: a Kestrel acceptor (`WebSocketServer.start`) that hands each connection to a handler as an `ITransport`. Pair with `MessageChannel.serve` to serve a protobuf-over-WS protocol. | -| **FS.GG.Net.Protobuf** | `IMessageCodec` for Google.Protobuf (`ToByteArray`/`MessageParser` — the reproducible raw path SC2-over-WS needs) and protobuf-net (code-first, owned schemas). Encodes the F#/protobuf-net registration gotchas absorbed from `fsGRPCSkills`. | -| **FS.GG.Net.Grpc** | A thin lifecycle bridge over grpc-dotnet — projects a channel's connectivity onto `ConnectionState`. Does **not** re-abstract gRPC method dispatch. The BAR substrate. | -| **FS.GG.Net.Elmish** | `Net.Cmd.exchange` / `Net.Sub.incoming` over an `IMessageChannel`. Depends on standard Elmish, never on FS.GG.UI. | - -## Build & test +## Building & contributing ```bash dotnet build # net10.0, warnings-as-errors, locked/deterministic restore dotnet test # FS.GG.Net.Core.Tests (Expecto) — the Sequential correlator + desync guard ``` -## Server side +The `ITransport` seam is symmetric, so the server reuses it: `FS.GG.Net.WebSocket.Server` +accepts a connection and hands it over as an `ITransport`, and `MessageChannel.serve` is the +inverse of the client's `Exchange`. What FS.GG.Net does **not** own is the rest of "server +infrastructure" — auth, TLS, deployment, orchestration — which belongs to the application host, +not the transport component. -The `ITransport` seam is symmetric, so the server reuses it: `FS.GG.Net.WebSocket.Server` accepts a -connection and hands it over as an `ITransport`, and `MessageChannel.serve` is the inverse of the -client's `Exchange` — it reads inbound requests, runs a handler, and sends the (id-echoed, via -`ServerEcho`) response. What FS.GG.Net does **not** own is the rest of "server infrastructure" — auth, -TLS, deployment, orchestration — which belongs to the application host, not the transport component. - -## Still an org-level follow-up +Decision record: **[ADR-0052](https://github.com/FS-GG/.github/blob/main/docs/adr/0052-onboard-fs-gg-net-transport-component.md)**. +The wire-contract registry dimension (vendored external `.proto`, owned `.proto`, code-first +surface) is tracked there and lands via the SDD-first schema-growth sequence +([SDD#589](https://github.com/FS-GG/FS.GG.SDD/issues/589)). -- The **wire-contract registry dimension** (vendored external `.proto`, owned `.proto`, code-first surface) is tracked in ADR-0052; it lands via the SDD-first schema-growth sequence ([SDD#589](https://github.com/FS-GG/FS.GG.SDD/issues/589)). +[adr-0039]: https://github.com/FS-GG/.github/blob/main/docs/adr/0039-nuget-org-is-the-read-path.md diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 0000000..2e8da41 --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,239 @@ +# Getting started with FS.GG.Net + +FS.GG.Net carries protobuf messages over a WebSocket or a gRPC channel, and nothing +above that: it knows no `.proto` and no game types — the message schemas live in the +application that consumes it. This guide installs the packages and then walks two +**real, runnable** exchanges end to end: + +1. [A code-first gRPC exchange](#walkthrough-1-a-code-first-grpc-exchange) — runs + entirely in-process, no external server, so you can run it from a clone in one terminal. +2. [The StarCraft II WebSocket handshake](#walkthrough-2-the-starcraft-ii-websocket-handshake) + — a full game handshake against Blizzard's headless SC2 server, over raw protobuf on a WebSocket. + +Both exist as buildable samples in this repo under [`samples/`](../samples); this guide +is the narrated path through them. + +## Install + +Every `FS.GG.*` package is **public on [nuget.org](https://www.nuget.org) and restores +with no credential** ([ADR-0039][adr-0039]) — no `--add-source`, no org feed, no token. +Start from the entry package and add the pieces your transport needs: + +```sh +dotnet add package FS.GG.Net.Core +``` + +### The packages + +`FS.GG.Net` ships as a coherent set. Take `FS.GG.Net.Core` plus whichever transport and +serialization pieces your app uses: + +| Package | Add it when you need | Responsibility | +|---|---|---| +| **FS.GG.Net.Core** | always (the seams) | Pure, BCL-only seams: `ConnectionState`, `ITransport`, `IMessageCodec`, `IdEcho`, `Correlation`, `IMessageChannel`, and a working `Sequential` correlator. | +| **FS.GG.Net.WebSocket** | a client talking protobuf over a WebSocket | `ITransport` over `ClientWebSocket` (`connectAsync`, or `ofSocket` for any open socket) — fragment reassembly, pooled buffers, initial connect-retry. The SC2 substrate. | +| **FS.GG.Net.WebSocket.Server** | serving that protocol | A Kestrel acceptor (`WebSocketServer.start`) that hands each connection to a handler as an `ITransport`. Pair with `MessageChannel.serve`. | +| **FS.GG.Net.Protobuf** | encoding/decoding messages | `IMessageCodec` for Google.Protobuf (the reproducible raw path SC2-over-WS needs) and protobuf-net (code-first, owned schemas). Absorbs the F#/protobuf-net registration gotchas. | +| **FS.GG.Net.Grpc** | a gRPC channel | A thin lifecycle bridge over grpc-dotnet — projects a channel's connectivity onto `ConnectionState`. Does not re-abstract gRPC method dispatch. | +| **FS.GG.Net.Elmish** | driving it from an Elmish app | `Net.Cmd.exchange` / `Net.Sub.incoming` over an `IMessageChannel`. Depends on standard Elmish, never on FS.GG.UI. | + +## Walkthrough 1: a code-first gRPC exchange + +This is the "others develop their own client" story — a service defined **once** as F# +records plus an interface (no `.proto`), hosted on ASP.NET Core, and consumed by a client +that composes the FS.GG.Net pieces. It runs entirely in-process, which is why it is the +runtime coverage for `FS.GG.Net.Grpc` in CI. The full sample is +[`samples/GrpcService`](../samples/GrpcService). + +Packages: `FS.GG.Net.Grpc`, `FS.GG.Net.Protobuf`. + +### 1. Define the wire contract + +The contract is plain F# — records tagged for protobuf-net, and a service interface. This +*is* the wire contract a third party generates a client against; there is no `.proto` to +vendor. + +```fsharp +namespace GrpcService.Contracts + +open System.Collections.Generic +open System.ServiceModel +open System.Threading.Tasks +open ProtoBuf +open ProtoBuf.Grpc + +[] +type GreetRequest = { [] Name: string } +[] +type GreetReply = { [] Message: string } +[] +type CountRequest = { [] To: int } +[] +type Tick = { [] N: int } + +[] +type IGreeter = + [] + abstract SayHello: request: GreetRequest * context: CallContext -> ValueTask + [] + abstract CountTo: request: CountRequest * context: CallContext -> IAsyncEnumerable + +module Contract = + let recordTypes: System.Type list = + [ typeof; typeof; typeof; typeof ] +``` + +> **F#/protobuf-net gotcha:** register each record before serializing (the next step does +> that), and prefer `array` over `list` and `Dictionary` over `Map` on the wire. + +### 2. Compose the client + +The client is pure FS.GG.Net composition — three existing pieces, no gRPC reimplementation: + +```fsharp +open System +open ProtoBuf.Grpc +open ProtoBuf.Grpc.Client +open FS.GG.Net.Grpc +open FS.GG.Net.Protobuf +open GrpcService.Contracts + +// The sample server is plaintext HTTP/2 (h2c) for zero-config local runs. +AppContext.SetSwitch("System.Net.Http.SocketsHttpHandler.Http2UnencryptedSupport", true) + +Registration.records Contract.recordTypes // FS.GG.Net.Protobuf — the F# registration +let channel = GrpcTransport.connect (Uri "http://localhost:5000") // FS.GG.Net.Grpc — channel + ConnectionState +let greeter = channel.CreateGrpcService() // protobuf-net.Grpc — the typed proxy (not ours) + +let reply = greeter.SayHello({ Name = "World" }, CallContext.Default).Result +printfn "SayHello -> %s" reply.Message +printfn "channel state via FS.GG.Net.Grpc: %A" (GrpcTransport.stateOf channel) +``` + +### 3. Run it + +From the sample, one terminal for the server and one for the client: + +```bash +# Terminal 1 — the server (prints its ephemeral address): +dotnet run --project samples/GrpcService/GrpcService.Server + +# Terminal 2 — the client, given that address: +dotnet run --project samples/GrpcService/GrpcService.Client -- http://localhost: +``` + +Expected output: + +``` +SayHello -> Hello, World! +CountTo 3: + tick 1 + tick 2 + tick 3 +channel state via FS.GG.Net.Grpc: Connected +``` + +The server is ASP.NET Core + protobuf-net.Grpc — FS.GG.Net does not wrap it. Auth (JWT/mTLS), +TLS, streaming policy, and deployment are deliberately not here: they belong to the +application host, not the transport component. + +## Walkthrough 2: the StarCraft II WebSocket handshake + +This drives a full game handshake against Blizzard's headless SC2 server, entirely over +raw protobuf on a WebSocket: + +``` +Ping → CreateGame → JoinGame(raw) → Observation → Step → Observation → LeaveGame → Quit +``` + +Every step is a correlated protobuf `Request`/`Response` over a WebSocket — +`WebSocketTransport` + `Codec.google` + `MessageChannel` with `Sequential (Some idEcho)`. +SC2's `Request`/`Response` both carry `id` (proto field 97), so the id-verified correlator's +desync guard is live: a lost or misordered response surfaces as a `CorrelationMismatch` +instead of a silently stale observation. The full sample is +[`samples/Sc2Handshake`](../samples/Sc2Handshake). + +Packages: `FS.GG.Net.WebSocket`, `FS.GG.Net.Protobuf` (Google.Protobuf codec). + +### The shape of one exchange + +`MessageChannel.create` wraps a transport with codecs and a correlation strategy; each +`Exchange` sends one request and awaits its correlated response: + +```fsharp +open System +open System.Threading +open SC2APIProtocol // message types generated by Grpc.Tools over the vendored s2clientprotocol .proto +open FS.GG.Net.Core +open FS.GG.Net.Protobuf +open FS.GG.Net.WebSocket + +// SC2's id lives on Request/Response field 97 — an IdEcho makes it the correlator's desync guard. +let idEcho: IdEcho = + { Stamp = fun (r: Request) id -> r.Id <- uint32 id; r + Read = fun (r: Response) -> uint64 r.Id } + +let run (uri: Uri) = + task { + let! transport = WebSocketTransport.connectAsync uri WebSocketOptions.defaults CancellationToken.None + let channel = + MessageChannel.create + transport + (Codec.google Request.Parser) + (Codec.google Response.Parser) + (Sequential (Some idEcho)) + + let! pong = channel.Exchange(Request(Ping = RequestPing()), CancellationToken.None) + printfn "ping -> %A" pong.Status + } +``` + +### Running against a real SC2 server + +There is no live SC2 in CI, so the sample **builds everywhere** but **runs** only against a +real install. It works with the same headless package the +[aiarena](https://github.com/aiarena/aiarena-docker-base) image wraps: + +```bash +# 1. Get Blizzard's SC2 Linux headless package (the password IS the EULA acceptance). +wget http://blzdistsc2-a.akamaihd.net/Linux/SC2.4.10.zip +unzip -P iagreetotheeula SC2.4.10.zip -d ~/sc2/ +ln -sfn ~/sc2/StarCraftII/Maps ~/sc2/StarCraftII/maps # SC2 resolves maps under lowercase maps/ on Linux + +# 2. Launch the headless server (opens ws://127.0.0.1:5000/sc2api). +cd ~/sc2/StarCraftII/Versions/Base75689 +./SC2_x64 -listen 127.0.0.1 -port 5000 -dataDir ~/sc2/StarCraftII/ -tempDir /tmp/sc2/ & + +# 3. Run the client (port, then a map path relative to Maps/). +dotnet run --project samples/Sc2Handshake/Sc2Handshake -- 5000 "Ladder2019Season1/CyberForestLE.SC2Map" +``` + +Expected tail: + +``` +observation -> status=InGame game_loop=0 raw_units=161 observation_payload=46225 bytes +step -> status=InGame +observation2 -> status=InGame game_loop advanced 0 -> 112 +quit -> status=Quit +OK — full game handshake over FS.GG.Net ... against a REAL SC2 server. +``` + +> **proto2 note:** SC2's proto is proto2 — an unset `optional` enum reads back as its +> *first* value, so check `HasError` (not an enum comparison) before trusting +> `ResponseCreateGame.Error`. + +## Serving the other side + +The `ITransport` seam is symmetric, so a server reuses it. `FS.GG.Net.WebSocket.Server` +accepts a connection and hands it over as an `ITransport`, and `MessageChannel.serve` is the +inverse of the client's `Exchange`: it reads inbound requests, runs a handler, and sends the +id-echoed response. What FS.GG.Net does **not** own is the rest of "server infrastructure" — +auth, TLS, deployment, orchestration — which belongs to the application host. + +## Where to go next + +- [`README.md`](../README.md) — the component overview and the two-tier seam. +- [ADR-0052](https://github.com/FS-GG/.github/blob/main/docs/adr/0052-onboard-fs-gg-net-transport-component.md) — why FS.GG.Net exists and the decisions behind it. +- [`docs/architecture.md`](https://github.com/FS-GG/.github/blob/main/docs/architecture.md) and the [platform vocabulary (ADR-0020)](https://github.com/FS-GG/.github/blob/main/docs/adr/0020-platform-workspace-component-vocabulary.md) — how the whole platform fits together. + +[adr-0039]: https://github.com/FS-GG/.github/blob/main/docs/adr/0039-nuget-org-is-the-read-path.md