Fable suggestion

[helto4real]

I asked Fable for ideas how to improve. This was just a fun query and this dores NOT mean that ND will move in this direction. I put it out here to have access to it.

—————-

NetDaemon vNext — Design Proposal

A ground-up redesign of NetDaemon: modern async-first C#, AI-agent friendly, AOT-capable, with a V5 compatibility shim. Not backwards compatible at the core, but feature complete and with a legacy API layer so existing apps recompile.

---

1. Design goals

1. One obvious way to express an automation (today there are several overlapping styles: Rx pipelines, direct state checks, scheduler callbacks).
2. Async-first: Task/ValueTask/IAsyncEnumerable as the primary model; Rx becomes an optional adapter, not the foundation.
3. Build-time everything: source generators replace the external nd-codegen tool; no runtime reflection; Native AOT publishable.
4. Machine-readable by design: the entity/service schema lives as a JSON snapshot in the repo so both the compiler and AI agents can validate against it offline.
5. Isolation: one misbehaving app can never block the event loop or other apps.
6. Testable by default: deterministic time, in-memory Home Assistant fake, no mocking frameworks needed.

---

2. New user-facing APIs

Three tiers, all built on the same core. Users pick one; they compose cleanly.

2.1 Tier 1 — Declarative fluent automations (the "minimal API" of home automation)

Inspired by ASP.NET Core minimal APIs. Ideal for 80% of automations and the easiest form for AI agents to generate correctly, because lifecycle, disposal, and threading are invisible.

var builder = HomeAutomation.CreateBuilder(args);
var home = builder.Build();
var e = home.Entities;

home.Automation("hallway_motion_light")
    .When(e.BinarySensor.HallwayMotion.TurnsOn())
    .OnlyIf(() => home.Sun.IsBelowHorizon)
    .Do(async ctx =>
    {
        await e.Light.Hallway.TurnOnAsync(brightnessPct: 80, ct: ctx.CancellationToken);
        await ctx.Delay(TimeSpan.FromMinutes(5));           // cancelled if re-triggered
        await e.Light.Hallway.TurnOffAsync(ct: ctx.CancellationToken);
    })
    .WithMode(AutomationMode.Restart)                       // HA-style modes: single/restart/queued/parallel
    .WithThrottle(TimeSpan.FromSeconds(2));

await home.RunAsync();

Key properties:

• Every automation has a name/id → traceable, listable, individually enable/disable at runtime.
• AutomationMode mirrors Home Assistant's semantics (single, restart, queued, parallel) — a concept V5 users constantly reimplement by hand with Switch()/Throttle().
• ctx carries a CancellationToken, structured trace, and helpers (Delay, WaitFor(trigger, timeout)).
• Triggers are composable values: Trigger.Any(a, b), Trigger.All(...), e.Sensor.Temp.DropsBelow(18), Trigger.Cron("0 22 * * *"), Trigger.AtSunset(offset: ...), Trigger.OnEvent("zha_event", data => ...).

2.2 Tier 2 — Class-based apps with attribute triggers

For larger apps with state and DI. Handlers are plain async methods; the runtime wires triggers via source-generated code (no reflection).

[NetDaemonApp]
public partial class ClimateManager(Entities e, IScheduler scheduler, ILogger<ClimateManager> log)
{
    [Trigger.StateChange("binary_sensor.hallway_motion", To = "on")]
    public async Task OnMotion(StateChange change, CancellationToken ct)
    {
        await e.Light.Hallway.TurnOnAsync(ct: ct);
    }

    [Trigger.NumericState("sensor.outside_temp", Below = 0, For = "00:10:00")]
    public Task OnFreezing(CancellationToken ct) => e.Switch.HeatCable.TurnOnAsync(ct: ct);

    [Trigger.Cron("0 6 * * MON-FRI")]
    public Task MorningRoutine(CancellationToken ct) => RunSceneAsync("morning", ct);

    [Trigger.HassEvent("mobile_app_notification_action")]
    public Task OnNotificationAction(HassEvent ev, CancellationToken ct) { ... }
}

• Constructor DI works exactly like today; partial lets the generator emit the registration code.
• Entity ids in attributes are validated at compile time against the schema snapshot (analyzer error ND0101: entity 'binary_sensor.hallway_motio' not found; did you mean 'hallway_motion'?).
• Optional durable state: [Persist] public int TriggerCount { get; set; } — snapshotted to disk/HA storage across restarts.

2.3 Tier 3 — Streams for power users

IAsyncEnumerable-based streams replace Rx as the low-level primitive:

await foreach (var change in e.BinarySensor.HallwayMotion.StateChangesAsync(ct))
{
    if (change.New.IsOn()) ...
}

Standard LINQ-for-async (System.Linq.Async) covers Where/Select/Throttle/Debounce needs. An NetDaemon.Reactive adapter package exposes the same streams as IObservable<T> for those who prefer Rx — but Rx is no longer a core dependency.

2.4 Entities and services — source-generated, typed end to end

• Entities, Services, typed attribute records, and typed service parameters are generated by a Roslyn source generator from .netdaemon/ha-schema.json (see §5). No external codegen tool, no generated files checked in, always in sync with the schema file.
• States are typed: LightEntity.State is LightState { OnOff IsOn, byte? BrightnessPct, ... } instead of string + attribute dictionary spelunking. Raw access remains available (entity.RawState, entity.Attributes["foo"]).
• Service calls are async and return results where HA supports response data: var forecast = await e.Weather.Home.GetForecastsAsync(...).
• Registry model is first-class: home.Areas["Kitchen"].Lights.TurnOnAsync(), labels, devices, floors — target sets (ServiceTarget) become typed collections you can query with LINQ.

---

3. Architecture redesign for maintainability

Strict layering, each layer its own package with interface seams:

NetDaemon.Transport        — websocket + REST, connection lifecycle, resilience
NetDaemon.Protocol         — HA message contracts, serialization (STJ source-gen)
NetDaemon.Model            — state cache, registry, typed domain model
NetDaemon.Runtime          — app lifecycle, trigger engine, scheduling, isolation
NetDaemon.Api              — Tier 1/2/3 user APIs (+ generators & analyzers)
NetDaemon.Testing          — in-memory HA fake, time control, assertions
NetDaemon.Compat.V5        — legacy IHaContext/HassModel shim (§6)
NetDaemon.Extensions.*     — MQTT entity manager, TTS, dashboards, persistence…
NetDaemon.Mcp / NetDaemon.Cli — agent & developer tooling (§5)

Principles:

• Dependency direction is one-way (Api → Runtime → Model → Protocol → Transport). Transport knows nothing about apps.
• Everything through Microsoft.Extensions.* idioms: DI, IOptions<T>, ILogger, IHostedService, and .NET 8+ TimeProvider everywhere — no custom scheduler abstraction to maintain, and tests get FakeTimeProvider for free.
• Extension model: INetDaemonExtension registers services, triggers, and CLI/MCP surface. MQTT entity management, Zigbee event helpers, notification engines, etc. are extensions — the core never grows integration-specific code.
• No runtime reflection: app discovery, trigger wiring, config binding, and JSON are all source-generated → trimming/AOT safe, and startup errors become compile-time errors.
• Analyzers ship with the API package: ND-prefixed diagnostics for blocking calls in handlers, missing ct propagation, unknown entity ids, service parameter typos. Compile output becomes the feedback loop (crucial for §5).

---

4. Performant networking & backend

• Transport: ClientWebSocket fed into System.IO.Pipelines; parse with Utf8JsonReader straight off the pipe (no intermediate strings/DOM). Serialization via source-generated System.Text.Json contexts.
• Use HA's subscribe_entities diff protocol instead of raw state_changed events where possible — HA sends compressed state diffs, dramatically reducing payload size and allocations on busy installations. Fall back to classic event subscription for full fidelity (contexts, event data) where needed.
• Dispatch: a single reader task materializes events into immutable records and fans out via bounded Channel<T> per app. Backpressure policy is configurable per app (drop-oldest / wait / fault). A slow or stuck app fills its own channel only; the global loop and other apps are unaffected. This replaces V5's shared Rx pipeline where one bad subscriber can stall or exception-poison others.
• State cache: immutable snapshots — FrozenDictionary rebuilt on registry changes, copy-on-write for state updates. Reads are lock-free; an automation always sees a consistent snapshot of the world at trigger time (passed in ctx.Snapshot).
• Resilience: Microsoft.Extensions.Resilience (Polly v8) pipelines for reconnect with jittered backoff; sequence-number gap detection triggers a state resync; triggers get a ConnectionRestored signal so apps can re-evaluate.
• Deployment: Native AOT publish target → add-on container measured in tens of MB with sub-second cold start and a fraction of V5's memory footprint; regular JIT build remains the default for dynamic-compilation users.

---

5. AI-agent friendliness (first-class, not bolted on)

The theme: make the whole system legible offline and verifiable in a tight loop.

1. Schema snapshot in the repo — .netdaemon/ha-schema.json: entities, attributes, services with parameter schemas, areas, devices, labels. Pulled with nd schema pull. This single file gives an agent complete knowledge of the home without a live connection, powers the source generator, and makes builds deterministic.
2. Compile-time validation as the agent feedback loop — unknown entity, wrong service param, missing await: all become ND#### diagnostics with fix-it suggestions. An agent's edit→build→read-errors loop converges fast because errors are specific and actionable.
3. CLI (nd) — nd new app, nd schema pull, nd validate, nd test, nd simulate --scenario scenarios/evening.yaml (drives the in-memory fake with scripted state changes and asserts outcomes), nd trace <automation-id>.
4. MCP server (NetDaemon.Mcp) — exposes tools: list_entities, get_state, search_schema, validate_project, run_simulation, get_automation_trace, tail_logs. An agent in Claude Code/Codex can inspect the home, write an automation, simulate it, and read the trace without a human in the loop.
5. Automation traces — every trigger execution records a structured JSON trace (trigger, conditions evaluated, service calls, timings, exceptions), like HA's automation traces. "Why didn't the light turn on?" becomes a queryable artifact instead of log archaeology.
6. Project template ships AGENTS.md + skills — conventions (one automation per file, naming, where schema lives, how to run nd validate) documented in the format agents actually consume, plus a SKILL.md for the nd workflow.
7. One obvious way — the Tier 1 API is deliberately small and regular; less API surface variance means fewer plausible-but-wrong generations.

---

6. Legacy compatibility layer (NetDaemon.Compat.V5)

Goal: existing V5 apps recompile without source changes against the new core.

• Implements IHaContext, Entity/NumericEntity, StateChange, IHaRegistry, INetDaemonScheduler, AppModel attributes ([NetDaemonApp], [Focus]), and yaml/JSON config injection — all as facades over NetDaemon.Model/Runtime.
• Entity.StateChanges() returns IObservable<StateChange> bridged from the per-app channel via the Rx adapter; scheduler maps onto TimeProvider-based scheduling.
• The V5 code generator's output shape (Entities, Services, LightEntities…) is reproduced by a compat mode of the new source generator, so generated-code-dependent apps also work.
• Known semantic differences (e.g., dispatch ordering, error isolation now per-app) documented in a migration note; a compat option UseSharedDispatchOrdering = true preserves V5 ordering for apps that depend on it.
• Migration tooling: nd migrate runs Roslyn codemods for mechanical rewrites (Rx Subscribe → attribute trigger or await foreach, Throttle → WithThrottle, scheduler calls → [Trigger.Cron]), and flags what needs a human. Compat layer is a permanent supported package, not a temporary bridge — but new features land only in the new API.

---

7. Feature-completeness checklist vs V5

V5 capability	vNext equivalent
Rx StateChanges()/StateAllChanges()	IAsyncEnumerable streams + Rx adapter + compat shim
Generated Entities/Services	Source generator from schema snapshot
CallService / service response data	Typed async service calls with results; raw CallServiceAsync escape hatch
Scheduler / cron	Trigger.Cron, Trigger.At, TimeProvider-based IScheduler
Sun/sunrise/sunset helpers	home.Sun + sunset/sunrise triggers with offsets
Events (HassEvent)	Trigger.HassEvent + typed event payload binding
App config (yaml)	IOptions<T> binding (yaml/json/env), source-generated binding
MQTT entity manager extension	NetDaemon.Extensions.Mqtt on the extension model
TTS extension	NetDaemon.Extensions.Tts
App enable/disable via HA	Runtime app registry exposed as HA entities (per automation, not just per app)
Registry (areas/devices/labels/floors)	First-class typed registry with LINQ
Testing (community fakes)	Official NetDaemon.Testing with HA fake + FakeTimeProvider

---

8. Suggested phasing

1. Phase 1: Transport + Protocol + Model with subscribe_entities, channels, snapshot cache; prove perf targets.
2. Phase 2: Runtime + Tier 3 streams + source generator + analyzers; NetDaemon.Testing.
3. Phase 3: Tier 1 fluent + Tier 2 attributes; CLI + schema snapshot workflow.
4. Phase 4: Compat.V5 + nd migrate; run the existing integration-test app suite through the shim as the acceptance gate.
5. Phase 5: MCP server, traces, simulation scenarios, AGENTS.md/skill templates.

[helto4real]

this is the spec it wrote :)

spec.md