Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 118 additions & 0 deletions .cursor/skills/functional-dsl/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
---
name: functional-dsl
description: Design DSLs the functional way — as pure data structures and composable functions, with the denotation (meaning) defined before syntax and effects isolated to interpreters. Apply when proposing a new DSL, reviewing an existing one, or modeling agent workflows, tool plans, and program graphs. Use the Zen and review criteria as hard gates.
---

# Functional DSLs

In functional programming (FP), language design isn't viewed as building a parser—it's viewed as algebraic design. FP design principles treat a DSL as a set of pure data structures and composable functions.

## The Zen of Functional DSLs

If you synthesized the FP community's core tenets into a Zen of Functional DSLs, it would read like this:

- Programs are data; interpreters are functions.
- Make illegal states unrepresentable.
- Parse, don't validate.
- Design the denotation (meaning) before the syntax.
- Decouple the domain algebra from its execution.
- If it doesn't terminate, it's a general-purpose language, not a DSL.
- Composition over execution.

Apply these as hard review criteria when proposing a new DSL or revising an existing one.

## 1. Conal Elliott: Denotational Design

Conal Elliott (pioneer of Functional Reactive Programming) advocates for designing software starting purely from mathematical semantics.

- **Read:** Denotational Design: From Meanings to Programs
- **Philosophy:** Don't start with AST nodes, classes, or parser rules. Start with the abstract mathematical meaning of your domain.
- **Example:** if you're building an animation DSL, a "Movement" is simply a pure function `Time -> Location`. Once you define the math, your DSL syntax and operations naturally fall out of standard mathematical laws (like Monoids and Functors).

**Practice**

- Write the semantic type of each domain concept first.
- List the laws (identity, associativity, composition).
- Only then invent syntax / host APIs that inhabit those meanings.

## 2. Alexis King: "Parse, Don't Validate"

Alexis King's famous 2019 essay is considered required reading across the functional programming landscape.

- **Read:** Parse, Don't Validate
- **Philosophy:** Never check data with booleans (validation) and pass raw data downstream. Instead, write a parser that transforms loose, raw input into a tight, domain-specific type. By pushing validation into the parsing phase, invalid states become unrepresentable by construction in the rest of your DSL engine.

**Practice**

- Prefer `Result<DomainAst, ParseError>` at the boundary
- Ban "validated flag" / parallel boolean checks deeper in the stack
- Use branded / refined types for IDs, amounts, tool names, etc.

## 3. Oleg Kiselyov: Typed Tagless Final Embedding

Oleg Kiselyov introduced one of the most influential patterns for constructing embedded DSLs.

- **Read:** Typed Tagless Final Interpreters
- **Philosophy:** Traditional DSLs build an explicit AST (Initial Encoding) and then traverse it. "Tagless Final" expresses the DSL as a set of functions or typeclasses instead. This lets you write code in your DSL once, and instantly run multiple pluggable interpreters against it (e.g., an evaluator, an optimizer, a type checker, or a printer) without modifying the DSL grammar or re-traversing trees.

**Practice**

- Define a capability interface / typeclass for the algebra
- Implement multiple interpreters: eval, pretty-print, simulate, optimize
- Prefer final encoding when you need many backends; prefer initial/Free when you need introspection, serialization, or time-travel

## 4. Gabriel Gonzalez: Total Functional Programming & Dhall

Gabriel Gonzalez created Dhall, a functional, programmable configuration language built to replace YAML and JSON without allowing infinite loops or security exploits.

- **Read:** Dhall Language Design & Philosophy
- **Philosophy:** Total Programming. A great functional DSL should guarantee termination (it is deliberately not Turing-complete). Users should be able to write functions, imports, and abstractions in the DSL, but the host environment must be 100% guaranteed that the DSL script will never hang, crash, or access the file system directly.

**Practice**

- For agent config / policy / skill metadata: prefer total languages
- Keep effects in the host interpreter, not in the config DSL
- Recursion, arbitrary loops, and ambient I/O are host concerns

## 5. Rúnar Bjarnason & Paul Chiusano: Algebra & Free Monads

Co-authors of *Functional Programming in Scala* ("The Red Book"), their work popularized using algebraic structures and Free Monads to construct DSLs.

- **Read:** Functional Programming in Scala (Manning)
- **Philosophy:** A program is just a data structure. You write a DSL by defining an "Algebra" (a set of pure data constructors representing operations). Building a script in your DSL doesn't execute anything; it merely builds an immutable description of intent. You then pass that data structure to a separate, isolated interpreter function that performs the actual computation or side effects.

**Practice**

```
Algebra (pure constructors)
-> Program value (immutable description)
-> Interpreter(s) (effects, optimization, tracing)
```

This maps cleanly to agent workflows, tool plans, and Effect-style program graphs.

## Design workflow (use this order)

1. **Denotation** — mathematical meaning / types / laws
2. **Algebra** — operations as data or typeclass methods
3. **Parse** — raw input → domain types (no boolean validation later)
4. **Interpreters** — eval, optimize, explain, dry-run, execute
5. **Syntax last** — host fluent API or external surface that reveals the algebra

## Mapping onto repos in context

| Repo | Functional-DSL lens |
|---|---|
| Effect | Programs-as-values; interpreters / layers; composition over execution |
| Smithers | Durable workflow as data + observable interpreters |
| Composio / Treg | Tool algebras; parse tool schemas; separate auth/execution |
| Centaur | Tools/workflows/skills as plugins over a control-plane interpreter |
| Mastra / Agno | Agent platform algebras; keep domain intent separate from runtime |

## Review questions

- Can illegal states be constructed?
- Is meaning defined before syntax?
- Can the same program run under eval / dry-run / optimize interpreters?
- Does the DSL guarantee termination where it should?
- Are effects isolated to interpreters?
117 changes: 117 additions & 0 deletions .cursor/skills/languages-as-libraries/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
---
name: languages-as-libraries
description: How to ship a derived language, typed layer, or dialect as a library over an existing host toolchain instead of forking the compiler — reusing scoping, namespaces, modules, and analysis. Apply when embedding DSLs, adding typed layers, or designing extensible language/plugin systems. Based on "Languages as Libraries" (Tobin-Hochstadt, St-Amour, Culpepper, Flatt, Felleisen; PLDI'11) and the Racket extension model.
---

# Languages as Libraries

Based on *Languages as Libraries* (Tobin-Hochstadt, St-Amour, Culpepper, Flatt, Felleisen; PLDI'11) and the Racket extension model.

## Thesis

Programming language design benefits from constructs for extending the syntax and semantics of a host language. The goal is not only a reusable VM — it is an extensible host language that supports linguistic reuse so derived languages can reuse scoping, namespaces, modules, and compilers.

A derived language should be able to:

- reuse host scoping mechanisms
- lift host namespace management into the experimental language
- manipulate surface syntax and AST
- interpose new context-sensitive static semantics
- communicate static results to the backend
- ship as a library with no host-compiler fork

## Guy Steele's growth principle

> I need to design a language that can grow. — Guy Steele, 1998

Growing a language requires more than a reusable virtual machine and libraries; it demands extension mechanisms across phases of language implementation. Racket shows that with enough extension surface, even a sophisticated typed sister language can be a library.

## Racket extension arsenal (patterns to reuse)

### Macros

Macros are functions from syntax → syntax, run at compile time. Prefer hygienic macros so generated binders do not capture user code.

Use macros for:

- notational shorthands
- embedding DSLs
- attaching out-of-band metadata (types, effects, docs)

### Syntax objects

Treat host ASTs as first-class values with:

- constructors / accessors
- source locations
- syntax properties for out-of-band communication (types, annotations) without breaking host forms

### Local expansion

Expand user code to a small fixed core language before analysis. This lets a typechecker / optimizer understand programs written with arbitrary macros without cataloging every extension.

**Rule: reduce sugar → core forms → analyze core forms.**

### Modules as language choice

Each module specifies its language (e.g. `#lang typed/racket`). A language L is a library providing:

- bindings for the base environment (forms + values)
- a whole-module hook (`#%module-begin`) for context-sensitive module semantics

This is the key move: **language choice is per module, not per process.**

## Typed Racket as the reference architecture

Typed Racket demonstrates the full stack as libraries:

- Annotate bindings with types (syntax properties on reused `define` / `λ`)
- Context-sensitive whole-module typechecking via module begin
- Typecheck an extensible language by expanding to core first
- Persist types across separate compilation (emit compile-time declarations)
- Safe typed↔untyped linking via contracts generated from types
- Type-driven source-to-source optimization before the host backend

### Challenges this solves

| Challenge | Library technique |
|---|---|
| Types on untyped binding forms | syntax properties |
| Whole-module context | module-begin wrapper |
| Macros / unknown sugar | local-expand to core |
| Separate compilation | residualize type env into compiled module |
| Untyped interop | contracts at boundary; skip checks typed↔typed |
| Optimization | rewrite using validated types + unsafe specialized ops |

## Design recipe (apply outside Racket)

When adding a dialect / skill language / typed layer on TS/Python/Rust hosts:

1. **Reuse the host compiler** — do not fork
2. **Define a tiny core IR** — expand/desugar everything into it
3. **Attach metadata out-of-band** — attributes, JSDoc/TS types, Rust attributes, decorators
4. **Whole-unit analysis hook** — file/module/plugin transform
5. **Boundary contracts** — protect invariants when crossing into untyped/untrusted code
6. **Optimize by rewriting** once static info is proven

## Interop principles

- Typed modules exporting to untyped clients need dynamic checks
- Typed↔typed should avoid redundant checks
- Macros / generative code that escapes typed modules can break invariants — gate or contract them
- Separate compilation must rehydrate static environments

## Relation to other skills

- **dsl** — decide internal vs external; languages-as-libraries is the strongest path for internal DSLs
- **functional-dsl** — denotation/algebra first; this skill is how to embed that algebra into a real host toolchain
- **meta-minimal-languages** — keep the host small/stable so libraries (languages) can evolve faster than the core

## Review checklist

- Can this language ship as a package?
- Is the core IR small enough to analyze?
- Are host binding/scoping rules reused rather than reimplemented?
- Is static info persisted across compilation units?
- Are untrusted boundaries contracted?
- Are optimizations library rewrites, not compiler forks?
131 changes: 131 additions & 0 deletions .cursor/skills/meta-minimal-languages/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
---
name: meta-minimal-languages
description: Design principles for keeping a platform's core language/runtime small and stable while letting layered libraries evolve as de-facto "languages". Apply when designing agent/AI platforms, tool ABIs, plugin systems, or any framework at risk of growing into a giant mega-language. Based on "Programming Language Requirements for the Next Millennium" (Griswold, Wolski, Baden, Fink, Kohn).
---

# Meta-Minimal Languages

Based on *Programming Language Requirements for the Next Millennium* (Griswold, Wolski, Baden, Fink, Kohn).

## Problem: the new world disorder

Technology change is continuous. Application software is increasingly performance-limited and must evolve quickly as problems and methods change. Less time is available per problem. Performance is sensitive to platform; shared / locally managed resources mean applications must adapt dynamically to contention and to evolving data structure during execution.

## Lesson from High Performance Fortran (language-only approach)

HPF tried to provide portable data-parallel abstractions with array-centric directives. It struggled because:

- sparse / irregular / evolving data were poorly served
- best performance needs task parallelism and instruction-level locality, not only data parallelism
- escape hatches (e.g. HPF `LOCAL`) couple performance tuning to partition directives, so retargeting becomes non-trivial
- the domain and architectures changed so fast that a widely acceptable frozen language definition remained elusive
- without a stable definition, commercial optimizing compilers are hard; with a huge language, retargeting optimizers is hard

**Takeaway:** large, ambitious domain languages freeze poorly under rapid change.

## Lesson from application-specific libraries

Scientists responded with special-purpose libraries in traditional languages + message passing. Advantages:

- high-level and retargetable
- application knowledge enables optimized abstractions
- easier to enhance with new algorithms than to extend a compiler
- source availability allows sophisticated users to tailor behavior

Successful libraries often share structure:

- **Open layering** — generic layer + specialized application layers
- **Limited interoperability** — e.g. layout/communication library + Fortran numeric kernels
- **Separate performance-tuning interface** — tell the library what data pattern to expect, or adapt at load/run time

Limits of libraries alone:

- cannot automatically detect stylized usage that should be optimized (users call composite routines or extend the library)
- historically weak at sensing runtime environment contention
- inter-library interoperability fails when data presentation assumptions diverge

## Solution: meta-minimal languages

Combine the best of languages and layered libraries:

> a small, stable programming language with abstraction features that support the development of self-tuning, optimizing, easily adaptable, integrable layered systems

Why small and stable?

- **small** → optimizing compiler can be retargeted quickly to new platforms
- **stable** → library/language ecosystems can invest
- **sophisticated abstractions** → performance interfaces and self-adapting types can be defined in libraries
- library uses should be optimizable like language primitives
- blur programming / compiling / executing — share information across those phases
- provide migration paths for old code

### Implication

There may be no single standard high-performance application language — only a language for defining high-level libraries. Those libraries become the de facto "standard languages" for coalitions facing the same problem.

## Required metalanguage features

### 1. Meta-level / extensibility for libraries

Support adding features and optimization directives for those features (cf. OpenC++ / MPC++ meta-object protocols):

- feature usage patterns that trigger specialized implementations
- compile-time mechanisms (avoid mandatory runtime MOP overhead)
- first-class performance interfaces / performance objects
- optional: algebraic specs for layer-level optimization and static checks (helpful, but not enough alone for rich performance interfaces)

Challenge: keep the metalanguage itself small and stable (C++'s long, expansive standardization is a cautionary tale).

### 2. Explicit layering

Layers are not classes. A layer is a virtual machine: complementary combination of data abstractions, control structures, performance parameters, event handlers, etc.

- modules can encode layers, but classes are the wrong unit
- layers and information-hiding modules serve different purposes and do not necessarily nest
- ignoring this makes evolution hard and performance suffer — especially when users ≠ developers
- named layering abstractions help library-level optimizers

### 3. Event support for runtime adaptation

Runtime sensors (e.g. Network Weather Service-style contention tracking) should integrate without awkward polling. Prefer proactive event propagation upward (network → scheduler) with flexible event mechanisms rare in traditional languages.

### 4. Export dataflow info to the runtime

Compile-time optimization makes static guesses; runtime systems often lack application structure. Preserve dataflow / communication estimates for dynamic decisions (e.g. which task to migrate under load imbalance).

## Design rules for modern agent / AI platforms

Translate the paper into today's stack:

| Paper idea | Modern mapping |
|---|---|
| Small stable core language | Tiny IR / runtime kernel (sandbox, tool ABI, workflow core) |
| Layered libraries as "languages" | Tools, skills, workflows, adapters as layered packages |
| Performance interface | Separate knobs for cost, latency, model choice, caching — not mixed into every domain call |
| Event / sensing | Runtime telemetry, load, rate limits, tool health → adaptive scheduling |
| Optimize stylized usage | Trace/plan optimizers that rewrite tool graphs |
| Interoperability | Explicit data contracts between toolkits; avoid hidden format assumptions |
| Don't freeze a mega-language | Prefer stable kernel + evolving library coalitions |

## Anti-patterns

- Growing one giant "agent language" that tries to anticipate every domain
- Putting tuning flags into every business-level API
- One opaque top layer with no safe lower layers for escape/performance
- Layers that are only organizational folders, not real virtual-machine boundaries
- Compilers/runtimes that throw away structure the scheduler needs

## Relation to other skills

- **dsl** — libraries that feel like languages are often internal DSLs over a small core
- **functional-dsl** — algebras + interpreters are how layered libraries stay optimizable
- **languages-as-libraries** — the implementation strategy for shipping those layers without forking the kernel

## Review checklist

- Is the core small enough to retarget/maintain?
- Can new domain methods appear as new layers without core changes?
- Is there a first-class performance / adaptation interface?
- Can the runtime sense environment + application structure?
- Are interoperability contracts explicit?
- Are we accidentally freezing a mega-language?
Loading
Loading