A small language where tensors are the primitive, grad is built in,
and a shape mistake is an error you see before the program runs.
Most machine-learning code is a language plus a numeric framework bolted on top. twill goes the other way. Tensors are the built-in data type, differentiation is a language operation rather than a library call, and a static checker reads your shapes before anything executes.
Here is the whole of it. This prices a European call by Monte Carlo and gets its delta and vega by differentiating the pricer, with no bumping and no second library:
seed(42)
let Z = randn(200000) # fixed shocks: the price is smooth in its inputs
fn call_price(S0, K, r, sigma, T) {
let drift = (r - 0.5 * sigma * sigma) * T
let ST = S0 * exp(drift + sigma * sqrt(T) * Z) # simulated terminal prices
exp(-r * T) * mean(relu(ST - K)) # discounted expected payoff
}
let price = call_price(100.0, 100.0, 0.05, 0.2, 1.0)
let delta = grad(fn(s) = call_price(s, 100.0, 0.05, 0.2, 1.0))(100.0)
let vega = grad(fn(v) = call_price(100.0, 100.0, 0.05, v, 1.0))(0.2)$ twill examples/montecarlo_option.tw
European call, S0=100 K=100 r=5% vol=20% T=1y, MC paths: 200000
price = 10.442696 (Black-Scholes 10.4506)
delta = 0.636269 (Black-Scholes 0.6368)
vega = 37.488476 (Black-Scholes 37.524)
grad went through 200,000 simulated paths, a relu payoff and a mean, and
landed on the closed-form Greeks. No tape object, no requires_grad, no
.backward(). The full program is examples/montecarlo_option.tw.
This is an early prototype; the current release is v1.12.0, and as of v1.4.0 the twill compiler written in twill runs on the Go bootstrap and reproduces the reference across every stage (see twill is being written in twill). The reference implementation is a single Go binary with no dependencies, so it is quick to build.
How fast it is, where the time goes and how it compares against PyTorch on the
same mathematics are measured in docs/BENCHMARKS.md.
docs/CORRECTNESS.md is the evidence for grad and for
the checker.
- Shape errors, before the program runs
- Why
- Install
- Run
- Differentiation
- The language in a few lines
- Units of measure
- Tensors and operations
- The standard library
- What twill is built out for
- twill is being written in twill
- What is not done yet
- Repository layout
- Documentation
- License
The single most useful thing twill does is refuse to start. twill check infers
tensor shapes across the whole program and reports the ones that cannot line up:
$ twill check bad.tw
bad.tw:3: shape error: shape mismatch in @: [2, 3] @ [2] (inner 3 != 2)
3 | let y = A @ x
Function parameters can carry shape annotations, which turn a contract into something the checker enforces at every call site:
fn matvec(A: [3, 2], x: [2]) -> [3] {
A @ x
}Break that contract and you get the mistake at the call and at the line it breaks, both before anything runs:
$ twill check model.tw
model.tw:6: shape error: argument 2 ("x") axis 0 is 3 but the signature expects 2
6 | let out = matvec(A, [1.0, 2.0, 3.0])
model.tw:2: shape error: shape mismatch in @: [3, 2] @ [3] (inner 2 != 3)
2 | A @ x
Use [2] for a vector, [3, 2] for a matrix, [] for a scalar, and _ for a
dimension you do not want to pin down. A dimension can also be a name, a shape
variable: a name used more than once must be the same size, which is what ties
shapes together across a signature and lets the checker verify the return type of
fn mm(A: [n, k], B: [k, m]) -> [n, m].
The checker only flags a mismatch when it is certain. Code whose shapes depend on runtime values is left alone rather than guessed at, so a clean run means what it says:
$ twill check examples/shapes.tw
examples/shapes.tw: no shape problems found
Autodiff as a runtime library, shapes known only once you run, and a layer of glue between the math and the program are all consequences of the same thing: the framework arrived after the language. twill is an experiment in the other direction, a language built around differentiable tensor programs from the start. Three things fall out of it.
Tensors are the primitive. Every number is a rank-0 tensor, vectors and
matrices are literals, and @ is matrix multiply. Broadcasting follows NumPy
rules, and the gradients broadcast back correctly.
grad is a builtin backed by a real reverse-mode engine. It follows the
structure of its argument, so a model held in a list gets a matching list of
gradients back, and a model held in a record gets a record.
Shapes and units are checked statically. [2,3] @ [4] is an error you see
before the program runs, not a stack trace forty minutes into training.
The language is deliberately small, and the reference implementation is about 27,000 lines of Go with no dependencies, of which the differentiable tensor engine is 4,700, the interpreter 7,100 and the static checker 4,700. Another 15,400 lines are tests. Large tensor operations run across CPU cores, deterministically: parallelism never changes a result.
Download a prebuilt binary for your platform from the
releases page and put it on your
PATH. With a Go toolchain (1.23 or newer) you can also:
go install github.com/twill-lang/twill/cmd/twill@latestOr build from source:
git clone https://github.com/twill-lang/twill.git
cd twill
go build -o twill ./cmd/twilltwill examples/autodiff.tw # run a program
twill check examples/shapes.tw # shape-check without running
twill check src std # ...or several paths, and directories
twill fmt examples/hello.tw # print canonically formatted source
twill fmt src --write # format every .tw file under a directory
twill fmt src --check # name what would change; exit 1 if any would
twill test std/tests # run every *_test.tw under a path
twill # start the REPL (multi-line aware)The REPL keeps reading until brackets balance, so block-body functions can be
defined interactively. Without installing, go run ./cmd/twill <file.tw> works
too, and go test ./... runs the suite.
twill test finds the suites; std/test is what a suite is written with. A
file prints one line per failure and a summary the runner reads, and its status
is its own return value:
mode systems
import "std/test" as t
fn main() -> I64 {
t.equal_i64("two and two is four", 2 + 2, 4)
t.near("a third is a third", 1.0 / 3.0, 0.3333333333, 0.0000000001)
t.report("arith")
}
near takes the tolerance and has no default, because a default epsilon is the
one number in a numeric test worth stating and the wrong one for something
whatever it is set to.
| Builtin | Returns |
|---|---|
grad(f) |
a function computing df/d(arg0), for scalar or tensor args |
grads(f) |
a function returning the gradient of every argument, as a list |
value_and_grad(f) |
a function returning [f(x), df/d(arg0)] |
jacobian(f) |
a function returning the full [m, n] Jacobian of a vector output |
hessian(f) |
a function returning the [n, n] Hessian of a scalar output |
grad, grads and value_and_grad differentiate a scalar output, as a loss
does. jacobian handles a vector output and returns every partial derivative at
once (examples/jacobian.tw). hessian gives exact
second derivatives, via forward-mode jets over the core ops, which is enough for
Newton's method:
$ twill examples/hessian.tw
Hessian of xᵀAx (equals A + Aᵀ):
tensor([[4, 1], [1, 12]], shape=[2, 2])
step 0 t = 1.5 f'(t) = 5.5
step 1 t = 1.238095 f'(t) = 1.162833
step 2 t = 1.144277 f'(t) = 0.127467
step 3 t = 1.131153 f'(t) = 0.002356
step 4 t = 1.130901 f'(t) = 0.000001
step 5 t = 1.130901 f'(t) = 0
converged to a minimum where f'(t) is ~0
The autodiff graph is only built while a value is being differentiated, so ordinary evaluation does not pay for it.
# Comments start with '#'.
let a = 3.0
let v = [1.0, 2.0, 3.0] # a vector, shape [3]
let m = [[1.0, 2.0], [3.0, 4.0]] # a matrix, shape [2, 2]
let d = v @ v # dot product -> 14
let mv = m @ [1.0, 1.0] # matrix-vector -> [3, 7]
# A function is one expression or a block; the last expression is returned.
fn rms(t) {
let n = len(t)
sqrt(sum(t * t) / n)
}
# Loops, for training code.
let total = 0.0
for i in range(10) { total = total + i }
# Differentiation.
fn energy(w) = sum(relu(w) * relu(w)) / 2.0
let g = grad(energy)([-1.0, 2.0, -3.0, 4.0]) # [0, 2, 0, 4]
# Two values back, when neither of them wants a name. The comma is what makes
# a tuple: `(x)` is still x in parentheses.
fn span(t) -> (F64, F64) = (min(t), max(t))
let (lo, hi) = span(v) # lo is 1, hi is 3The language guide covers everything, and the design notes explain how it works and what is next.
Scalars carry units too. Declare base units, annotate quantities, and the checker tracks units through arithmetic, so price times quantity is money but dollars plus shares is refused. Units are erased at runtime and cost nothing.
unit USD
unit share
fn notional(px: USD/share, qty: share) -> USD { px * qty }
let price: USD/share = 150.0
let value = notional(price, 200.0) # USD$ twill check bad.tw
bad.tw:6: shape error: unit mismatch: USD*share^-1 + share
6 | let bad = price + qty
See examples/units.tw and the
language guide.
Elementwise ops broadcast NumPy-style, a row vector across a matrix, a column
against rows, a scalar against anything. Beyond arithmetic and @ the builtins
cover:
| Group | Operations |
|---|---|
| Elementwise | relu, sigmoid, tanh, exp, log, sqrt, square, abs, clip |
| Normalizing | softmax, logsumexp |
| Selection | maximum, minimum, where, and elementwise comparisons |
| Reductions | sum, mean, max, min, prod, median, argmax, argmin (optional axis) |
| Rearranging | flip, roll, diff, reshape, broadcast_to, transpose, concat, split |
| Sorting | sort, argsort, topk, argtopk |
| Scans | cumsum, cumprod, cummax, cummin (optional axis) |
| Contraction | einsum("ij,jk->ik", A, B), differentiable and general |
| Deep learning | conv2d, maxpool2d, gather |
Tensors and lists also support differentiable first-axis slicing (v[1:3],
m[:2]). The language guide has the full list.
On a list, sort also takes a comparison, so a list of anything can be ordered:
sort(items, fn(a, b) = a.n < b.n). Every form is stable.
The std/ libraries are written in twill itself and compiled into the binary, so
import "std/nn" works from any directory with nothing to install alongside it.
| Module | Contents |
|---|---|
std/nn |
dense layers, activations (gelu, softplus, ...), He and Xavier initializers, losses including softmax cross-entropy, nn.conv, multi-head causal attention |
std/transformer |
the GPT-style decoder: pre-norm blocks, tied embedding head, next-token loss, greedy generation, over the std/nn pieces |
std/optim |
SGD, momentum and Adam, over a model held as a positional list or a named record |
std/data |
standardizing, train/test splitting, minibatching |
std/backtest |
returns, moving averages, equity curves, drawdown, Sharpe, Sortino, CAGR |
std/num, std/shapes |
numerics and rearrangements the builtins leave out |
std/text, std/float |
string handling for mode systems, and exact float formatting and parsing |
std/term |
the terminal layer: capability detection, colour, boxes, frames, display width |
std/io, std/json, std/hash |
text and line reading, a JSON reader and writer, SHA-256 |
std/test |
the assertions a *_test.tw file is written with, printing the verdict twill test reads |
That table is a selection. std/ also carries batch, frame, linalg,
llama, loss, metrics, random, sample, stats and gradcheck.
The optimizers walk a model's tensor leaves with map_leaves and zip_leaves,
which is why the same optim.adam works on a list of matrices and on a record of
named weights. A library imported as a namespace, import "std/nn" as nn, exposes
its fields in declaration order, so the same program prints the same thing every
run.
Load your own data with read_csv("data.csv"), which gives a [rows, cols]
tensor, or read_frame("data.csv"), which reads a header CSV as a frame: a
record of named column tensors, so df.close, slicing and grad all work on it.
Trained models persist with save(model, "model.bin") and load("model.bin").
Tensors, scalars, strings, lists and records of them, and fitted
gradient-boosted forests round-trip bit for bit, so you can train once and ship
the model with the single binary for inference. Functions and closures do not
serialise and save rejects them.
Randomness is deterministic by default and seeded, so a program reproduces
exactly. seed(n) picks the starting point. That is a claim about one machine:
Go's math.Exp differs by one ULP between arm64 and amd64, so a program that
calls exp can differ in the last bit across architectures, and an iterative
method can turn that into a visible difference. docs/CORRECTNESS.md section 4
has the measurement.
The same grad, the same checker and the same binary cover the stack:
| Example | What it does |
|---|---|
nn_xor.tw |
a small net, with grad taken over the whole parameter list |
classifier.tw |
a 3-class MLP with softmax cross-entropy and Adam |
cnn.tw |
a convolutional net, conv to relu to max-pool to dense, trained end to end with the kernel included |
attention.tw |
a self-attention sequence classifier; grad differentiates the attention softmax and the learned embeddings together |
minibatch.tw |
a full training loop: standardize, split, reshuffled minibatches each epoch |
gbm.tw |
gradient-boosted trees, native and deterministic, no XGBoost |
records.tw |
parameters in a record, so grad returns a record of gradients |
frames.tw |
realized volatility from a price series, over a named-column frame |
backtest.tw |
a vectorized backtester built on the cumulative builtins |
signal_opt.tw |
tuning a trading signal by gradient ascent straight through the backtest |
signal_opt.tw is the one worth a second look. Because the Sharpe ratio is
differentiable in the return series, the backtest is a function you can climb,
which a plain Python backtest cannot do without reaching for JAX.
docs/finance.md sets out where twill aims to beat a Python
stack for financial ML, and how.
The checker also verifies declared record types, so a model's fields and shapes are part of the contract:
type Model = { w: [3, 2], b: [3] }
fn predict(m: Model, x: [2]) -> [3] { m.w @ x + m.b }The front end matches the reference across the corpus. The evaluator does not. Measured 2026-09-04 over every
.twfile this repository tracks:checkagrees on all of them,fmtagrees on all of them apart from blank lines, and the self-hosted evaluator still refuses a substantial part of the builtin table.docs/roadmap.md, "What the second implementation agrees on, and what it does not", has the measurements and how to repeat them.
The reference implementation is Go. The second one is twill: the lexer, parser,
checker, evaluator, tensor kernels, formatter and CLI, written in the language
itself under src/. The whole src/+std/ tree type-checks clean and runs on
the Go bootstrap.
What is finished is the front end. Running both sides over every .tw file
git ls-files '*.tw' reports -- the whole tree, read recursively -- twill check agrees with the Go command on every one, and twill fmt now agrees on
every one byte for byte: the blank-line rule the two used to disagree about has
been ported into internal/format, so the claim no longer carries an "except".
Budget minutes rather than seconds: src/eval.tw takes minutes through the
self-hosted checker against well under a second on the bootstrap.
This paragraph carries no file count on purpose. Two earlier versions of it carried one and both were wrong, in opposite directions: one quoted a v1.4.0 snapshot of a corpus that has since changed shape, and the other counted four directory names as top-level globs and so left out everything in their subdirectories. The corpus grows; the claim is about every file in it.
The evaluator is not finished, and it is the half the word "self-hosted" is
usually taken to promise. src/eval.tw dispatches a builtin by name and ends
its chain with "named in the builtin table but has no implementation", and a
large minority of the names in src/builtins.tw still reach it -- the
filesystem, the clock, the process, the RNG, the f64_* scalar intrinsics, the
GPU calls and the memory counters. The lists, dictionaries, byte buffers and
string primitives were ported in main and are no longer among them.
docs/BUGS.md entry 12 and its Open section carry the current split and the
one-line probe that re-derives it.
Numeric-mode programs mostly do agree, and where they do not the disagreement is
larger than this section used to claim. examples/gbm.tw exits 0 on both sides
and prints a test regression RMSE of 0.660285 on the bootstrap against
0.659657 self-hosted, reproducible across runs: a fourth-decimal
disagreement, not the 1-ULP float noise the old text described.
examples/save_load.tw reloads a model and the self-hosted gbm_predict
refuses it. examples/frames.tw fails for a third reason that is not a semantic
one at all: the self-hosted CLI resolves the program's relative data path
against its own directory, so it looks for src/prices.csv.
internal/interp/selfhost_run_test.go compares the two implementations over
run for a directory of fixtures, and CI runs it. Nothing compares them at
corpus scale: the harness under tools/diff/ is referenced by neither the
Makefile nor CI, and it compares two Go binaries rather than the two
implementations.
src/ runs on the bootstrap rather than as its own Go-free binary;
bootstrapping to a standalone twill-built compiler is the next step, and closing
the evaluator gap is a prerequisite for it. The shortest statement of that gap
is that src/ cannot run src/:
$ ./twill run src/main.tw run "$PWD/src/main.tw" run "$PWD/examples/hello.tw"
<repo>/src/main.tw:341: runtime error: undefined variable "SFn"
341 | if len(diags) == 0 {
That line moved when the filesystem, clock and process port landed. It used to
be line 47 and the builtin args, which is to say the inner CLI stopped on the
first statement of its own main. It now gets through argument handling and
into the checker and stops on an enum case that src/check.tw uses unqualified
from src/ast.tw, so what is left there is about a module and an enum rather
than about a missing name. The same two levels answer --version correctly.
(The line number is the entry
file's rather than the module's, which is a second defect visible in the same
report and recorded in docs/BUGS.md.)
(The inner paths are absolute because the self-hosted CLI resolves the path it
is given against its own directory rather than the caller's, which is the same
bug examples/frames.tw hits. A relative path inside a program being run is a
different question and no longer has that answer: the evaluator is told which
file it is running and resolves against the program's directory.)
The paragraph above is about the front end, and about the example corpus, and
it is worth being exact about where the agreement stops.
docs/conformance.md is generated by running both
implementations rather than by reading either, and it says which names in the
shared builtin table src/eval.tw dispatches and which it does not. It has the
count and the list, and make conformance regenerates both, which is why
neither is repeated here. A program that reaches for rng_open, any of the
f64_* scalar functions, the gpu_* device intrinsics or the mem_* counters
checks clean on both sides, runs on the bootstrap, and is a runtime error
self-hosted. The gap is not only missing names: most of the suites in
std/tests/ do not produce identical bytes under the two implementations, and
gradcheck_test.tw passes 19 of 19 on the bootstrap and 17 of 19 self-hosted,
with no error on either side. Every one of those divergences is on a checked-in
allow-list that make conformance-check enforces, keyed to the divergence it
names, so a new divergence fails the build, a changed one fails the build, and
the list can only get shorter.
The same gate runs the cases under testdata/conformance/cases/, which have no
allow-list: each is written to pin one builtin and is checked in only once both
implementations agree on it, so a divergence there is a regression rather than
an unfinished port.
Designing the subset a compiler needs was the point of doing it. A .tw file
declares its mode on the first line, and mode systems turns that subset on: a
real 64-bit integer with defined wrapping, byte strings, arrays, dictionaries,
structs, file reading. Designing it is the actual project; the compiler is
downstream of it and is the easy half. Writing the compiler first is how you find out what the subset
has to be, instead of guessing.
The output so far is two things.
A specification of what the language still needs. Every wall the port hits is
written down in docs/needs.md, one numbered entry per feature,
naming the file and line that reaches for it and what the Go bootstrap does in the
same place. Implementing an entry is then a matter of making twill do what Go
already does there. It is a work queue, ordered by dependency.
Bugs in the reference implementation. src/lex.tw was run against
internal/lexer/lexer.go over the whole corpus, 385 files at the time, and
4,000 seeded fuzzer cases,
compared on token kind, literal text, line, column, the comment list, and the
error message and its position. Zero divergences on the corpus and the fuzzer,
three on targeted edge cases. One of the three is a bug in the Go lexer: source
ending in an unterminated string whose last byte is a backslash makes it index
past the end of its rune slice and panic. The twill lexer checks, and reports
"unterminated string" at the opening quote, which is the better diagnosis. It is
recorded as NEEDS-33 and fixed in the Go lexer, with
TestUnterminatedStringEndingInABackslash covering it.
The design, including why file-level modes are the mechanism and what each
feature costs the numeric language, is in
docs/self-hosting.md.
spool, the package manager, is the same experiment run a second time: a real program written against the subset, with its own list of what is missing.
This is a prototype, and some of it is deliberately left for later.
- It is interpreted. Tensor ops loop in Go, and there is no vectorized or GPU backend. The interpreter is the reference for the semantics. docs/gpu-feasibility.md measures what a GPU backend would actually buy and recommends against it for now.
- There is a compiler, and it is off.
TWILL_TRACE=1turns on a tracer that records tensor operations as the interpreter runs them, compiles the graph to C and calls it. It is correct, and on every program measured it is slower end to end, between a quarter and a factor of two and a half (docs/CODEGEN.md §11.2). That is a scope boundary rather than a bug: a statement is the largest region whose live values are known exactly and for free, and a training loop does not fit in one statement, so a loop traces and then escapes on the next statement instead of compiling. Where the work does fit in a statement it wins,montecarlo_option.twat 1.65x with a third less memory. Widening it means tracing across statements, which needs a real liveness analysis over the interpreter's environments and its Go stack. That is a different project and it is not started. Five attempts and three reverts are written up in §11 rather than summarized here. - Autodiff is reverse-mode and first-order.
grad(grad(f))is refused rather than silently answered with zero; usehessianfor second derivatives. - The shape checker is best-effort, not a full type system. It catches mismatches when shapes are statically knowable and stays quiet otherwise.
- Imports are files and
std/modules. Versioning and third-party libraries are spool's job rather than the compiler's: aspool.tomlnames a dependency and spool fetches it, and this repository carries one so that the language itself resolves to a real package. Nothing in the compiler knows about spool, so a vendored dependency is reached by the ordinary file import rule and not by a package path. - The self-hosted compiler runs on the Go bootstrap, not yet as its own Go-free binary. Bootstrapping to a standalone twill-built compiler is the next step.
- The self-hosted evaluator does not implement the whole shared builtin table,
and disagrees with the bootstrap on most of the standard-library suites.
docs/conformance.mdhas the count and the list, regenerated from a real run bymake conformance, andtestdata/conformance/suite-allow.txthas the suites with the reason for each.
The design notes go into the roadmap.
cmd/twill/ the `twill` command (run / check / fmt / repl)
internal/lexer/ source text -> tokens
internal/parser/ tokens -> AST
internal/ast/ AST node types
internal/tensor/ the differentiable tensor engine
internal/gbm/ native gradient-boosted trees
internal/value/ runtime values and environments
internal/interp/ the tree-walking interpreter and its builtins
internal/checker/ static shape and unit analysis
internal/format/ the source formatter (twill fmt)
src/ the self-hosted implementation, written in twill
std/ the standard library, written in twill and embedded
examples/ runnable .tw programs
editors/vscode/ syntax highlighting for .tw files
assets/ the mark, the wordmark and the icons
docs/ the guides, the design notes and the specification
Start at docs/README.md, which indexes the lot. The short version:
| Document | For |
|---|---|
| tutorial.md | from nothing to a trained model |
| tutorial-systems.md | mode systems, ending in a working parser |
| language-guide.md | the reference |
| design.md | why it is built this way, and the roadmap |
| self-hosting.md | the systems subset, and the port |
| conformance.md | which builtins each implementation actually runs |
| needs.md | what the language still has to provide |
| finance.md | the financial-ML case, assessed honestly |
| brand.md | the mark, the palette and the asset rules |
Bug reports, small fixes and design discussion are all welcome. See CONTRIBUTING.md.
MIT.