MotifCL

MotifCL is a Vulkan-first C++17 neural compute framework for legacy AMD GPUs, especially Polaris/RX 580-class hardware, with OpenCL as the optional legacy backend. It is designed as a compact research-first alternative to heavy ML runtimes: Vulkan compute runtime (cached pipelines, device-local pooled memory, batched command buffers, embedded SPIR-V kernels), Tensor API, eager autograd for core training loops, neural-network modules, Transformer forward stack, LoRA/Motif/SARC research modules, Python bindings, tests, tools, benchmarks, and CMake install/export support.

Headline result: ternary LM pretrained from scratch on a 2017 Radeon RX 580

A stories15M-shaped model (288 dim, 6 layers, 32k vocab) trained from random init on a single RX 580 8 GB: 48 h, 160k steps, 655M tokens, ~4,150 tok/s, zero NaNs, zero restarts. Every attention/FFN linear is a ternary counter synapse — no FP32 master weights and no optimizer moments ever existed; the ~6-bit counter state is the model. Best val ppl 6.06 vs 4.94–4.99 for the same architecture trained BitNet-style (STE over FP32 latents) on an L40S. Full write-up + raw 2,083-line training log: docs/TERNARY15M_PRETRAIN_RESULTS.md Method package: memory-native.

This release is a productionized source tree rather than a single-file kernel demo. The C++ core builds as a reusable library target MotifCL::motifcl; examples, tests, tools and benchmarks are separate targets; kernels are installed and discoverable through MOTIFCL_KERNEL_DIR or the installed data directory.

Project role

MotifCL is a supporting systems project for memory-native, the primary reference research repository for finite-state counter synapses and reversible training. MotifCL tests how those ideas map onto a native Vulkan runtime and legacy AMD hardware. Each repository keeps its own tests and evidence: PyTorch/T4 results from memory-native are not presented as MotifCL results, and MotifCL Vulkan witnesses are not used as convergence evidence for the reference method.

The target hardware also defines an intentional precision policy. RX 580/Polaris has no modern tensor-core or native BF16 training path, and FP16 has not shown a reliable training-performance advantage over the optimized FP32 route on this target. Therefore Vulkan FP32 is the primary training path. Full-model FP16/BF16 backward is a non-goal for the RX 580 profile, not a readiness requirement. Quantized formats target inference and compact-state research rather than pretending to provide generic mixed-precision training.

See docs/PROJECT_ROLE.md for the evidence boundary and project scope.

Backends

• Vulkan (default, first-class). Backend::create_vulkan() gives device-resident tensors backed by a persistent VulkanRuntime: pipeline/descriptor caches, a pooled device-local allocator with staging transfers, batched one-submit command buffers, dispatch capture/replay, and GPU timestamp timing. A full SGD training step (forward → backward → optimizer, including tiled matmul, grouped-query attention, RMSNorm/SwiGLU backward, softmax cross-entropy, and the compact-counter fused update) runs Vulkan-only: tests/test_vulkan_train_step.cpp is the witness and reports/vulkan-perf/SUMMARY.md the measured record (the ported op mix beats the OpenCL path on RX 580). Kernels live as GLSL sources under kernels/vulkan/ and ship as embedded SPIR-V regenerated by tools/gen_vulkan_spirv.py; building MotifCL needs no Vulkan SDK or shader compiler.
• OpenCL (optional legacy). The original OpenCL runtime/kernels remain for regression parity and paths not yet ported (causal/windowed/masked attention, quantized layouts beyond scalar-scale Q8, decode replay tooling). Build with -DMOTIFCL_ENABLE_OPENCL=OFF to drop the OpenCL loader entirely: the full library still compiles (a local no-op stub replaces the ICD loader), Backend::create_opencl() fails cleanly at runtime, and the Vulkan test suite keeps passing.

Implemented stack

• Vulkan runtime: persistent loader/instance/device ownership without an SDK link dependency, cached compute pipelines and descriptor-set pools keyed by embedded SPIR-V, push-constant scalar arguments, device-local storage buffers with a pooled allocator and staging upload/download, batched command-buffer recording with one submit per pass, dispatch capture/replay, VK_QUERY_TYPE_TIMESTAMP GPU timing, and Vulkan-native forward+backward kernels for matmul (NN/NT/TN/M=1), softmax rows, RMSNorm, SwiGLU, GELU, add, SGD, softmax cross-entropy, causal/batched grouped-query attention, and compact-counter training.
• Legacy OpenCL runtime (optional): OpenCL context/device selection, command queue with profiling, command-buffer capability detection plus extension-gated cl_khr_command_buffer replay/update, shared OpenCL runtime-state ownership for buffers/programs/kernels/replay, RAII buffers, programs, kernels, events, profiler, kernel cache.
• Tensor engine: Tensor, Storage, Shape, DType, views, CPU upload/download, empty, zeros, ones, full, randn, uniform.
• Ops: elementwise math, scalar ops, row bias, workgroup row reductions, register-blocked F32 matmul, generated F32 matmul tile variants, Q8_0/Q4_0 symmetric quantize/dequantize, per-tensor/per-axis/blockwise quantization metadata, mixed Q8/Q4 quantized matmul, register-blocked unscaled quantized matmul paths, extension-gated Q8 integer-dot path with portable fallback, Q4 dot4-unrolled unscaled matmul specialization, activations including SwiGLU, experimental env-gated fused SwiGLU/down-input MLP backward, GPU dropout/masked-fill, softmax rows, causal mask, RoPE, fused QKV split, GQA/MQA attention with staged backward for GPT-sized shapes, KV-cache append, FlashAttention-style tiled multi-head attention forward/backward, workgroup-reduced RMSNorm/LayerNorm with fused residual backward helper, MSE loss, softmax cross entropy for I32 targets, Adam and SGD kernels.
• Autograd: eager backward path for add/sub/mul/div/scale, matmul, ReLU, GELU, MSE, softmax-cross-entropy logits, RMSNorm, token/position embedding, and correctness-first fused multi-head attention backward.
• Graph capture: thread-local static op-sequence tracing through CapturedGraph, GraphCaptureGuard, tensor IDs, replayable captured OpenCL kernel launches, same-shape cl_mem rebinding through GraphExecutor::bind_tensor, driver-level command-buffer replay/update when the ICD exposes cl_khr_command_buffer(_mutable_dispatch), host-reduction replay for scalar MSE / softmax-cross-entropy reductions, linear scheduler/executor, tensor-spec metadata, buffer-planning liveness estimates, shape-polymorphic signatures, and C++/Python APIs for inspecting/executing captured dependencies.
• NN: Parameter, Module, Linear, QuantizedLinear, ReLU, GELU, Sequential, Embedding, RMSNorm, legacy SelfAttention/MLP/TransformerBlock/GPTModel, plus TransformerConfig, ModernSelfAttention, ModernMLP, ModernTransformerBlock, ModernGPTModel, KVCache, PagedKVCache, DeltaStateCache, and a generic HF-style modern transformer compatibility layer (HFTransformerConfig, architecture registry/probe, chat templates, safetensors loading, GGUF F16/F32/BF16 and Q4_0/Q8_0/Q4_K/Q5_K loading, packed K-quant tensor reads with q8-activation matmul kernels, tokenizer loading, cached/batched generation, and mixed Q4/Q8 QuantizationPolicy). Gemma is now one adapter/profile on that path rather than a separate branch.
• Memory-native training: nn::CounterStateLinear (finite-state counter synapse, 0.75 bytes/weight packed 6-bit state) now has register-blocked Vulkan forward and grad_x kernels that decode packed state directly into shared tiles without materializing a dense FP32 weight, followed by the fused in-backward state/scale update. nn::ReversibleBlock provides reversible activations: NoGrad forward stores no activations, backward recovers inputs via inverse-coupling and recomputes through autograd::IsolatedBackwardScope. See docs/MEMORY_NATIVE_TRAINING_METHOD.md. OpenCL-free witnesses include tests/test_vulkan_counter_fused.cpp (fused-path engagement, forward/grad_x parity, exact packed-state update), tests/test_vulkan_reversible.cpp, and tests/test_vulkan_memory_native.cpp.
• Motif/research: MotifLinear, MotifLoRA, Router, MotifTransformerBlock, SARCResidual.
• Training: Adam with grouped multi-buffer fused Adam/AdamW kernel updates, MixedPrecisionAdam with FP32 master weights and DynamicLossScaler, SGD, generic Trainer with dataloader and loss callback.
• Python: pybind11 extension source plus thin package wrappers and a PyTorch-like motifcl.nn facade with Python-side Module, __call__, recursive parameters(), state_dict(), train()/eval(), no_grad(), and Tensor operator overloads.
• QA: CTest suite, CPU/GPU compare tool, OpenCL info dumper, kernel tuner, microbenchmarks.

Build

cmake -S . -B build \
  -DMOTIFCL_BUILD_EXAMPLES=ON \
  -DMOTIFCL_BUILD_TESTS=ON \
  -DMOTIFCL_BUILD_TOOLS=ON \
  -DMOTIFCL_BUILD_BENCHMARKS=ON \
  -DMOTIFCL_BUILD_PYTHON=OFF
cmake --build build -j
ctest --test-dir build --output-on-failure

Python wheel builds use the isolated PEP 517 dependencies from pyproject.toml:

python -m pip install build
python -m build --wheel
python -m pip install dist/motifcl-*.whl
python -c "import motifcl as mcl; print(mcl.Backend)"

Direct CMake Python builds are also supported when pybind11 and Python development metadata are visible to CMake:

cmake -S . -B build -DMOTIFCL_BUILD_PYTHON=ON
cmake --build build -j
PYTHONPATH=$PWD/build/python python -c "import motifcl as mcl; print(mcl.Backend)"

The source build works even when OpenCL development headers are missing, because a minimal vendored CL/cl.h is included. With the default -DMOTIFCL_ENABLE_OPENCL=ON you still need an OpenCL ICD loader/library and driver at runtime for the legacy backend; with -DMOTIFCL_ENABLE_OPENCL=OFF no OpenCL loader or driver is needed at all (Vulkan is the only GPU backend).

Install and consume from another CMake project

cmake --install build --prefix /opt/motifcl

Consumer:

find_package(MotifCL REQUIRED CONFIG)
add_executable(app main.cpp)
target_link_libraries(app PRIVATE MotifCL::motifcl)

Kernel discovery order:

1. MOTIFCL_KERNEL_DIR, if set;
2. build-tree build/kernels;
3. source-tree kernels;
4. installed share/motifcl/kernels;
5. relative kernels directory.

C++ quick start

#include <motifcl/motifcl.hpp>

int main() {
    auto backend = motifcl::Backend::create_opencl();
    auto x = motifcl::Tensor::randn(backend, {32, 128});
    auto w = motifcl::Tensor::randn(backend, {128, 64});
    auto y = motifcl::gelu(motifcl::matmul(x, w));
    auto host = y.to_vector<float>();
}

Tiny GPT forward and cross entropy

#include <cstdint>
#include <motifcl/motifcl.hpp>

int main() {
    auto backend = motifcl::Backend::create_opencl();
    motifcl::nn::GPTModel model(backend, 128, 64, 128, 4, 2, 256);

    std::int32_t ids_host[] = {1, 2, 3, 4};
    std::int32_t tgt_host[] = {2, 3, 4, 5};
    auto ids = motifcl::Tensor::from_cpu(backend, {1, 4}, motifcl::DType::I32, ids_host);
    auto tgt = motifcl::Tensor::from_cpu(backend, {4}, motifcl::DType::I32, tgt_host);

    auto logits = model.forward(ids).view({4, 128});
    auto loss = motifcl::softmax_cross_entropy(logits, tgt);
    loss.backward();
}

Tools

./build/tools/motifcl_dump_opencl_info
./build/tools/motifcl_compare_cpu_gpu
./build/tools/motifcl_kernel_tuner
./build/tools/motifcl_generate_transformer --list-architectures
./build/tools/motifcl_generate_transformer --model ./model.gguf --inspect
./build/tools/motifcl_generate_transformer --model-dir ./model --prompt "Hello" --max-new-tokens 32 --quant q4
./build/tools/motifcl_generate_transformer --model ./model.gguf --system "You are concise." --user "Hello" --chat-template auto --max-new-tokens 32
./motifcl.cmd install
motifcl run "Hello from MotifCL"

motifcl_kernel_tuner writes motifcl_tuning.json with local F32, generated F32 tile variants, Q8_0, Q4_0, mixed Q8/Q4, per-axis Q8, blockwise Q4 matmul timings, and the selected integer-dot mode for the active OpenCL device. Set MOTIFCL_MATMUL_F32_TILE=4|8|16 to force a generated F32 tiled matmul variant for regression testing. motifcl_generate_transformer loads HF-style decoder-only model directories through the modern transformer stack and also accepts a single .gguf file via --model model.gguf. It accepts --arch auto|gemma|llama|mistral|qwen2|qwen3.5|mixtral|generic_decoder for runnable paths, exposes --list-architectures/--inspect for broader modern-family readiness probing, and reports explicit blockers for detected but unsupported model families. Runnable Qwen3.5/Mixtral-style hybrid text cores are dispatched through HybridGPTModel instead of the dense ModernGPTModel path. The initial implemented weight layouts are the common LLaMA/Gemma/Mistral/Qwen2-style model.layers.* safetensors layout and common LLaMA-style GGUF blk.* F16/F32/BF16 plus Q4_0/Q8_0/Q4_K/Q5_K tensors; whole-model loading installs native packed Q4_0/Q8_0/Q4_K/Q5_K tensors directly into split Linear projections and the LM head when GGUF tensor layout already matches MotifCL [in,out], and safely dequantizes/transposes/repackages packed [out,in] tensors instead of reinterpreting them; F16/F32/BF16 and non-packed fallback paths still load as F32. --system/--user/--message plus --chat-template auto|chatml|llama2|llama3|mistral|gemma|generic|none cover common instruct prompt formats. --quant q4|q8 enables inference-only quantized Linear projections and LM head after loading, prompt prefill uses one cached forward pass by default (--no-prefill disables it for debugging), and greedy/top-k/top-p/temperature decode uses a GPU sampler unless --cpu-sampling is requested. Tokenizer loading prefers binary tokenizer.model or vocab.json/merges.txt for HF directories, can read GGUF tokenizer.ggml.tokens, and falls back through tokenizer.json, vocab.txt, or byte fallback. motifcl.cmd is the one-command Ollama-like front door on Windows. Run .\motifcl.cmd install once to install a motifcl shim into the user PATH; after that motifcl works like ollama from any terminal. motifcl builds motifcl_generate_transformer if needed, auto-selects the newest local GGUF under models/, starts the hot server in the background, warms it, and leaves /api/generate ready on 127.0.0.1:11435; motifcl run "Hello" auto-starts that server and streams tokens to the terminal; motifcl list|status|down cover the local lifecycle. motifcl_generate_transformer --jsonl-repl --completion-only remains the lower-level persistent machine REPL for harnesses. python tools/motifcl_ollama_server.py --model ./model.gguf --name motifcl-local:latest --port 11435 wraps that REPL in an Ollama-style local HTTP server with /api/generate, /api/chat, /api/tags, /v1/completions, and /v1/chat/completions; see docs/OLLAMA_COMPAT_SERVER.md. The server starts the runner once, waits for readiness, performs a one-token warmup by default, then serves hot requests without reloading the model; stream:true now forwards per-token deltas from the C++ generation loop. python tools/release_check.py orchestrates local dev/release/python/test/perf/install checks; add --wheel --hf-run --require-clean-git for a pre-tag sweep. python tools/ci_gate.py --parallel 2 is the stricter local gate for release build, full release CTest, Python build/pytest, kernel validation-contract coverage, and release_check.

Additional docs

• docs/ARCHITECTURE.md — runtime/tensor/ops/autograd/graph/training layering and dynamic graph status.
• docs/MODERN_TRANSFORMER.md — RoPE, fused QKV, GQA/MQA, SwiGLU, mask/dropout, KV-cache, and TransformerConfig.
• docs/HF_COMPAT.md — generic modern HF-style decoder compatibility, current architecture dispatch, safetensors/tokenizer/generation path, and adapter boundaries.
• docs/GEMMA_COMPAT.md — compatibility note for the Gemma profile on top of the generic HF path.
• docs/TRAINING.md — dataloaders, schedulers, gradient clipping, history, checkpoints.
• docs/PYTHON_API.md — Python wrapper/stub usage.
• docs/PERFORMANCE.md — tuning and regression workflow.
• docs/VALIDATION_AND_STABILITY.md — artifact parser invariants, stable/experimental boundaries, kernel validation contracts, and CI gate.
• docs/ROADMAP.md — remaining FP16, graph, HF compatibility, perf, and release hardening work.
• docs/OPENCL_TROUBLESHOOTING.md — ICD/kernel discovery/runtime guidance.
• docs/OLLAMA_COMPAT_SERVER.md — hot JSONL REPL plus Ollama/OpenAI-compatible local server usage.

Benchmarks

cmake -S . -B build -DMOTIFCL_BUILD_BENCHMARKS=ON
cmake --build build -j
./build/benchmarks/bench_matmul
./build/benchmarks/bench_softmax
./build/benchmarks/bench_attention
./build/benchmarks/bench_lora

Current engineering boundaries

MotifCL is now structured and buildable as a real library, but it is still intentionally small. It does not attempt to be a full PyTorch replacement. Transformer forward and a small GPT-style backward/training path work through token/position embeddings, RMSNorm, FlashAttention-style tiled multi-head attention, MLP, and cross entropy. Modern HF-style inference now routes through HFTransformerConfig + ModernGPTModel; Gemma/LLaMA/Mistral/Qwen2-style configs share the same modern stack and common safetensors weight layout instead of living as separate model branches. Biasless/dropout-free modern SwiGLU MLP residual branches can use an experimental custom fused backward node with MOTIFCL_ENABLE_FUSED_MLP_BACKWARD=1; MOTIFCL_ENABLE_HIGH_LEVEL_MLP_FUSION=1 additionally tries the normed-buffer-free row-inverse cached RMSNorm+gate/up projection path. These modes currently exist for experimentation and must beat the default register-blocked GEMM path before becoming default. Q8_0 and Q4_0 quantize/dequantize and matmul are implemented as correctness-first symmetric paths with per-tensor, per-row/per-column, flat blockwise, mixed Q8/Q4 matmul support, register-blocked unscaled quantized kernels, mixed per-layer transformer inference policy, a cl_khr_integer_dot_product generated-kernel path for Q8 when the driver exposes it, a Q4 dot4-unrolled unscaled specialization for K % 4 == 0, and portable fallback kernels. F32 matmul now uses a 4x4 per-thread register-blocked 32x32 workgroup tile for normal non-transposed matmul, while transpose and scaled-quant metadata paths keep conservative fallback kernels. Rowwise sum/max, RMSNorm, RMS-per-row, LayerNorm, and RMSNorm backward-X use cooperative workgroup row reductions instead of one-thread-per-row reductions when the device supports 256-workitem groups; RMSNorm+residual backward has a fused helper. Static graph capture can replay captured GPU kernel launches and scalar host reductions, exposes a linear schedule, records tensor specs, estimates buffer reuse/liveness, can emit shape-polymorphic signatures, supports exact same-shape runtime tensor rebinding for captured kernel buffer arguments, and can switch pure captured kernel graphs to cl_khr_command_buffer replay/update when the driver exposes the extension. Runtime OpenCL handles are shared across buffers/programs/kernels/replay so tensor storage can be downloaded/released safely after a Backend scope ends; creating new ops still intentionally requires a live Backend/KernelCache. Remaining limits are mainly true dynamic-shape rebinding/recompilation, planner-materialized buffer allocation for arbitrary captured graphs, hand-written vendor ISA assembly kernels, full FP16 backward coverage across all training ops, exact byte-for-byte SentencePiece precompiled_charsmap parity for arbitrary ICU rule sets, architecture-specific HF weight mappers beyond the implemented LLaMA/Gemma/Mistral/Qwen/Qwen3.5/Mixtral aliases, fully compact packed ragged KV-cache storage, exact full-vocab GPU top-p without the current candidate cap, deeper backend-handle ergonomics for post-Backend new ops, and broader production performance tuning.

How this was built

Built entirely by AI (Claude) under sustained human direction, over months, by someone with no formal CS/math background. Working protocol: every claim needs an executable witness — tests, frozen pre-run forecasts, raw logs committed next to results, negative results reported first-class. The repo, not the author, answers technical questions. Status: frozen (July 2026) — out of money and hardware, not out of ideas. Everything reproduces from a cold clone.

FOG v3 on Radeon RX 580 (Vulkan port)

A native Vulkan port of the FOG register_machine_v3 research path is available under ports/rx580/fog. It targets MotifCL main and reuses the existing Vulkan transformer/autograd stack rather than requiring ROCm.

Russian quick start: FOG_RX580_QUICKSTART_RU.md.