Skip to content

index.md

Latest commit

 

History

History
81 lines (61 loc) · 3.18 KB

index.md

File metadata and controls

81 lines (61 loc) · 3.18 KB
Raw

Documentation

Complexity Framework is a PyTorch research stack for deterministic TR-Hash Mixture-of-Experts (TR-MoE) language and multimodal models.

Start here

  1. Architecture and naming
  2. Getting started
  3. TR-Hash execution engine
  4. TokenRoutedMLP (removed) and migrating to TR-Hash
  5. Training
  6. Two-dimensional full-shard SFT weighting
  7. Run configurations
  8. GPU and dispatch paths
  9. API reference

Architecture vocabulary

Public name Attention FFN
TR-GQA GQA TR-MoE
TR-MHA MHA TR-MoE

TR-GQA and TR-MHA share the same TRHashEngineMLP. They differ only in the attention head layout. The framework is currently scoped to TR-Hash MoE only — dense and learned-router baselines were removed and will return later as explicit comparisons.

The registry values tr_mha and tr_mha_v2 refer to experimental routed residual adapters inside attention. They are documented separately in ../TR_MHA.md to avoid conflating them with the main MHA + TR-MoE architecture.

Additional guides

Document status

Area Status Entry point
Architecture and TR-Hash runtime current Architecture, TR-Hash engine
LoRA instruction tuning current Training, 2D full-shard weighting
CUDA and dispatch current GPU and dispatch paths
Multimodal and generative modules experimental Multimodal prototypes and the modality guides below
TokenRoutedMLP migration compatibility only Migration guide
o200k/Dense pretraining records historical, non-runnable Run configurations, 200M B200 runbook

Historical documents preserve evidence provenance. Their commands must not be treated as supported entrypoints unless a current guide explicitly says so.

Multimodal and generative models

Operations

Evidence policy

A configuration file or model implementation is not a completed experiment. Documentation distinguishes:

  • implemented: represented in code and tests;
  • pilot: bounded evidence, usually short or single-seed;
  • controlled comparison: matched protocol with tracked metrics;
  • planned: a launch or cluster plan without completed metrics.

Claims should point to the tracked configuration, metrics, or artifact that supports them.