Open, efficient AI research
Research and open tooling for compact language and vision models,
deterministic token routing, multimodal generation, and accessible training.
|
A deterministic token-routed Mixture-of-Experts language model with base and instruction-tuned checkpoints, post-training data, and an interactive demo.
|
A compact hierarchical token-routed vision model with native multi-scale features, shifted-window attention, PAN fusion, and a lightweight detector.
Status: ImageNet-1K pretraining active. COCO 640 evaluation comes next. |
TR-HASH replaces learned routing with stable identity-based selection. Token or spatial identity chooses a small parameter subspace while shared computation continues to process the complete contextual hidden state.
identity ──► fixed layer-specific routing ──► selected experts
│ │
└──────── contextual hidden state ──────────┴──► output
|
The research and training layer: model definitions, distributed training, Triton kernels, exact resume, evaluation, ablations, multimodal generation, and Vision v6. |
The product-facing runtime: prediction, validation, fine-tuning, export, benchmarking, and HTTP serving without carrying the research framework. |
|
Open TR-HASH checkpoints, model cards, demos, and progressively published training artifacts. |
Open text, image, and image-edit datasets with provenance-oriented releases. |
| Hierarchical tower P3 · P4 · P5 native features |
Small objects P2 · PAN · STAL |
Detection O2M + optional NMS-free O2O |
| Training EMA · multi-scale · strong augmentation |
Distributed 4-GPU DDP · exact resume |
Runtime Triton · PyTorch · MPS fallback |
The current ImageNet-1K classifier has 0.76M parameters, including a 0.63M transferable hierarchical tower. The complete 80-class COCO detector is approximately 1M parameters. Accuracy claims and YOLO comparisons will be published only after same-protocol COCO evaluation at 640 px.
|
Text instruction and chat SFT corpus. |
336K provenance-aware image-text pairs. |
336K instruction-guided editing triplets. |
Framework scope
- deterministic TR-HASH MoE with separate expert learning rates;
- language models with GQA/MHA and shared plus routed feed-forward paths;
- hierarchical detection, classification, segmentation, depth, pose, and OBB models;
- image generation/editing, speech, and video research components;
- single-device, DDP, FSDP, CUDA, MPS, and CPU execution;
- exact resumable checkpoints with optimizer, scheduler, cursor, and distributed RNG state.
We separate implemented architecture, active training, and validated results. Claims are tied to realized checkpoints and explicit evaluation protocols. Parameters, compute, latency, memory, and accuracy are reported together whenever possible. Planned runs are never presented as completed benchmarks.
Community contributions, replications, and critical evaluations are welcome.