TiGrIS Runtime

Portable C runtime for TiGrIS. It loads compatible .tgrs plans and executes them against caller-owned memory arenas. The reference runtime has no dependency beyond the C library and does not call a general-purpose allocator during inference.

What it does

A plan records the operator order, tensor metadata, stage boundaries, and tile strategy selected by the compiler. At runtime, TiGrIS:

• validates the plan version and structural limits before exposing its tables;
• assigns activation addresses from caller-provided fast and slow arenas;
• executes normal, spatially tiled, and streamed-chain stages;
• reads uncompressed weights in place or decompresses stage blocks into the fast arena; and
• reports allocation, tiling, and kernel failures to the caller.

The schedule and activation bound are compiled, but addresses are assigned by bounded bump allocation, reset, and compaction during execution. A finite arena can still be exhausted. The loader validates plan-version compatibility before exposing plan tables.

Kernel backends

Backend	Model dtype	Target	Dispatch
reference	float32	Any C target	tigris_dispatch_kernel
s8_ref	int8	Any C target	tigris_dispatch_kernel_s8
esp-nn	int8	ESP32 family	tigris_dispatch_kernel_esp_nn
cmsis-nn	int8	Cortex-M family	tigris_dispatch_kernel_cmsis_nn

Accelerated adapters fall back to s8_ref for explicitly supported non-native variants. They are not interchangeable with arbitrary plans: generate a harness with tigris codegen --backend ... so the compiler can validate the selected dtype/operator route.

ESP-NN and CMSIS-NN also require a successful preparation call after tigris_mem_init() and before inference. ESP-NN preparation obtains platform-managed workspace; CMSIS-NN reserves vendor and quantization workspace from the top of the fast buffer and reduces mem.fast_size. ESP preparation may be repeated safely to replace its workspace and tigris_esp_nn_deinit() releases it after inference. CMSIS preparation is idempotent for the same arena when its existing scratch is sufficient; call tigris_cmsis_nn_deinit() before changing arena or plan. Use tigris_cmsis_nn_fast_arena_required() to size that arena before initialization; it preserves the full core activation/weight capacity below the exact workspace requirement reported by the linked CMSIS-NN library.

Memory contract

The caller owns:

• a fast arena, normally SRAM;
• a slow arena, normally PSRAM or another writable RAM region; and
• one void * entry per tensor.

Generated integrations reserve a plan-sized executor buffer automatically; no workspace limit tuning is required. Manual integrations can query tigris_executor_workspace_required() after loading the plan and pass that many caller-owned bytes to tigris_run_with_workspace_buffer(). The fixed tigris_executor_workspace_t and tigris_run_with_workspace() remain available for source compatibility. tigris_run() uses one process-global fixed workspace and is not safe for concurrent inference. Arena sizing, stack provisioning, and re-entrant execution are documented in the runtime integration guide.

The plan's budget is the modeled activation requirement. For an arena whose base satisfies TIGRIS_TENSOR_ALIGN, tigris_fast_arena_required() returns the core capacity needed for that budget plus any simultaneous compressed weights. Account for base-alignment padding when using an unaligned buffer and provision backend workspace separately. Optimized Cortex-M builds require the plan base and tensors to satisfy TIGRIS_TENSOR_ALIGN (16 bytes with DSP enabled).

For CMSIS-NN, “separately” is enforceable rather than an estimate: tigris_cmsis_nn_scratch_required() queries the linked vendor library and tigris_cmsis_nn_fast_arena_required() combines that result with the core requirement. Generated standalone CMSIS harnesses reserve 4 KiB by default, validate it at startup, and report the exact required total when a larger model needs TIGRIS_CMSIS_NN_SCRATCH_BYTES overridden at build time.

The compiler's current cost model aligns each activation to 32 bytes, which is conservative for the supported host, Cortex-M, and ESP targets. mem.fast_peak is the observed allocator high-water mark, not the scheduled minimum: a roomy arena may defer compaction and therefore report a larger value. Contract tests run with the compiler's scheduled limit to prove that compaction, tiling, and weight reservation still execute within that bound.

The core executor performs no unbounded heap fallback. Normal stages may spill to the supplied slow arena; if neither bounded arena can satisfy an operation, the executor returns an error. mem.fast_peak records the measured core fast-arena high-water mark.

Build and test

cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failure

CTest registers only self-contained tests. Fixture-driven executables are built but require plans generated by the compiler. Generate them with:

pip install tigris-ml
tigris gen-fixtures model.onnx -o test/fixtures/ -m 256K
./test/run_all.sh test/fixtures

The top-level build also compiles the canonical POSIX int8 integration example:

./build/tigris_posix_example model.tgrs

See examples/posix/main.c for checked loader, arena, input, execution, and output handling. Production integrations normally use the harness emitted by tigris codegen.

Install for CMake consumers

Install the runtime to a prefix:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build
cmake --install build --prefix /path/to/tigris-runtime

An external project can then consume the installed headers and library without depending on the TiGrIS source tree:

find_package(tigris_runtime 0.6 REQUIRED CONFIG)
target_link_libraries(my_app PRIVATE tigris::runtime)

Pass the installation prefix through CMAKE_PREFIX_PATH when it is outside a standard system location.

ESP32 (ESP-IDF)

The ESP-IDF example lives in examples/esp32:

cd examples/esp32
idf.py set-target esp32s3
idf.py build
idf.py flash

The example's portable int8 path is the default. With the espressif/esp-nn managed component available, pass -DTIGRIS_ENABLE_ESP_NN=ON to build the ESP-NN adapter.

The plan should live in a memory-mapped flash partition. Allocate the fast arena from internal SRAM and the slow arena from PSRAM when available. When using ESP-NN, call tigris_esp_nn_prepare() and check its return value before tigris_run(), then call tigris_esp_nn_deinit() after the final inference.

Further reading

• Getting started
• Compiler and plan compatibility data
• Runtime integration
• Runtime API
• Memory model