[programming-model] Define hosted multi-PE startup and runtime ABI

[jiale-wangOwO]

Request

Define the hosted multi-PE programming model and runtime ABI required to run a normal hosted musl ELF across four PEs.

The current toolchain emits a normal hosted process entry:

_start -> _start_c -> __libc_start_main -> main -> exit

The specification does not currently define how this entry is used when four PEs execute the same ELF.

Observed failure

In SuperScalarModel#346, the res_on=on hosted multi_thread/matmul ELF is run with four PEs. Cooperative TMATMUL completes, then the hosted close/exit path reports:

ecall warning: close function failed! errno: 9 Bad file descriptor
gfrun: illegal instruction: ASSERTION FAILED:
currentBlock->barg.target != 0 && "Block BARG target"

The compiler review is recorded at SuperScalarModel#346 comment.

Required specification decisions

Define a normative contract for:

Process launch

• Whether all PEs enter the same ELF entry;
• PE0 hosted entry and PE1..PE3 worker entry, if applicable;
• Worker activation and startup ordering;
• Startup completion synchronization.

Per-PE state

• Stack base, size, stride, alignment, guard, and lifetime;
• TLS and libc state ownership;
• PE identity and leader identity exposure.

Syscalls and file descriptors

• Guest-to-host fd mapping;
• Whether fd state is process-shared or PE-local;
• Which PE may perform open/read/write/close;
• exit and exit_group ownership and semantics;
• Behavior when one PE exits or faults.

Synchronization and completion

• Startup, input-ready, kernel-completion, and exit barriers;
• Leader/worker participation around cooperative Tile operations;
• Core-wide completion and error propagation.

Toolchain and ELF contract

• Startup objects and linker mode;
• Worker entry symbols;
• ELF note/ABI identity;
• Compiler/runtime APIs for leader detection, worker activation, and barriers;
• A minimal four-PE hosted reference sample.

Acceptance criteria

• The normative document or ADR defines the complete launch, stack, libc/TLS, fd, synchronization, exit, and ELF ABI contract.
• The contract assigns implementation requirements to compiler/linker, musl/runtime, gfrun/QEMU, and benchmark runtime.
• A runtime-only reference sample covers startup, stack isolation, fd lifecycle, barriers, and exit.
• A four-PE hosted matmul precision-validation sample is defined with an independent golden result.
• The compiler, runtime, functional models, and benchmark can implement the same contract without private assumptions.

References:

• SuperScalarModel#346
• gfrun fail-closed PR #360
• PTO ISA v0.58.4
• ADR-0097
• ADR-0098

[zhoubot]

A minimal reference implementation of the agreed PE0/worker direction is now available:\n\n- LinxISA/SuperScalarModel#360 implements gfrun ELF detection, PE0/worker initial PCs, independent stacks, syscall ownership, and fail-closed behavior.\n- PTO-ISA/SuperNPUBench#87 implements the static single-shot lightweight runtime and multi-thread matmul res_check carrier.\n- LinxISA/linx-skills#47 records the reusable libc/runtime gate.\n\nThe first implementation is intentionally static-only, exposes __linx_group_worker_start and __linx_group_worker_main, keeps libc/I/O on PE0, and leaves dynamic PT_INTERP support for a later ABI. The runtime-only four-PE carrier passes; this issue should remain open until the PRs merge and the real hosted matmul precision lane passes.

[zhoubot]

The first implementation is now split by component ownership:\n\n- LinxISA/Linx-TileOP-API#34 owns the public header and ABI documentation.\n- LinxISA/llvm-project#71 owns the freestanding compiler-rt implementation.\n- LinxISA/musl#14 packages it into the installed liblinx_builtin_rt.a.\n- PTO-ISA/SuperNPUBench#87 only defines the workload worker body and links the archive.\n- LinxISA/SuperScalarModel#360 owns ELF validation, PE initial PC/SP, and completion semantics.\n- LinxISA/linx-skills#47 records the reusable workflow.\n\nThe packaged archive exports linx_group_run and retained __linx_group_worker_start, leaves only __linx_group_worker_main unresolved for the workload, and passes the four-PE runtime carrier with R2=0. PT_INTERP and missing-worker carriers remain fail-closed.

[zhoubot]

The compiler runtime, musl packaging, and skill ownership leaves are now merged:\n\n- LinxISA/llvm-project#71 -> baef2b3a47a2162ef4e0542eb715562bfdbe7a6a\n- LinxISA/musl#14 -> 7db729944eeb6fa530526466d746c5d79bf32481\n- LinxISA/linx-skills#47 -> ad90a843433d2a38a6694b8d67cad027ed587801\n\nExact merged trees match the reviewed head trees. LinxISA/linx-isa#186 carries the three exact pins and component-lock update. The API, workload, and gfrun PRs remain open, so this issue remains open.

[jiale-wangOwO]

Concrete artifacts required to unblock implementation

The PE0/worker direction is now implemented in review PRs, but the end-to-end res_check=on + 4 PE validation cannot continue without the following concrete ABI and toolchain artifacts.

Required ABI specification

Please record the exact normative values for:

• ELF execution-mode identity and ELF note/ABI version;
• PE0 entry and PE1..PE3 worker entry symbols;
• worker activation and startup-ready ordering;
• worker function signature and shared context layout;
• per-PE stack base, size, alignment, guard, and lifetime;
• libc/TLS ownership;
• guest fd namespace, host fd mapping, and fd ownership;
• allowed syscall set for PE0 and workers;
• SYS_exit and SYS_exit_group behavior;
• startup, input-ready, kernel-completion, and exit synchronization;
• worker fault/early-exit propagation and core-wide completion.

Required compiler/linker/runtime artifacts

Please provide or link the matching implementation for:

• startup object implementing the PE0/worker entry protocol;
• compiler-rt/musl runtime object or archive, including liblinx_builtin_rt.a;
• linker mode and linker script requirements;
• definitions of __linx_group_worker_start and __linx_group_worker_main;
• required MUSL_SYSROOT and compiler/toolchain revision;
• whether static PIE is supported and the exact PT_INTERP/PT_DYNAMIC rules;
• a minimal four-PE hosted runtime-only ELF.

Required benchmark artifact

Please provide a regenerated multi_thread/matmul ELF built with the matching runtime and toolchain, including:

• res_check=on build command;
• exact compiler/linker/runtime commits;
• ELF note and worker symbols;
• input/output context ownership;
• one reproducible command for gfrun with four PEs;
• expected res.bin size and independent golden;
• QEMU and gfrun results from the same ELF.

The previous ELF

kernel_multi_thread_matmul_matmul_shared_cube_layout_B1_M256_N256_K256_tM128_tN32_tK64_DTypefloat.elf

is a legacy direct-boot image. It has no __linx_group_worker_start and its multi-PE Shared TLOAD destinations do not carry the PTO 0.58.4 B.ASSEMBLE form, so it cannot validate the hosted SMT4 ABI.

Current implementation references

• SuperScalarModel PR #360 — gfrun ABI implementation and modelSpec/hosted_smt4_runtime.md;
• SuperNPUBench PR #87 — runtime object, startup carrier, and matmul harness;
• linx-skills PR #47 — reusable libc/runtime contract.

Unblock acceptance criteria

The implementation can proceed to final validation once the following are available:

1. the ABI is recorded in a reviewed normative document or explicitly linked normative programming-model document;
2. the matching runtime archive/sysroot/toolchain is available;
3. the runtime-only four-PE ELF runs with the declared worker symbols and exits cleanly;
4. a newly generated res_check=on four-PE matmul ELF runs to completion;
5. res.bin matches an independent golden in both gfrun and QEMU;
6. the implementation PRs and the specification reference the same ABI/version.

Without these artifacts, further gfrun changes would require inventing unverified compiler/runtime behavior.