[Feature] Replace L2 kernel runtime and old L3 generate path with unified L3 worker dispatch

[ndleslx]

Summary

Replace the current non-L3 compiled-kernel execution path, which uses Worker(level=2), with an L3-worker based dispatch path using Worker(level=3) and orch.submit_next_level(...).

As part of this migration, remove the old dedicated run_generate_l3() generation path. L3 execution should become the unified runtime mechanism for prefill/decode kernel dispatch instead of maintaining a separate one-shot L3 generation path.

Area

Executor or runtime

Motivation / Use Case

The current non-L3 path directly runs compiled chip callables through an L2 worker, while the repository also carries a separate one-shot L3 generation path. This creates two runtime models for generation:

• the normal prefill/decode path based on L2 worker dispatch
• the dedicated run_generate_l3() path with separate generated L3 artifacts and control flow

Newer Simpler/PyPTO runtime features, dependency tagging, child-memory handling, and future serving orchestration are centered around L3 worker execution. Moving the normal prefill/decode kernel path to L3 worker dispatch provides a migration bridge and removes the need to maintain a second L3-specific generation implementation.

This enables:

• serving to keep its existing prefill/decode scheduler semantics
• compiled @pl.jit prefill/decode kernels to run through the same hierarchical runtime model as future L3 serving
• tensor dependencies to be expressed through TaskArgs and TensorArgType
• one unified runtime path for offline generation and HTTP serving
• removal of duplicated one-shot L3 generation code once the L3-worker dispatch path covers the same workflows

Related: #18

Proposed API / Behavior

For non-L3 compiled kernels:

• construct Worker(level=3, device_ids=[device_id], num_sub_workers=0)
• register chip callables before worker.init()
• build TaskArgs with explicit tensor dependency tags
• submit chip callables inside an orchestration callback using orch.submit_next_level(...)
• keep full KV tensors as host INOUT tensors for correctness unless a later optimization explicitly introduces resident KV handling
• pre-share CPU tensors before L3 worker initialization so child processes can access host mappings
• provide runtime wrapper methods for submit_next_level, orchestrator-scoped memory operations, and cleanup
• remove the old one-shot run_generate_l3() path and its dedicated generated-L3 artifacts from the serving runner once the L3-worker dispatch path covers offline and HTTP generation
• keep one runtime path for offline generation and serving, with feature flags only for rollout/debugging rather than permanent separate implementations

The first implementation may need conservative one-shot L3 worker cleanup while Simpler worker lifecycle behavior is being fixed. The Simpler-side lifecycle issue is tracked in:

• [Bug] L3 Worker leaves chip child defunct after submit_next_level run simpler#980

Alternatives Considered

Keep the current Worker(level=2) runtime for non-L3 kernels and only use L3 for the dedicated run_generate_l3() path.

That avoids L3 lifecycle complexity in the short term, but it keeps two separate runtime models in serving, duplicates generation control flow, and makes it harder to migrate scheduled prefill/decode execution into L3 DAG form.

Another alternative is to keep run_generate_l3() as a permanent fast path while adding L3-worker dispatch for serving. That would still leave long-term maintenance cost: separate artifacts, separate sampling/prepare logic, separate KV handling, and separate correctness/performance validation.

Additional Context

A prototype implementation exists in PR #22:

• Rewrite non-L3 Qwen3 kernels through L3 worker #22

The prototype rewrites non-L3 Qwen3 kernel dispatch through Worker(level=3) + orch.submit_next_level(...) while keeping full KV tensors and adding focused tests for worker submission, KV dependency tags, and cleanup behavior.

The prototype passed offline generation with larger ring settings:

task-submit --device auto --max-time 0 --run \
  "PTO2_RING_HEAP=4294967296 PTO2_RING_TASK_WINDOW=1048576 PTO2_RING_DEP_POOL=1048576 \
   python examples/model/qwen3_14b/npu_generate.py \
     --model-dir /data/linyifan/models/Qwen3-14B \
     --prompt 'Huawei is' \
     --platform a2a3 \
     --max-seq-len 512 \
     --max-new-tokens 5"

Observed output:

text:  a Chinese company. The
token_ids: [264, 8453, 2813, 13, 576]
finish_reason: length

The smaller default ring settings still fail in prefill with AICPU 507018, so the feature should document required runtime settings or reduce the resource requirement before becoming the default:

PTO2_RING_HEAP=536870912 PTO2_RING_TASK_WINDOW=131072 PTO2_RING_DEP_POOL=131072