Initial Structured Concurrency

[lePereT]

What type of PR is this? (check all applicable)

• 🍕 Feature
• 🧑‍💻 Code Refactor
• ✅ Test

Description

This PR introduces a new, higher-level fibers facade, refactors the existing fiber runtime into a clearer layering, and wires in a structured concurrency model based on scopes. It also updates all examples and tests to use the new API and directory layout.

High-level summary:

• Introduce src/fibers.lua as the top-level public API, wrapping:

  • fibers.runtime (scheduler and raw fibers)
  • fibers.op (CML-style event engine)
  • fibers.scope (structured concurrency)
  • primitives (sleep, channel, etc.)

• Replace the old fibers.fiber module with a new fibers.runtime module.

• Add scope-aware event performance via fibers.performer.

• Move library sources under src/ and update examples/tests accordingly.

• Add dedicated tests for the new scope and performer semantics and adjust existing tests for the revised op behaviour.

More detailed breakdown:

New top-level facade and runtime

• Add src/fibers.lua:

  • Exposes fibers.run(main_fn, ...) as the preferred entry point.
  • Exposes fibers.spawn(fn, ...) which spawns a fiber under the current scope.
  • Re-exports primitives: fibers.sleep, fibers.channel, and CML combinators (choice, guard, with_nack, always, never).
  • Exposes fibers.perform as the scope-aware event performer.
  • Provides fibers.now as a thin wrapper over the runtime clock.

• Introduce src/fibers/runtime.lua:

  • Encapsulates the global scheduler (sched.new()), current fiber tracking, and basic operations:

    • spawn, suspend, yield, now, stop, main.

  • Retains the low-level Fiber class and error handling (with traceback capture and process exit on unhandled fiber error).

  • Exposes current_scheduler and current_fiber() for internal consumers (e.g. scopes, cond, context).

Structured concurrency and performer

• Add src/fibers/scope.lua:

  • Implements a tree of Scope objects with:

    • scope.root() and scope.current().

    • scope.run(body_fn, ...) to run a function in a fresh child scope, wait for child fibers, run defers, and then either:

      • return results on success; or
      • raise the scope’s primary error on failure/cancellation.

    • Scope:spawn(fn, ...) to spawn child fibers attached to the scope, with fail-fast semantics (child exceptions cause the scope to fail and cancel).

    • Scope status management: "running" | "ok" | "failed" | "cancelled", including primary error and additional failures.

    • Scope defers (Scope:defer(handler)), executed in LIFO order at scope exit.

    • Cancellation and join events:

      • Scope:done_ev() (fires on cancellation/failure, returns reason).
      • Scope:join_ev() (fires on terminal status, returns (status, error)).

    • Event wrapping:

      • Scope:run_ev(ev) wraps an event so that scope cancellation races against ev.
      • Scope:sync(ev) performs the wrapped event under this policy.

  • Maintains a weak mapping from Fiber objects to their current Scope, plus a process-wide global_scope for non-fiber contexts.

• Add src/fibers/performer.lua:

  • Provides performer.perform(ev) as the scope-aware synchronisation primitive:

    • Looks up the current scope (if any) and delegates to s:sync(ev).
    • Falls back to op.perform_raw(ev) when no scope is active.

  • Used throughout the primitives instead of calling op.perform directly.

• Add scope.with_ev(build_ev):

  • Exposes scopes as first-class events.

  • Returns an Event that:

    • Creates a child scope of scope.current().

    • Installs it as current (per fiber or globally) for build_ev(child) and the lifetime of the event.

    • Ensures, via op.bracket, that on commit, abort, or error:

      • the previous current scope is restored;
      • child fibers are waited on;
      • defers are run; and
      • the scope’s join_ev is signalled.

    • Propagates the inner event’s result or error to the caller.

Event engine (fibers.op) changes

• Rename and clarify primitive event construction:

  • Replace new_base_op with new_primitive(wrap_fn, try_fn, block_fn) and update callers.
  • Export new_primitive and perform_raw in the module API.

• Extend and refine event semantics:

  • Unified Event type with kinds: prim, choice, guard, with_nack, wrap, abort, wrap_handler.

  • always(...) now creates a primitive event that is always ready, returning arbitrary values.

  • never() returns a primitive event that never becomes ready.

  • Event:wrap(f) and Event:on_abort(f) remain as combinators over the event AST.

  • Introduce Event:wrap_handler(handler) and Event:finally(cleanup) to support CML-style exception handling and “finally” semantics on post-synchronisation actions.

  • with_nack and abort now share a unified condition (new_cond) mechanism, with:

    • condition wait events (wait_op) for nack/on_abort paths;
    • signal() waking waiters in the current turn and running optional abort handlers.

• Rework compilation and commit logic:

  • compile_event(ev) compiles the AST into primitive “leaves”, each capturing:

    • try_fn, block_fn, a composed wrap (with wrap_handler chain embedded), and a list of active nacks.

  • perform(ev) now:

    • tries a single non-blocking attempt via try_ready;
    • if no leaf is ready, blocks via a scheduler suspension (runtime.suspend) with block_choice_op;
    • identifies the winning leaf by the wrap function identity;
    • triggers nacks for losing branches using trigger_nacks.

• Introduce Event:or_else(fallback_thunk):

  • Biased choice: attempts to commit to self once; if no leaf is ready, fires all nacks for self and evaluates fallback_thunk() as the result.
  • Implemented using guard and always(...) to ensure clear “try once and decide” semantics.

Primitives updated to the new runtime and performer

All primitives and ancillary modules are updated to depend on fibers.runtime and fibers.performer rather than the removed fibers.fiber and older op.perform/new_base_op:

• src/fibers/alarm.lua:

  • Uses runtime.current_scheduler for scheduling absolute and buffered wakeups.
  • Returns events using op.new_primitive and performs them via fibers.performer.

• src/fibers/channel.lua:

  • put_op and get_op now use op.new_primitive.
  • Channel operations are performed via performer.perform.

• src/fibers/cond.lua:

  • Uses performer.perform for Cond:wait().

• src/fibers/context.lua:

  • Replaces fiber usage with runtime and performer.
  • base_context:done_op() now uses op.new_primitive.
  • Deadlines use runtime.current_scheduler:schedule_at_time.
  • Adds cancel_context:done() convenience method that performs done_op().

• src/fibers/pollio.lua:

  • Integrates with runtime.current_scheduler as a task source.
  • File descriptor and stream readiness events use op.new_primitive.
  • Blocking operations go through performer.perform.

• src/fibers/sleep.lua:

  • Sleep events implemented as op.new_primitive with runtime.now() and scheduler integration.
  • sleep.sleep uses performer.perform.

• src/fibers/stream.lua and src/fibers/stream/file.lua:

  • Read and write operations create op.new_primitive events.
  • All I/O blocking paths now go via performer.perform.

• src/fibers/waitgroup.lua:

  • Wait operation uses op.new_primitive and performer.perform.

Other modules (epoll, timer, queue, syscall helpers, etc.) are moved under src/fibers/... with minimal or no behavioural changes, aside from imports.

Examples updated to use fibers facade

All example programmes have been updated to:

• Point package.path at ../../src/?.lua instead of the old top-level fibers/ directory.
• Import the new facade with local fibers = require 'fibers' instead of fibers.fiber.
• Use fibers.run(main) as the top-level entry point rather than manual combinations of fiber.spawn, fiber.main, and fiber.stop.
• Use fibers.spawn instead of fiber.spawn.
• Use fibers.perform (or require 'fibers.performer'.perform) instead of op.perform where appropriate.

This affects:

• examples/1-basic-usage/1-fibers.lua through 10-luacat.lua
• examples/2-lua-http/cq-http-fiber-test.lua

Where relevant, examples that previously called fiber.yield() have been moved to use sleep.sleep(0) to maintain non-blocking behaviour under the new facade.

Tests updated and extended

The test suite has been updated to reflect the new module layout and semantics:

• All package.path entries now target ../src/?.lua.

• Imports of fibers.fiber are replaced with fibers.runtime or fibers as appropriate:

  • tests/test_runtime.lua (renamed from test_fiber.lua) now tests fibers.runtime directly.
  • Tests that need top-level APIs (e.g. perform, choice) now import from fibers or fibers.performer.

• tests/test_op.lua:

  • Updated to use performer.perform and runtime rather than fiber.

  • Adjusts expectations around Event:or_else to reflect the new “single non-blocking attempt” semantics:

    • If the main event is not ready immediately, the fallback branch wins.

• tests/test_cond.lua:

  • Now uses runtime.spawn and runtime.current_scheduler:run.
  • Adjusts expected log ordering, reflecting the change in condition signalling behaviour (waiters now run in the current turn via task:run()).

• tests/test_context.lua:

  • Updated to use fibers facade and performer.perform.
  • Synchronisation and cancellation tests rely on sleep rather than fiber.yield.

• tests/test_alarm.lua, tests/test_channel.lua, tests/test_pollio.lua, tests/test_stream-file.lua, tests/test_waitgroup.lua, etc.:

  • Updated imports (fibers, fibers.runtime, fibers.performer).
  • Top-level execution now uses fibers.run(main) for tests that rely on the full scheduler/fiber environment.

• New tests/test_scope.lua:

  • Adds comprehensive coverage for scope behaviour, including:

    • scope.current() behaviour inside and outside fibers.
    • Parent/child relationships and children() invariants.
    • scope.run success, failure, and explicit cancellation.
    • Scope:defer ordering, including on failure.
    • Interaction between Scope:sync, event failures, and scope status.
    • join_ev and done_ev on failed and cancelled scopes.
    • Fail-fast behaviour when child fibers fail under a scope.

• tests/test.lua:

  • Module list updated to cover new layout and components:

    • Replaces fiber with runtime, and adds scope.

Screenshots/Recordings

Not applicable. No user-facing visual changes.

Manual test

• 🙅 no

Manual test description

Not performed as part of this change description. The intended manual verification is:

• Run the full test suite under tests/ against the new src/ layout.

• Run the example programmes under examples/1-basic-usage and examples/2-lua-http to confirm that:

  • fibers.run(main) starts and stops the scheduler as expected.
  • Structured concurrency semantics (fail-fast and cancellation) behave as designed.

Added tests?

• 👍 yes

New tests:

• tests/test_scope.lua for the structured concurrency API.

Existing tests have been updated to:

• use the new fibers facade, fibers.runtime, and fibers.performer;
• validate revised fibers.op semantics, including the new or_else behaviour and nack handling.

Added to documentation?

• 🙅 no documentation needed

No documentation files were updated in this PR. A follow-up may be appropriate to:

• document the new fibers facade as the primary public entry point;
• describe scope semantics (scope.run, Scope:spawn, Scope:sync, scope.with_ev);
• clarify the role of fibers.performer.perform versus op.perform_raw.

[optional] Are there any post-deployment tasks we need to perform?

• Update any external consumers that:

  • import fibers.fiber directly, to move to either fibers (preferred) or fibers.runtime for low-level access;
  • rely on the old top-level fibers/*.lua layout, to use src/fibers/... or the packaged module path.

• Review and, if necessary, update any external documentation and example code to use fibers.run/fibers.spawn and the scope-aware performer.