Skip to content

Add chapter: subprocesses and external streams; cover Ox 1.0.6 additions - #22

Merged
adamw merged 8 commits into
masterfrom
docs/structured-external-resources
Jul 23, 2026
Merged

Add chapter: subprocesses and external streams; cover Ox 1.0.6 additions#22
adamw merged 8 commits into
masterfrom
docs/structured-external-resources

Conversation

@adamw

@adamw adamw commented Jun 28, 2026

Copy link
Copy Markdown
Member

What

Adds guidance for the case structured concurrency is easiest to get subtly wrong: driving a long-lived subprocess (or socket / SSE) whose output is read line by line — and covers the Ox 1.0.6 additions across the skill.

  • New chapter 170-subprocesses-and-external-streams.md:
    • own the work with a supervised scope; the reader's return value is the result (fork{…}.join()), not a cross-thread AtomicReference/CAS;
    • never return a live object that owns running forks to be driven later;
    • teardown: a classic java.io stream read (subprocess pipe, file, stdin) ignores interruption, and Ox releases scope-managed resources only after joining forks — so destroy the subprocess in the scope body's finally, before the join. Blocking socket reads are the exception: on Ox's virtual-thread forks they are interruptible since Java 21 (destructively — the interrupt closes the socket);
    • kill the whole process tree (a launcher's forked child inherits the pipe fds; an orphan keeps the pipe open so the reader never EOFs);
    • abandonOnInterruptReads / abandonOnInterrupt (Ox ≥ 1.0.6) for reads that closing can't unblock (stdin, FileInputStream) and per-read cancellation — the read is delegated to a detached virtual thread so the fork waits interruptibly. Body-finally destroy stays the primary subprocess teardown, since the wrapper doesn't terminate the child;
    • drain/inherit stderr so an unread pipe can't stall the child.
  • Chapter 150 (150-shared-state-across-threads.md): new CPU-intensive work section — virtual threads are never preempted, so long CPU-bound work runs via computeIntensive (platform-thread pool, structured by the blocking caller) and instrumentable loops call cede(); scope context (forks, ForkLocals, MDC/OTel) deliberately does not propagate into the compute pool. Rows added to both pattern-selection tables.
  • Chapter 100 (100-resource-management.md): new resourceScope section — scoped cleanup without claiming a concurrency capability (Ox's Using.Manager analogue, compile-time-guarded against lexically visible concurrency scopes), and using ResourceScope as the narrower capability for registration-only methods.
  • SKILL.md: always-loaded decision triggers — the ownership/teardown rules above, a Performance bullet for CPU-bound starvation (computeIntensive/cede), the ResourceScope refinement of the factories rule — plus updated index entries.

Why

The detailed how lives in the chapters; the decision triggers live in the always-loaded SKILL.md so the agent picks the right ownership and teardown shape before writing code — the failure modes here (shutdown-time deadlock, virtual-thread starvation) don't show up until teardown or load.

Updated for Ox 1.0.6, which addresses part of the original concerns: abandonOnInterruptReads/abandonOnInterrupt (softwaremill/ox#479) make waiting on uninterruptible operations cancellable, and the interruptions docs now officially catalogue which JVM operations are (un)interruptible and state that scope-managed resources (useInScope/releaseAfterScope) are released only after all forks complete. The chapter is grounded in that documented behavior (rather than paraphrased scope internals), and the earlier claim that blocking socket reads ignore interruption was corrected — on virtual threads they are destructively interruptible since Java 21. The remaining 1.0.6 additions with pattern-level impact — computeIntensive/cede() (softwaremill/ox#483, softwaremill/ox#481) and resourceScope (softwaremill/ox#480) — are covered in chapters 150 and 100; the release's finalizer-freeze and compute-pool tuning details were deliberately left out as non-pattern material.

🤖 Generated with Claude Code

adamw and others added 4 commits June 28, 2026 08:31
Adds guidance for the case structured concurrency is easiest to get subtly
wrong: driving a long-lived subprocess (or socket/SSE) whose output is read
line-by-line. The reader is a fork whose return value is the result; the killer
is teardown — a blocking native read ignores interruption, and Ox joins forks
before running `releaseAfterScope` finalizers, so the resource must be destroyed
in the scope body's `finally` (before the join), and a launcher subprocess needs
its whole tree killed or an orphan keeps the pipe open.

- New chapter 170-subprocesses-and-external-streams.md.
- SKILL.md: two always-loaded design rules so the agent decides the owning scope
  and the teardown point UP FRONT (model readers as forks with return-value
  results; never return a live fork-owning handle; destroy blocking resources in
  the body finally, not releaseAfterScope), plus the index entry.

Grounded in Ox's documented scope-teardown semantics (interruptAllAndJoin runs
in the body finally; finalizers after).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tions

Ox 1.0.6 (#479) adds abandon-on-interrupt utilities and officially
documents which JVM operations are (un)interruptible. Update the chapter
accordingly:

- correct the socket claim: blocking socket reads on virtual threads
  (which Ox forks run on) ARE interruptible since Java 21, destructively;
  the teardown deadlock is specific to classic java.io stream reads
  (subprocess pipes, files, stdin);
- ground the releaseAfterScope-runs-after-the-join claim in the now
  documented behavior instead of a paraphrased-internals snippet;
- add a section on abandonOnInterruptReads / abandonOnInterrupt for
  reads that can't be unblocked by closing (stdin, FileInputStream) and
  per-read cancellation — while keeping body-finally destroy as the
  primary subprocess teardown, since the wrapper doesn't terminate the
  child.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The teardown text (chapter + always-loaded SKILL.md bullet) presented
close/destroy as applicable to all uninterruptible stream reads, but
closing does not unblock a pending FileInputStream/stdin read — scope
the destroy guidance to subprocesses and direct non-closeable streams
to abandonOnInterruptReads. Also: closeOnAbandon permanently closes the
wrapper (no next read); disambiguate "process output stream" to
process.getInputStream; drop releaseAfterScope from the dependency list
(the chapter only warns against it); mention abandonOnInterruptWrites
for the write side; trim a comment duplicating prose.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Chapter 150 gains a "CPU-intensive work" section (plus rows in both
  selection tables): virtual threads are never preempted, so long
  CPU-bound work must run via computeIntensive (or cooperate with
  cede()); scope context deliberately doesn't propagate into the
  compute pool. A matching decision-trigger bullet goes in SKILL.md's
  Performance section, since the skill otherwise pushes all work onto
  virtual threads with no starvation warning.
- Chapter 100 gains a resourceScope section: scoped cleanup without
  claiming a concurrency capability, and `using ResourceScope` as the
  narrower alternative to `using Ox` for registration-only methods —
  reflected in the SKILL.md factories bullet and chapter 150's
  capability callout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamw adamw changed the title Add chapter: subprocesses and external streams (structured teardown) Add chapter: subprocesses and external streams; cover Ox 1.0.6 additions Jul 23, 2026
adamw and others added 4 commits July 23, 2026 10:14
- SKILL.md: fix overstated starvation claim (one computation holds one
  carrier; a few suffice to starve), extend the parent-scope bullet
  with the ResourceScope option.
- Chapter 100: Dependencies.create now declares `using ResourceScope`,
  matching the rule the chapter itself introduces; state the actual
  harm behind the resourceScope compile-time guard (use-after-release).
- Chapter 150: "(by default)" on the carrier count, `import ox.*` in
  the computeIntensive example, note that a `timeout` around a
  non-cooperating computation overshoots.
- Harmonize version phrasing on "(Ox >= 1.0.6)" across additions.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The resources-only -> (using ResourceScope) rule was stated in two
adjacent bullets; keep the full statement in the factories bullet only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Each fact is now stated once: the anti-pattern paragraph merged into its
warning callout, the pipe-EOF mechanism lives only in the process-tree
callout, the teardown-ordering paragraph no longer restates itself, and
the intro no longer pre-summarizes the sections. No facts or callouts
dropped. The SKILL.md Performance bullet loses its chapter link — the
index below already carries it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The bullet had grown into a restatement of the chapter's full teardown
rules. Keep only the hazard (uninterruptible reads, shutdown deadlock)
and the instruction to read the chapter before writing such code.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamw
adamw merged commit 4c9a6fa into master Jul 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant