feat: python -m shard.stage + P0-#2 portability fixes#104
Merged
Conversation
Both broke the first residential join (LAUNCH.md P0-#2): - node_kv's flat `import transport` (the pushed-flat box layout) now falls back to shard.transport, so a repo checkout needs no hand-set PYTHONPATH. - m25_pull_range read /root/.hf_token unconditionally and died on any non-vast box: now env HF_TOKEN wins, else ~/.hf_token, else huggingface_hub's own login chain (public/cached repos need none). Pinned by tests/test_shard_stage.py (lands with the shard.stage entrypoint): the clean-env no-PYTHONPATH subprocess gate + the token-resolution unit.
…emon Promotes the operator SSH launch string (m25_scatter_pipe.stage_cmd) into the CLI the Leg-7 node daemon execs (c0mpute NODE_DAEMON.md §4): the ring assignment arrives as flags, the engine env is derived in-process, and the stage speaks a machine-readable stdout contract a supervisor can wait on (SHARD_STAGE_OK preflight / SHARD_STAGE_READY from serve() / SHARD_STAGE_FATAL + nonzero exit). Secrets stay env-only, never argv. --check gives the daemon a no-network preflight (engine import + model-dir sanity). Also: the receipt node-key default is ~-relative (identical on root boxes, works elsewhere). tests/test_shard_stage.py: clean-env subprocess --check (PYTHONPATH stripped — the P0-#2 regression gate), FATAL contract on bad dir/missing assignment, the READY line + a frame flowing through the real serve() loop on CPU, HF-token resolution order. Full suite 554 passed / 1 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First engine seam of Leg 7 (NODE_DAEMON.md §4 / §8-2), plus the two portability landmines that broke the first residential join (LAUNCH.md P0-#2):
python -m shard.stage— the operator SSH launch string promoted into the CLI the node daemon execs: assignment as flags, engine env derived in-process, machine-readable stdout contract (SHARD_STAGE_OK/SHARD_STAGE_READY/SHARD_STAGE_FATAL+ nonzero exit),--checkpreflight. Secrets stay env-only.import transport→ falls back toshard.transport(no hand-set PYTHONPATH off the flat vast layout)./root/.hf_tokenhardcode → envHF_TOKEN→~/.hf_token→ huggingface_hub login chain.~-relative (identical on root boxes).Tests: clean-env no-PYTHONPATH subprocess gate, stdout-contract pins, a frame through the real
serve()loop on CPU, token-resolution order. Full suite 554 passed / 1 skipped.