Skip to content

feat: browser-native Globus Transfer + plugin detection - #59

Merged
scttfrdmn merged 2 commits into
mainfrom
feat/globus-transfer-and-plugins
Aug 1, 2026
Merged

feat: browser-native Globus Transfer + plugin detection#59
scttfrdmn merged 2 commits into
mainfrom
feat/globus-transfer-and-plugins

Conversation

@scttfrdmn

Copy link
Copy Markdown
Contributor

Phase 3 of the parity/advisor/execution plan. Data movement in and out of a
launched instance, plus the ability to say what's deployed on one — with an
honest split between what a browser can do and what belongs to the CLI.

Globus does not need a local machine

This corrects an earlier claim of mine, and the correction is what makes the
phase possible. The globus-personal-endpoint plugin needs globus whoami
locally, but that plugin wraps Globus Connect Personal specifically. Globus
Transfer is a REST API over managed collections — an HPC DTN, an S3
collection — and it is fully CORS-enabled. Preflighted live from
Origin: https://spore.host:

endpoint ACAO
transfer.api.globus.org/v0.10/endpoint_search *
transfer.api.globus.org/v0.10/task_list *
auth.globus.org/v2/oauth2/token *

So src/transfer/globus.ts is fetch only — no SDK — mirroring how
src/auth/globus.ts already talks to Globus.

It reuses the sign-in we already have. The transfer scope goes into the
requested scopes, and completeLogin already walks the per-resource-server
other_tokens array, so the Transfer token comes back from the same sign-in
that produces the STS id_token. GlobusTokens grows a transferToken; there
is no second auth flow.

submit_transfer fetches a submission_id first, as the API requires — that's
what makes a retried submit idempotent rather than moving the data twice.

The browser's job with plugins is detection, not execution

spore-sync (mutagen), rclone, tailscale, gha-runner and Globus Connect Personal
are launched by the CLI. That's their proper home, and the web view should not
reimplement them. What the web view must do is report what's deployed, and the
wire already supports it: spawn's recordPluginProvenanceTag writes a
spore:plugin:<name> EC2 tag valued
version=…;sha256=…;commit=…;verify=signature|manifest|none, and
DescribeInstances is already in the browser's reach.

parsePluginTag is pure, and its central property is that an unrecognised or
absent tag yields "unknown" — never a fabricated "not installed". Those are
different claims: one says we couldn't tell, the other asserts a fact about the
instance we have no basis for. Same invariant as Go #63.

This splits the plugin surface three ways, and the split is part of the
deliverable:

browser can install browser can detect
7 remote-only (jupyterlab, code-server, rstudio-server, docker, mountpoint-s3, cloudwatch-agent, vscode-tunnel) yes — pure userdata yes
Globus Transfer yes — REST + the existing auth yes
Globus Connect Personal, spore-sync, rclone, tailscale, gha-runner no — the CLI's job yes, via the tag

Reporting that third group as absent would read as "no plugin support in the
browser", which is false.

mountpoint-s3 pairs with Transfer for the other direction — S3-as-filesystem on
the instance, also remote-only. Between them, data moves in and out with zero
locally installed software.

Verification

npm run typecheck clean; 382 tests passing, with no credentials and no
network anywhere:

  • the transfer client takes an injected fetch (the convention
    completeLogin already uses), driven through a canned
    submit_transfertask/<id> polling sequence
  • parsePluginTag is pure and round-trips the real Go tag format, including the
    unknown/absent cases

Live Globus transfer moves real data, so it stays a manual, opt-in check —
never CI.

Docs: docs/data-movement.md.

…-compatible overstatement

Two things in integration.md were wrong in ways that changed which features
read as portable:

- The "live AWS data ... needs credentials a browser can't safely hold and hits
  CORS" paragraph was false on both counts. Preflighted from
  Origin: https://spore.host, ec2 / api.pricing / servicequotas in us-east-1
  each return access-control-allow-origin: *, POST, and
  content-type,x-amz-target,authorization. And credsFromIdToken already puts
  short-lived STS credentials in the tab. BundledFinder stays the default for
  cost and latency, which is the real reason.

- "wire-compatible" read as complete. Go's buildLaunchTags stamps 55 tags;
  spawn-ts stamps 33. Now names what is absent in the same tier language as the
  rest of the section, so the base-identity block reads as tier B (#51) rather
  than as an unstated hole.

Refs #57.
Two halves of the same problem — a job is useless without a way to get data in
and out — with the honest split between what a browser can do and what belongs
to the CLI.

## Globus Transfer (src/transfer/globus.ts)

Globus does NOT need a local machine. My earlier reading was wrong in a way worth
stating plainly, because it changes the tier: the globus-personal-endpoint plugin
needs `globus whoami` locally, but that plugin wraps Globus Connect *Personal*
specifically. Globus *Transfer* is a REST API over managed collections (an HPC
DTN, an S3 collection), and it is fully CORS-enabled — preflighted live from
Origin: https://spore.host, ACAO `*` on endpoint_search, task_list, and the auth
token endpoint.

So this is `fetch` only, no SDK, mirroring how src/auth/globus.ts already talks
to Globus. And it reuses the sign-in we already have rather than adding a second
auth system: the transfer scope goes into the requested scopes, and completeLogin
already walks the per-resource-server `other_tokens` array, so the Transfer token
comes back from the SAME sign-in that produces the STS id_token. GlobusTokens
grows a transferToken; there is no new flow.

submit_transfer fetches a submission_id first, as the API requires — that's what
makes a retried submit idempotent instead of moving the data twice.

## Plugin detection (src/core/plugins.ts)

The browser's job here is detection, not execution. spore-sync (mutagen), rclone,
tailscale, gha-runner and Globus Connect Personal are launched by the CLI; that's
their proper home and the web view should not reimplement them. What the web view
must do is report what's deployed, and the wire already supports it: spawn's
recordPluginProvenanceTag writes a `spore:plugin:<name>` EC2 tag valued
`version=…;sha256=…;commit=…;verify=signature|manifest|none`. DescribeInstances
is already in the browser's reach.

parsePluginTag is pure, and its central property is that an unrecognised or
absent tag yields "unknown" — never a fabricated "not installed". Those are
different claims: one says we couldn't tell, the other asserts a fact about the
instance we have no basis for. Same invariant as Go #63.

This splits the plugin surface three ways rather than two, and the split is the
deliverable: 7 remote-only plugins (jupyterlab, code-server, rstudio-server,
docker, mountpoint-s3, cloudwatch-agent, vscode-tunnel) are pure userdata, so the
browser can both install and detect them; Globus Transfer the browser can do
itself; the local-half plugins it can only detect. Reporting the third group as
absent would read as "no plugin support in the browser", which is false.

mountpoint-s3 pairs with Transfer for the other direction — S3-as-filesystem on
the instance, also remote-only. Between them, data moves in and out with zero
locally installed software.

## Tests

382 passing, no credentials and no network anywhere: the transfer client takes an
injected fetch (the convention completeLogin already uses), driven through a
canned submit_transfer → task/<id> polling sequence, and parsePluginTag is pure
and round-trips the real Go tag format. Live Globus transfer moves real data, so
it stays a manual opt-in check — never CI.

Docs: docs/data-movement.md.
@scttfrdmn
scttfrdmn merged commit 3da2170 into main Aug 1, 2026
1 check passed
@scttfrdmn
scttfrdmn deleted the feat/globus-transfer-and-plugins branch August 1, 2026 05:12
scttfrdmn added a commit that referenced this pull request Aug 1, 2026
Cutting 0.7.0 surfaced that the process had already drifted, so this does both:
the release, and the mechanism that stops the drift recurring.

The drift found while reconstructing the notes:

- 0.6.0 and 0.6.1 were both published to npm with NO CHANGELOG section, and
  0.6.0 was never tagged at all. Both sections are now written retroactively
  from the commit range.
- `[Unreleased]` therefore still compared from v0.5.0 and had accumulated
  entries that had already shipped in 0.6.0 (#28, #38). Moved.
- The six audit fixes (#50, #51, #54, #55, #56, #58) had landed on main with no
  CHANGELOG entry at all. Written from the commit bodies.
- publish.yml's subpath guard was a hardcoded list that never gained
  `./transfer` when #59 added that subpath — so the guard had silently stopped
  covering a real entry point. That is the failure mode of a duplicated list, so
  scripts/check-exports.mjs now derives the check from `exports` itself and
  checks every target, not only the .d.ts. Verified it fails on exactly the
  subpath the old list was missing.

New `npm run release:check` is the preflight. Every assertion in it maps to a
drift above rather than to a hypothetical, and each was verified to fail when
violated (undated section, missing compare link, stale [Unreleased] link,
version with no section, already-published version). It reports ALL problems at
once, and a check it cannot run (no git, no network) is reported as unknown
rather than as passing.

It immediately earned itself: it caught LIB_VERSION still reading 0.6.1.
That constant is stamped into the `spawn:version` launch tag and read by Go's
pkg/aws/ami_mgmt.go, so shipping it stale would have mislabelled every instance
launched by 0.7.0. Bumped, and the coupling is now documented at the bump step
in docs/releasing.md rather than left for someone to remember.

docs/releasing.md writes the process down: tag last, tag from main, tag only a
pushed green commit, and never move a published tag — fix forward, as truffle-ts
v0.4.1 did.

0.7.0 is MINOR, not PATCH: `launch` now refuses a spec it previously accepted
(#51) and the unbounded-launch guard changed which options satisfy it (#55). A
type-compatible change that rejects previously-valid input is still a break.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant