test(workflow-tengo): e2e coverage for running a command in a 10k-file workdir

[DenKoren]

What

Adds an end-to-end test that runs a command in a working directory holding 10000 files.

• tests/workflow-tengo/src/exec/run/large_workdir.tpl.tengo — fills the workdir with 10000 one-byte files through exec.builder().writeFile in a loop, then runs hello-world in it and saves stdout.
• tests/workflow-tengo/src/exec/large-workdir.test.ts — renders the template and asserts the stdout.

Why

Platforma 4.3.2 fixed: "K8s jobs no longer fail to start when the working directory holds more than ~7300 files; the workdir item list is passed to the job by path". Before the fix the k8s runner passed the expected-workdir-item list to the job inline; above roughly 7300 entries execve returned E2BIG and the job failed before the container started. ExpectedItems []string became ExpectedItemsFileRelativePath string, with the list written to .pl/expected_items inside the workdir (util/k8s/job_params.go, DefaultExpectedItemsFileName).

The fix shipped with unit coverage only (core/pl/util/k8s/template_test.go, TestTemplateWithExpectedItemsFile). Nothing anywhere built a real large workdir and ran a command in it — the defect was found by hand during release validation. This test closes that gap.

Choices

• 10000 files, comfortably above the ~7300 threshold. The template carries a comment saying why, so the number is not "optimised" down later. A count under ~7300 passes on the buggy runner and is worthless for this purpose.
• One byte per file — the item count is what breaks, not the payload size.
• writeFile rather than addFile — no blob, no upload, no signature; just a small value resource per entry.
• hello-world (@platforma-open/milaboratories.software-test-utils:hello-world) because it has both a docker and a binary entrypoint (verified in support/small-binaries/hello-world/package.json), so the same test works under both execution modes.
• The message passed in carries a random suffix: an exec step with a stable CID is deduplicated by the backend, and a cached result would never start a job.

Execution-mode coverage

• monorepo-localfs in core/pl/.github/workflows/test.yaml runs --filter=@platforma-sdk/workflow-tengo-tests with PL_PKG_DEV=local on the local exec runner, so this test gets binary coverage now.
• monorepo-k8s-s3 currently excludes that filter (the external software ships no docker image, so the backend fails with "docker is not set"). When the filter returns, the same test gets docker coverage on the k8s runner. Re-adding it is still blocked on software-anarci and software-conda-empty, which that CI comment also names.

Do not overread this. The original defect is k8s-specific: the local_exec path has no inline-argv mechanism for the expected-item list. The localfs run is therefore adjacent coverage — large-workdir handling, workdir fill rules, the new .pl service-dir reservation — not the same defect in a second mode. The defect itself is covered only once the k8s filter is back.

Verification

Local backend built from core/pl at origin/release/4.3 (60d61697d, reports 4.3.2-1-g60d616), local exec runner, PL_PKG_DEV=local, 8-core macOS.

• New test alone: passes, 35.8 s (44.9 s wall clock including vitest startup).
• Workdir content verified once with a throwaway template that ran ls filler | wc -l in the same workdir: output 10000. That template is not committed (it uses cmd("/usr/bin/env") bash, which has no docker equivalent).
• src/exec suite (63 tests) with the new test: 63/63 passed, 3 m 05 s. Baseline without the new test on the same backend: 62/62 passed, 2 m 17 s. Under suite load the new test takes 62–101 s (it shares two vitest workers with the rest).
• Typecheck (tsc --noEmit) and the package's fmt (oxlint + oxfmt): clean, no reformatting.
• changeset status --since=origin/main: no packages to bump. Only @platforma-sdk/workflow-tengo-tests changed, which is private: true, and .changeset/config.json has privatePackages: false — so no changeset entry applies here.

Honest caveats:

• This 8-core laptop is marginal for the concurrent src/exec suite. Across five runs, timeout-flakiness hit different files each time (writable.test.ts twice, formula.test.ts once) both with and without the new test; one run was invalid because the local backend was killed mid-run. The two clean back-to-back runs above are the comparison that matters. The new test does add measurable load (~50 s of extra suite wall clock here); CI runners are larger (--runner-local-cpu=12, 24 GiB) and the suite already runs with retry: 2.
• Fails-on-buggy-code is unverified. Reproducing the original failure needs a pre-fix backend (v4.3.0) behind a real k8s runner; no k8s runner was available in this environment, and the local exec runner cannot exhibit the defect at all. Treat the test as pinning the fixed behaviour, not as demonstrated to have caught the bug.

No Notion ticket was supplied, and a search of the workspace found no clear match for this defect, so the branch carries no ticket ID.

Greptile Summary

Adds end-to-end regression coverage that creates a 10,000-file Tengo execution workdir and verifies that hello-world starts and returns the expected stdout. The randomized input prevents backend deduplication from satisfying the test without launching a new execution.

• Adds a Tengo fixture that writes 10,000 one-byte files through exec.builder().writeFile.
• Adds a concurrent Vitest case with a five-minute timeout for the resource-heavy render.
• Verifies command execution using software that supports local binary and Kubernetes Docker modes.
• Important touched terms:
  • Large working directory — An execution workdir containing many staged entries; the new fixture creates 10,000 files, above the historical failure threshold.
  • E2BIG — The operating-system error returned when an execve argument or environment payload is too large; the test pins successful execution beyond the former inline-list limit.
  • Tengo template — A workflow script rendered into Platforma backend resources; this PR adds exec.run.large_workdir to construct and execute the large-workdir scenario.
  • Expected-workdir-item list — Metadata describing files expected in a managed workdir; the regression scenario exercises the path-based handling that replaced inline transmission.
  • writeFile — An exec-builder operation that stages value-resource content as a workdir file; the fixture invokes it 10,000 times with one-byte values.
  • hello-world — The test software executed in the populated workdir; its stdout proves that the command started and completed.
  • CID deduplication — Backend reuse of an execution with identical inputs; the randomized message changes the execution identity so each test run starts fresh work.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete correctness or security issues identified in the added regression coverage.

The new test and fixture consistently construct the intended large-workdir condition, force a fresh backend execution, and validate successful command completion through stdout.

Important Files Changed

Filename	Overview
tests/workflow-tengo/src/exec/large-workdir.test.ts	Adds a randomized, extended-timeout integration test that renders the large-workdir fixture and verifies command stdout.
tests/workflow-tengo/src/exec/run/large_workdir.tpl.tengo	Adds a Tengo fixture that stages 10,000 files and runs the cross-mode hello-world software in the resulting workdir.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  T[Vitest creates randomized message] --> R[Render exec.run.large_workdir]
  R --> W[Stage 10,000 one-byte files]
  W --> E[Run hello-world in managed workdir]
  E --> S[Save stdout]
  S --> A[Assert message and newline]

Loading

_{Reviews (1): Last reviewed commit: "test(workflow-tengo): run a command in a..." | Re-trigger Greptile}

Context used:

• Knowledge Base — workflow-tengo: the block workflow authoring SDK

[changeset-bot]

⚠️ No Changeset found

Latest commit: b4b30ac

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 53.03%. Comparing base (df0f3b6) to head (b4b30ac).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1792   +/-   ##
=======================================
  Coverage   53.03%   53.03%           
=======================================
  Files         414      414           
  Lines       21347    21347           
  Branches     4762     4762           
=======================================
  Hits        11321    11321           
  Misses       8662     8662           
  Partials     1364     1364           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.