Skip to content

fix: forward step_name on the Rust runtime and add e2e test-rust script - #1040

Merged
seant-aws merged 1 commit into
aws-deadline:mainlinefrom
seant-aws:rust-forward-and-e2e-script
Aug 7, 2026
Merged

fix: forward step_name on the Rust runtime and add e2e test-rust script#1040
seant-aws merged 1 commit into
aws-deadline:mainlinefrom
seant-aws:rust-forward-and-e2e-script

Conversation

@seant-aws

@seant-aws seant-aws commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What was the problem/requirement? (What/Why)

The Rust runtime accepted step_name but did not forward it to the _v1 session wrapper (the pinned openjd-sessions release lacked the kwarg). Additionally, {{ Task.File.* }} references in command were not resolved in the attachment-sync stopgap path, and there was no way to run the full e2e suite pinned to the Rust runtime.

What was the solution? (How)

  1. Bump openjd-sessions pin from == 0.10.11 to == 0.10.13 (carries fix: forward step_name through the _v1 Session.run_task wrapper OpenJobDescription/openjd-sessions-for-python#345).
  2. Forward step_name to self._session.run_task(...) in rust.py and use the real step name in deserialize_step instead of "Placeholder".
  3. Apply {{ Task.File.* }} regex substitution to command (same regex already used for args).
  4. Add hatch run e2e:test-rust script and a --session-runtime pytest option (WORKER_AGENT_SESSION_RUNTIME env var fallback) so the full suite can run pinned to any runtime.

What is the impact of this change?

The Rust session runtime now resolves WrappedStep.Name correctly for RFC 0008. The Task.File command fix closes a latent gap in the attachment-sync stopgap. The e2e script gives developers a one-command way to validate the Rust runtime.

How was this change tested?

E2E run summary

  • started: 2026-08-05T23:59:10Z
  • commit: 5190130 fix: thread step_name to run_task for wrap-action support
  • wheel: dist/deadline_cloud_worker_agent-0.30.2.post1.dev22-py3-none-any.whl
run attempt exit pytest summary
linux-python 1 0 ================= 53 passed, 25 skipped in 5393.90s (1:29:53) ==================
linux-rust 1 0 ================= 53 passed, 25 skipped in 5468.02s (1:31:08) ==================
windows-python 1 0 ====== 51 passed, 19 skipped, 8 xfailed, 3 warnings in 9228.77s (2:33:48) ======
windows-rust 1 0 = 51 passed, 19 skipped, 3 xfailed, 5 xpassed, 4 warnings in 9871.06s (2:44:31) =
  • finished: 2026-08-06T08:12:33Z

  • Unit tests: 98 passed against a local openjd-sessions wheel carrying fix: install-deadline-worker on Linux assumes agent os group matches username #345

  • Ruff check + format clean

  • E2E (overnight 2026-08-05): full matrix (linux-python, linux-rust, windows-python, windows-rust) all green — 53/53 Linux, 51/51 Windows per run

  • CI will fail on the pin until openjd-sessions 0.10.13 publishes to PyPI

Was this change documented?

Yes — DEVELOPMENT.md updated with the new e2e invocations.

Is this a breaking change?

No.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

file_paths[embedded_file.name] = path

command = str(step_script.actions.onRun.command)
command = re.sub(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question - this seems like a workaround for openjd file interpolation. Can we put it back inside openjd code?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea we have talked about this before. I can provide some better refactoring along with further rust / extension bugfixes that we need to put out

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i put this as a ticket I will tackle after a first initial release with the python containers stuff working, sounds good ?

@leongdl

leongdl commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

The command fix is the right call — the args path had this and command didn't, so a step whose
command is the embedded script would have executed the literal {{ ... }} token.

Two follow-ups on the pattern itself:

The pattern can't match property access. \{\{\s*Task\.File\.(\w+)\s*\}\} requires }}
immediately after the name, so the RFC 0005 form {{Task.File.Run.name}} doesn't match at all and
passes through as a literal. openjd-rs has a fix at the model validation layer for exactly this form
(branch fix/task-file-property-access, "fix(model): Accept Task.File property access", with a note
in validate_v2023_09/structure.rs:299 that RFC 0005 uses {{Task.File.Run.name}} directly). I
haven't confirmed whether the attachment-sync generator emits that form today — but the coupling is
the point: this regex has to track whatever the generator emits, where the engine wouldn't.

Beyond property access, the hand-rolled substitution also drops, relative to
resolve_action_args (openjd-rs runner/mod.rs:461): expression syntax and function-library calls,
list-valued arg expansion and ExprValue::Null arg skipping (:483-491), the runnable execute
bit (embedded_files.rs:88 sets 0o700 vs 0o600; the adapter always writes S_IRUSR|S_IWUSR),
filename honoring plus validate_resolved_filename, and all other symbols
(Task.Param.*, Session.*, script-level let bindings).

Nit: the pattern and lambda are now duplicated verbatim between command and the args loop.
Worth a module-level _TASK_FILE_RE = re.compile(...).

Both of these argue for the openjd-sessions#332 API instead — I've left a note there asking to
reopen, since the "Rust runtime is POSIX-only" premise it was closed on isn't enforced by
_select.py, and this PR's new --session-runtime=rust knob makes a Windows Rust-runtime worker
easy to configure.

Forward step_name to the _v1 session's run_task so RFC 0008's
WrappedStep.Name resolves correctly on the Rust path. Also use
the real step name in deserialize_step instead of 'Placeholder'.

Bump openjd-sessions pin from 0.10.11 to 0.10.13 — the first
release whose _v1 wrapper accepts step_name.

Add hatch run e2e:test-rust (runs the full e2e suite pinned to
the Rust session runtime) and a --session-runtime pytest option /
WORKER_AGENT_SESSION_RUNTIME env var for ad-hoc runtime selection.

Signed-off-by: Sean Tang <171081544+seant-aws@users.noreply.github.com>
@seant-aws
seant-aws force-pushed the rust-forward-and-e2e-script branch from c6f0f4b to a955d74 Compare August 6, 2026 23:34
@github-actions github-actions Bot added the waiting-on-maintainers Waiting on the maintainers to review. label Aug 7, 2026
@seant-aws
seant-aws marked this pull request as ready for review August 7, 2026 17:52
@seant-aws
seant-aws requested a review from a team as a code owner August 7, 2026 17:52
@seant-aws

Copy link
Copy Markdown
Contributor Author

I will address the refactoring for OpenJobDescription/openjd-sessions-for-python#332 as a followup after initial python container release

@seant-aws
seant-aws merged commit 085fccf into aws-deadline:mainline Aug 7, 2026
34 of 64 checks passed
@seant-aws
seant-aws deleted the rust-forward-and-e2e-script branch August 7, 2026 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

waiting-on-maintainers Waiting on the maintainers to review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants