feat: added queue env example that circumvents the Windows path limit with junctions - #277
feat: added queue env example that circumvents the Windows path limit with junctions#277yuanmich2 wants to merge 1 commit into
Conversation
…tions Signed-off-by: Michael Yuan <160265179+yuanmich2@users.noreply.github.com>
| The junction is removed when the session ends. | ||
|
|
||
| This queue environment is only effective on Windows workers. On non-Windows | ||
| workers the onEnter script exits successfully without creating a junction. |
There was a problem hiding this comment.
The description promises that "on non-Windows workers the onEnter script exits successfully without creating a junction," but nothing in the script implements that guard. command: powershell invokes Windows PowerShell, which does not exist on Linux/macOS workers, so the onEnter action will fail to launch (or, if pwsh were used, cmd /c mklink would fail) rather than exiting 0. Either add an early OS check that exits successfully on non-Windows (e.g. if (-not $IsWindows) { exit 0 }), or adjust the description so it does not claim graceful non-Windows behavior. As written the environment will error the session on any non-Windows worker.
| } | ||
|
|
||
| # Create junction | ||
| $result = cmd /c "mklink /J $junctionPath $($assetroot.FullName)" 2>&1 |
There was a problem hiding this comment.
The path arguments to cmd /c are not quoted, so any space in $junctionPath or in the assetroot path will break these commands. mklink /J C:\my dir\dc-root C:\...\assetroot-123 parses as multiple arguments and fails; the same applies to rmdir $junctionPath on lines 51 and 83. The JunctionPath default is space-free, but a user can override it, and session working directories can contain spaces. Quote the paths inside the cmd /c string, e.g. cmd /c "mklink /J "$junctionPath" "$($assetroot.FullName)"" (and likewise rmdir "...").
| type: TEXT | ||
| data: | | ||
| $ErrorActionPreference = 'Stop' | ||
| $junctionPath = '{{Param.JunctionPath}}' |
There was a problem hiding this comment.
JunctionPath is a single fixed location (default C:\tmp\dc-root), but a Deadline Cloud worker can run multiple sessions concurrently. Two sessions using this environment will fight over the same junction: the second onEnter deletes the first session's junction (via the "remove stale junction" block) and repoints C:\tmp\dc-root at its own assetroot, so the first session's DEADLINE_ASSETROOT_ALIAS now silently resolves to the wrong assetroot. Worse, whichever session exits first removes the junction out from under the still-running session. Consider making the junction path unique per session (e.g. append a short session-derived suffix) so concurrent sessions do not corrupt each other's asset paths, or document that this environment is only safe on single-session fleets.
There was a problem hiding this comment.
A worker can run multiple sessions concurrently??
|
|
||
| # Find the assetroot folder in the session working directory | ||
| $sessionDir = $PWD.Path | ||
| $assetroot = Get-ChildItem $sessionDir -Directory | Where-Object { $_.Name -like 'assetroot-*' } | Select-Object -First 1 |
There was a problem hiding this comment.
This assumes a single assetroot folder and blindly takes Select-Object -First 1. A session can have more than one assetroot-* folder (multiple input path mappings / storage profile locations). When that happens, only one is aliased and jobs referencing paths under the other assetroots will resolve incorrectly through the junction. The selection is also order-dependent (whatever Get-ChildItem returns first), so which assetroot wins is not deterministic. Consider handling the multi-assetroot case explicitly, or at least documenting the single-assetroot assumption.
There was a problem hiding this comment.
Will address tomorrow
|
this relies on aws-deadline/deadline-cloud-for-after-effects#324, right? worth mentioning in the PR description |
|
I wrote |
Fixes: aws-deadline/deadline-cloud-for-after-effects#152
What was the problem/requirement? (What/Why)
Windows has a default 260 character path limit. Even if you enable longer paths with a registry key, Some applications like C4D and After Effects will still fail to find paths with more than 260 characters.
What was the solution? (How)
This queue environment creates junctions at a short path that points to the longer assetroot directory that deadline renders use. This saves roughly 80 to 90 characters.
What is the impact of this change?
Users who use this queue environment should be able to render successfully more often with longer name file and folder paths.
How was this change tested?
I tested this queue environment with an After Effects render. Note that for this queue environment to have any effect, the render command must be adjusted.
This was the output:

Was this change documented?
The sample's description describes the problem it's solving and how it works. Here is the description:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.