diff --git a/queue_environments/README.md b/queue_environments/README.md index b3a88e13..25fbc190 100644 --- a/queue_environments/README.md +++ b/queue_environments/README.md @@ -17,6 +17,7 @@ This table covers every immediate user-selectable queue environment or collectio | [Rez shim environment](rez_shim/) | Wrapping each task in a resolved Rez context through `PATH` shims | Rez software needs shell functions, aliases, or ordered `PATH` edits | | [Pip environment](pip_queue_env.yaml) | Creating a Python `venv` and installing job-selected pip packages | Jobs need Python packages without Conda or Rez | | [Disconnect UBL](disconnect_ubl_queue_env.yaml) | Removing Deadline Cloud Usage Based License environment variables | A queue must use only a custom license server | +| [Short path mapping junctions](windows_path_limit_junction_fix.yaml) | Junctioning job attachment directories to short paths and republishing path mapping rules through them | A Windows application fails on long input paths despite long path support | ## Create a queue environment for your queue @@ -144,3 +145,15 @@ Workers need `python3` or `python` on `PATH`. Service-managed fleets provide one ### Disconnect UBL The disconnect environment unsets Deadline Cloud Usage Based License variables so jobs use a custom license server. Give it a higher-precedence position than other environments, such as priority `0`, so later licensing setup is not removed. Review [Bring Your Own License](https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/smf-byol.html). Additional UBL variables can be introduced over time, so review the template against current service behavior before deployment. + +### Short path mapping junctions + +Windows imposes a 260 character path limit, and applications that use legacy Win32 APIs, such as Adobe After Effects and Cinema4D, keep failing on longer paths even when long path support is enabled through the registry. Deadline Cloud downloads job attachments into a hash-named directory inside the session working directory, which alone accounts for roughly 107 characters on a default Windows worker. + +This environment creates a Windows directory junction to each job attachment directory at a short numbered path, then writes a new [`pathmapping-1.0`](https://github.com/OpenJobDescription/openjd-specifications/wiki/How-Jobs-Are-Run#path-mapping) rules file whose destinations point through those junctions, and names it in `DEADLINE_ASSETROOT_PATHMAPS`. Junctions are a Windows feature, so on a Linux or macOS worker both actions exit immediately without creating anything, which makes the environment harmless to leave on a queue that also serves non-Windows fleets. + +The session's own path mapping rules cannot be modified, which is why the environment emits its own copy of them rather than adjusting them in place. Jobs therefore have to opt in by feeding that copy to whatever already consumes path mapping rules in place of `{{Session.PathMappingRulesFile}}`. For an adaptor built on the [Open Job Description adaptor runtime](https://github.com/OpenJobDescription/openjd-adaptor-runtime-for-python), pass it as `--path-mapping-rules` and the existing `map_path()` calls return short paths unchanged. Applications without an adaptor need the same substitution done by hand. + +Junction slots are claimed by attempting to create them and moving to the next number when creation fails, which is safe against the several sessions a single worker host can run at once. `JunctionPathPrefix` is the only parameter; the slot search is capped at 100, matching the most sessions one worker agent can run concurrently. + +The actions are Python, run with `command: python` as the [job attachments copy](../job_bundles/copy_s3_prefix_to_job_attachments/) sample does. Workers have Python because the worker agent depends on it. diff --git a/queue_environments/windows_path_limit_junction_fix.yaml b/queue_environments/windows_path_limit_junction_fix.yaml new file mode 100644 index 00000000..19890896 --- /dev/null +++ b/queue_environments/windows_path_limit_junction_fix.yaml @@ -0,0 +1,276 @@ +specificationVersion: 'environment-2023-09' +parameterDefinitions: +- name: JunctionPathPrefix + type: STRING + default: 'C:\tmp\dc' + description: > + Prefix for the short junction paths. A number is appended to it to produce + each junction, so the default yields C:\tmp\dc1, C:\tmp\dc2, and so on. Keep + this as short as you can: every character here is one less character of + headroom under the 260 character limit. +environment: + name: Short Path Mapping Junctions + description: | + Shortens the paths that a session's input files are reachable through, to work + around the Windows MAX_PATH (260 character) limitation that affects + applications using legacy Win32 APIs (e.g. Adobe After Effects, Cinema4D). + Enabling long path support with the Windows registry key does not help those + applications, because the limit is imposed by the application rather than by + the filesystem. + + Deadline Cloud downloads job attachments into a directory inside the session + working directory, which produces a long prefix -- roughly 107 characters on a + default Windows worker, made up of the session root, the session directory + name, and the hashed attachment directory name. This environment creates a + directory junction to that directory at a much shorter path, saving about 95 + characters. + + This environment does the following: + + 1. Reads the session's path mapping rules and finds every rule whose + destination points into the session working directory. + 2. Creates a directory junction to each of those destinations at a short + numbered path (see JunctionPathPrefix). + 3. Writes its own copy of those rules, in the same `pathmapping-1.0` format + the session uses, with the destinations repointed through the junctions. + Every other rule and field is copied over untouched. A copy is necessary + because the session's own rules cannot be modified. + 4. Sets DEADLINE_ASSETROOT_PATHMAPS to the absolute path of that file. + + The junctions are removed when the session ends. + + Using it is the job's responsibility. This environment does not and cannot + change the rules the session itself applies, so paths your job receives from + Deadline Cloud are still the long ones. A step that wants the short paths has + to feed the file named by DEADLINE_ASSETROOT_PATHMAPS to whatever already + consumes path mapping rules, in place of {{Session.PathMappingRulesFile}}. + + For an adaptor built on the Open Job Description adaptor runtime, that means + passing this file as the `--path-mapping-rules` value, after which the + existing `map_path()` calls on both the adaptor and the in-application client + return short paths with no other changes. + + Applications without an adaptor need the equivalent done by hand, and how + differs from application to application -- a command-line flag, a project file + that has to be patched, a scripting API. + + Junctions are a Windows feature, so on a Linux or macOS worker both actions + exit immediately without creating anything. That makes the environment + harmless to leave on a queue that also serves non-Windows fleets. + + The actions run `python`, which Deadline Cloud workers have because the worker + agent depends on it. + script: + actions: + onEnter: + command: python + args: + - "{{Env.File.Enter}}" + - "{{Session.WorkingDirectory}}" + - "{{Session.PathMappingRulesFile}}" + - "{{Param.JunctionPathPrefix}}" + onExit: + command: python + args: + - "{{Env.File.Exit}}" + - "{{Session.WorkingDirectory}}" + embeddedFiles: + - name: Enter + filename: junction-enter.py + type: TEXT + data: | + """Point this session's path mapping rules through short junctions. + + Values arrive as arguments rather than being substituted into the script + body, so no template value has to survive Python quoting. + + Usage: junction-enter.py SESSION_DIR RULES_FILE JUNCTION_PATH_PREFIX + """ + import json + import os + import subprocess + import sys + + # Upper bound on the numbered slots to try. The Deadline Cloud worker agent + # runs each of its assigned sessions on a thread from a pool of 100, so 100 + # is the most sessions one agent can have competing for slots. Walking all + # of them is cheap, so this is deliberately not a parameter. + MAX_SLOTS = 100 + + # Junctions are recorded here as they are claimed so that onExit can clean + # up whatever was claimed even if this script fails partway through. + STATE_FILENAME = "dc-short-path-junctions.txt" + + OUTPUT_FILENAME = "dc-short-path-mapping-rules.json" + + + def claim_junction(prefix, target, state_path): + """Create a junction to target at the first free numbered slot. + + mklink fails when the link name already exists, and creating the + directory entry is atomic, so attempting the create is itself the claim. + Concurrent sessions on this host race safely: the loser sees a non-zero + exit code and moves on to the next number. Nothing here removes an + existing junction, because it may belong to a live session. + """ + for slot in range(1, MAX_SLOTS + 1): + link = "{}{}".format(prefix, slot) + result = subprocess.run( + ["cmd", "/c", "mklink", "/J", link, target], + capture_output=True, + text=True, + ) + if result.returncode == 0: + # Recorded before anything else can fail, so cleanup never + # misses a junction that exists. + with open(state_path, "a", encoding="utf-8") as state_file: + state_file.write(link + "\n") + print("Junction created: {} -> {}".format(link, target)) + return link + + sys.exit( + "Could not claim a junction for {}: all {} slots starting at {}1 are " + "in use. If this host is not running that many sessions, junctions " + "have most likely been left behind by sessions that did not exit " + "cleanly; remove the stale {}* junctions on the worker.".format( + target, MAX_SLOTS, prefix, prefix + ) + ) + + + def session_relative_parts(destination, session_dir): + """Split destination into (top level session subdirectory, remainder). + + Returns None when destination is not inside session_dir, which covers + shared storage locations that are already wherever the worker mounted + them and so are not worth shortening. + """ + try: + relative = os.path.relpath(os.path.abspath(destination), session_dir) + except ValueError: + # Different drive, so not inside the session directory. + return None + + # os.path.relpath normalizes case on Windows, so this comparison is + # case-insensitive there, as the filesystem is. + if relative == os.curdir or relative.startswith(os.pardir): + return None + + top, _, remainder = relative.partition(os.sep) + return top, remainder + + + def main(argv): + session_dir, rules_file, prefix = argv[1], argv[2], argv[3] + + if sys.platform != "win32": + print("Not a Windows worker; no junctions created.") + return 0 + + with open(rules_file, encoding="utf-8") as handle: + document = json.load(handle) + + # The rules file is allowed to be an empty object when the session has + # no path mapping rules. + rules = document.get("path_mapping_rules") or [] + if not rules: + print("Session has no path mapping rules; nothing to shorten.") + return 0 + + session_dir = os.path.abspath(session_dir) + state_path = os.path.join(session_dir, STATE_FILENAME) + + # The junction root has to exist before mklink can create a link inside + # it. Another session may be doing this at the same moment, so tolerate + # the directory already being there. + prefix_parent = os.path.dirname(prefix) + if prefix_parent: + os.makedirs(prefix_parent, exist_ok=True) + + junction_for_subdir = {} + + # Walk the session's own rules and repoint the destinations that get a + # junction. Rules left alone here are written back out unchanged, as is + # every field of every rule other than the destinations replaced below. + for rule in rules: + destination = rule.get("destination_path") + if not destination: + continue + + parts = session_relative_parts(destination, session_dir) + if parts is None: + continue + top, remainder = parts + + # One junction serves every rule pointing into the same attachment + # directory; anything deeper is carried across unchanged. + if top not in junction_for_subdir: + junction_for_subdir[top] = claim_junction( + prefix, os.path.join(session_dir, top), state_path + ) + + junction = junction_for_subdir[top] + rule["destination_path"] = ( + os.path.join(junction, remainder) if remainder else junction + ) + + if not junction_for_subdir: + print( + "No path mapping rule points into the session directory; " + "nothing to shorten." + ) + return 0 + + output_path = os.path.join(session_dir, OUTPUT_FILENAME) + with open(output_path, "w", encoding="utf-8") as handle: + json.dump(document, handle) + + print("Shortened path mapping rules written to: {}".format(output_path)) + print("openjd_env: DEADLINE_ASSETROOT_PATHMAPS={}".format(output_path)) + return 0 + + + if __name__ == "__main__": + sys.exit(main(sys.argv)) + - name: Exit + filename: junction-exit.py + type: TEXT + data: | + """Remove the junctions this session claimed. + + Usage: junction-exit.py SESSION_DIR + """ + import os + import sys + + STATE_FILENAME = "dc-short-path-junctions.txt" + + + def main(argv): + state_path = os.path.join(os.path.abspath(argv[1]), STATE_FILENAME) + + if not os.path.exists(state_path): + print("No junctions were recorded for this session; nothing to remove.") + return 0 + + with open(state_path, encoding="utf-8") as handle: + links = [line.strip() for line in handle if line.strip()] + + # Best effort: keep going after a failure so that one junction we cannot + # remove does not leave the rest behind, and do not fail the session over + # cleanup. + for link in links: + try: + # rmdir removes the junction itself and leaves its target alone. + os.rmdir(link) + print("Junction removed: {}".format(link)) + except FileNotFoundError: + pass + except OSError as error: + print("Warning: failed to remove junction {}: {}".format(link, error)) + + return 0 + + + if __name__ == "__main__": + sys.exit(main(sys.argv))