-
Notifications
You must be signed in to change notification settings - Fork 63
feat: add conda recipe for houdini 22.0 #281
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,202 @@ | ||
| # Houdini 22.0 Conda Recipe for AWS Deadline Cloud | ||
|
|
||
| ## Overview | ||
|
|
||
| This directory contains a conda build recipe for Houdini 22.0.368, configured for use with AWS Deadline Cloud. The package runs Houdini rendering and processing jobs on Deadline Cloud service-managed fleets. | ||
|
|
||
| ## Package Information | ||
|
|
||
| - **Application**: Houdini 22.0.368 | ||
| - **Supported Platforms**: linux-64 | ||
| - **Source**: SideFX Houdini downloads page | ||
| - **License**: SideFXEULA | ||
| - **Build Tool**: rattler-build | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Before building this package, ensure you have: | ||
|
|
||
| 1. **AWS Deadline Cloud infrastructure** set up with: | ||
| - A farm configured for package building. See https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/cloudformation/farm_templates/starter_farm for instructions to create a Farm. | ||
| - A queue for building packages. The submit command looks for a queue whose name starts with "Package". See https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/configure-jobs-s3-channel.html for instructions on creating one. | ||
| - Linux-64 fleet for building linux packages | ||
|
|
||
| 2. **Deadline Cloud CLI** installed on your workstation | ||
|
|
||
| 3. **SideFX account** for downloading Houdini installer | ||
|
|
||
| 4. **Source archive** (see [Archive File Instructions](#archive-file-instructions) below) | ||
|
|
||
| ## Archive File Instructions | ||
|
|
||
| ### Linux | ||
|
|
||
| #### Download from SideFX | ||
| 1. Download the `houdini-22.0.368-linux_x86_64_gcc14.2.tar.gz` from SideFX Houdini's downloads page | ||
| 2. Clone this repository locally. | ||
| 3. Place the downloaded file in the `conda_recipes/archive_files` directory | ||
|
|
||
| Note that Houdini 22.0 builds are compiled with GCC 14.2, whereas Houdini 21.0 and | ||
| earlier used GCC 11.2. Make sure you download the `gcc14.2` archive, or the build | ||
| will fail to find the installer. | ||
|
|
||
| ## Plugin Integration | ||
|
|
||
| ### Plugins | ||
|
|
||
| Houdini supports plugins through the use of package files. A package is a json file that tells Houdini where to find plugins. | ||
| [Houdini Plugin Reference](https://www.sidefx.com/docs/houdini/ref/plugins.html). | ||
|
|
||
| Create your package files in `$PREFIX/opt/houdini/packages` and point them to the location of your plugins. See our Redshift for | ||
| Houdini recipe as [an example](../houdini-redshift-2026/). | ||
|
|
||
| ### Plugin Installation Paths | ||
|
|
||
| The conda recipe configures the following environment variables and paths for plugin discovery: | ||
|
|
||
| ```bash | ||
| # Environment variables set by this package | ||
| export HOUDINI_LOCATION=$PREFIX/opt/houdini | ||
|
|
||
| # Plugin search paths (in order of precedence) | ||
| $PREFIX/opt/houdini/packages | ||
| ``` | ||
|
|
||
| ### Creating Plugin Packages | ||
|
|
||
| 1. **Plugin Package Structure** | ||
| ``` | ||
| my-houdini-plugin/ | ||
| ├── recipe/ | ||
| │ ├── recipe.yaml | ||
| │ ├── build.sh | ||
| │ └── bld.bat | ||
| ├── deadline-cloud.yaml | ||
| └── README.md | ||
| ``` | ||
|
|
||
| 2. **Plugin Installation Script Example** | ||
|
|
||
| NOTE: Your plugin files can be anywhere as long as your package file points to their directory. | ||
|
|
||
| ```bash | ||
| # In build.sh | ||
| mkdir -p $PREFIX/opt/houdini/packages | ||
| cp my-plugin.json $PREFIX/opt/houdini/packages/ | ||
| cp -r plugin-files/ $PREFIX/opt/houdini/plugin/ | ||
| ``` | ||
|
|
||
| 3. **Plugin Dependencies** | ||
| - Add this Houdini package as a dependency in your plugin's `recipe.yaml` | ||
| - Specify version constraints: `houdini >=22.0,<22.5` | ||
|
|
||
| ### Plugin Sync | ||
|
|
||
| This recipe includes Plugin Sync support, which allows customers to deliver plugins | ||
| to workers via S3 without building a separate conda package. | ||
|
|
||
| To use Plugin Sync, upload your plugin files and a Houdini package descriptor | ||
| (`.json` file) to the S3 path: | ||
|
|
||
| ``` | ||
| s3://<job-attachments-bucket>/<root-prefix>/plugins/linux/houdini/22.0/ | ||
| ``` | ||
|
|
||
| The `.json` package descriptor should reference `$DEADLINE_CLOUD_HOUDINI_PLUGIN_SYNC_DIR` | ||
| for plugin file paths. At activation time, the conda package downloads plugins from S3 | ||
| and copies `.json` files to `~/houdini22.0/packages/` for Houdini's native discovery. | ||
|
|
||
| See the [Plugin Sync documentation](https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/plugin-sync.html) | ||
| for more details. | ||
|
|
||
| ## Application-Specific Requirements | ||
|
|
||
| ### Licensing | ||
|
|
||
| See the [AWS Deadline Cloud licensing documentation](https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/license.html) for detailed guidance on license configuration. Houdini requires proper licensing configuration for rendering operations. | ||
|
|
||
| ### Adaptor Compatibility | ||
|
|
||
| Jobs submitted with the Deadline Cloud Houdini submitter use the `houdini-openjd` | ||
| adaptor package. The adaptor's `houdini >=19.5,<22.5` version constraint already | ||
| covers Houdini 22.0, so no separate adaptor change is needed to run 22.0 jobs. | ||
|
|
||
| ### System Requirements | ||
|
|
||
| - Linux x86_64 with GCC 14.2 compatibility | ||
| - Sufficient memory for scene processing | ||
| - Optional: a GPU to render with Karma XPU | ||
|
|
||
| ## Adapting to Other Versions | ||
|
|
||
| ### Version Update Checklist | ||
|
|
||
| To adapt this recipe for Houdini 21.0, 20.5 or 20.0: | ||
|
|
||
| 1. **Update Version Information** | ||
| ```yaml | ||
| # In recipe/recipe.yaml | ||
| context: | ||
| version_partial: "21.0" | ||
| version_minor: "596" | ||
| gcc_version: "gcc11.2" # 22.0 and later use gcc14.2 | ||
|
|
||
| # In deadline-cloud.yaml | ||
| sourceArchiveFilename: houdini-[version]-linux_x86_64_[gcc_version].tar.gz | ||
| ``` | ||
|
|
||
| 2. **Update Source Archives** | ||
| - Download new version archives from SideFX | ||
| - Update SHA256 hashes in `recipe.yaml` | ||
| - Update source filename in `deadline-cloud.yaml` | ||
|
|
||
| 3. **Check Dependencies** | ||
| - Review and update dependency versions | ||
| - Houdini 22.0 needs `libatomic` from the system package manager, which 21.0 did not | ||
|
|
||
| 4. **Update Build Scripts** | ||
| - Check for changes in installation directory structure | ||
| - Update file copy operations in build scripts | ||
| - Verify environment variable paths | ||
|
|
||
| 5. **Update the Plugin Sync scripts** | ||
| - The activate and deactivate scripts hard-code the `houdini22.0` version in | ||
| both the S3 plugin prefix and the `~/houdiniXX.X/packages` directory | ||
|
|
||
| 6. **Test Plugin Compatibility** | ||
| - Verify plugin paths haven't changed | ||
| - Test with existing plugin packages like Redshift | ||
| - Update plugin integration documentation | ||
|
|
||
| ### Common Version Migration Issues | ||
|
|
||
| - **Path Changes**: Installation directories may change between versions | ||
| - **Compiler Version**: The archive filename encodes the GCC version, which changed from `gcc11.2` to `gcc14.2` in Houdini 22.0 | ||
| - **Dependency Updates**: New versions may require different dependencies | ||
| - **Plugin API Changes**: Plugin interfaces may be incompatible between major versions | ||
| - **License Changes**: Licensing requirements may change | ||
|
|
||
| ## Recipe Structure | ||
|
|
||
| ``` | ||
| houdini-22.0/ | ||
| ├── README.md # This file | ||
| ├── deadline-cloud.yaml # Deadline Cloud configuration | ||
| └── recipe/ | ||
| ├── recipe.yaml # Rattler-build recipe | ||
| ├── build.sh # Linux build script | ||
| ├── zzz-houdini-plugin-sync-activate.sh # Plugin Sync activation script | ||
| └── zzz-houdini-plugin-sync-deactivate.sh # Plugin Sync deactivation script | ||
| ``` | ||
|
|
||
| ## Resources | ||
|
|
||
| - **Houdini Documentation**: https://www.sidefx.com/docs/houdini/ | ||
| - **AWS Deadline Cloud Developer Guide**: https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/ | ||
| - **Rattler Build Documentation**: https://prefix-dev.github.io/rattler-build/ | ||
| - **Plugin Development**: https://www.sidefx.com/docs/houdini/ref/plugins.html | ||
| - **Plugin Sync**: https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/plugin-sync.html | ||
|
|
||
| --- | ||
|
|
||
| This recipe is configured for Houdini 22.0.368 on Linux x86_64 platforms. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| condaPlatforms: | ||
| - platform: linux-64 | ||
| defaultSubmit: true | ||
| sourceArchiveFilename: houdini-22.0.368-linux_x86_64_gcc14.2.tar.gz | ||
| sourceDownloadInstructions: Download the Houdini 22.0.368 Linux installer archive (not the launcher) from the SideFX downloads page. | ||
| buildTool: rattler-build | ||
| jobParameters: | ||
| # conda-forge is used to get patchelf on Linux | ||
| - name: CondaChannels | ||
| value: conda-forge |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| #!/bin/bash | ||
| set -xeuo pipefail | ||
|
|
||
| mkdir -p $PREFIX/opt | ||
| cd $PREFIX/opt | ||
|
|
||
|
|
||
| # The Houdini installer expects `bc` to run, but does not fail when | ||
| # it is missing. Ensure that it is installed before running the installer | ||
| bc --help | ||
| # Example messages: | ||
| # houdini.install: line 1516: bc: command not found | ||
| # houdini.install: line 1517: bc: command not found | ||
| # houdini.install: line 1518: bc: command not found | ||
|
|
||
|
|
||
| # Install Houdini | ||
| INSTALLER=$SRC_DIR/installer/houdini.install | ||
| # date of the EULA agreement, not the current date | ||
| EULAdate=2021-10-13 | ||
| $INSTALLER \ | ||
| --auto-install \ | ||
| --accept-EULA $EULAdate \ | ||
| --no-install-engine-maya \ | ||
| --no-install-engine-unity \ | ||
| --no-install-menus \ | ||
| --no-install-bin-symlink \ | ||
| --no-install-hfs-symlink \ | ||
| --no-install-license \ | ||
| --no-install-hqueue-server \ | ||
| --no-root-check \ | ||
| --make-dir $PREFIX/opt/houdini | ||
|
|
||
| HOUDINI_DIR=$PREFIX/opt/houdini | ||
| # The Houdini version without the build number | ||
| HOUDINI_VERSION=${PKG_VERSION%.*} | ||
|
|
||
| # Remove the documentation, it's not needed on the farm | ||
| rm -r $HOUDINI_DIR/houdini/help | ||
|
|
||
| # Create symlinks | ||
| mkdir -p $PREFIX/bin | ||
| for BINARY in houdini houdini-bin houdinicore houdinifx \ | ||
| hscript husk hython hbatch karma karma_cc mantra mantra-bin \ | ||
| vmantra vmantra-bin; do | ||
| ln -r -s $HOUDINI_DIR/bin/$BINARY $PREFIX/bin/$BINARY | ||
| done | ||
|
|
||
| # Install Houdini dependencies from local package manager | ||
| mkdir -p $SRC_DIR/download | ||
| cd $SRC_DIR/download | ||
| dnf download --resolve -y alsa-lib fontconfig libXScrnSaver libxkbfile libatomic | ||
|
|
||
| for rpm_file in $(realpath $SRC_DIR/download/*.rpm); do | ||
| rpm2cpio "$rpm_file" | cpio -idm | ||
| done | ||
|
|
||
| # Copy .so's to Houdini installation | ||
| for so_file in $(find . -iname "*.so.*"); do | ||
| patchelf --add-rpath '$ORIGIN' $so_file | ||
| cp $so_file $HOUDINI_DIR/dsolib/. | ||
| done | ||
|
|
||
| # Script to set environment variables during activation | ||
| mkdir -p $PREFIX/etc/conda/activate.d | ||
| cat <<EOF > $PREFIX/etc/conda/activate.d/houdini-$PKG_VERSION-vars.sh | ||
| export "HOUDINI_LOCATION=\$CONDA_PREFIX/opt/houdini" | ||
| export "HOUDINI_VERSION=$HOUDINI_VERSION" | ||
| export "HOUDINI_BINARY_PATH=\$HOUDINI_LOCATION/bin" | ||
| export "HOUDINI_HOUDINI_PATH=\$HOUDINI_LOCATION/houdini" | ||
| export "HOUDINI_INCLUDE_PATH=\$HOUDINI_LOCATION/toolkit/include" | ||
| export "HOUDINI_LIBRARY_PATH=\$HOUDINI_LOCATION/dsolib" | ||
| export "HOUDINI_DONT_PURGE_SEARCH_PATH_CACHE_AFTER_STARTUP=true" | ||
| export "HOUDINI_DONT_PURGE_INDEX_FILE_CACHE_AFTER_STARTUP=true" | ||
| EOF | ||
|
|
||
| mkdir -p $PREFIX/etc/conda/deactivate.d | ||
| cat <<EOF > $PREFIX/etc/conda/deactivate.d/houdini-$PKG_VERSION-vars.sh | ||
| unset HOUDINI_LIBRARY_PATH | ||
| unset HOUDINI_INCLUDE_PATH | ||
| unset HOUDINI_HOUDINI_PATH | ||
| unset HOUDINI_BINARY_PATH | ||
| unset HOUDINI_VERSION | ||
| unset HOUDINI_LOCATION | ||
| unset HOUDINI_DONT_PURGE_SEARCH_PATH_CACHE_AFTER_STARTUP | ||
| unset HOUDINI_DONT_PURGE_INDEX_FILE_CACHE_AFTER_STARTUP | ||
| EOF | ||
|
|
||
| # Install Simple Plugin Sync scripts | ||
| cp $RECIPE_DIR/zzz-houdini-plugin-sync-activate.sh $PREFIX/etc/conda/activate.d/ | ||
| cp $RECIPE_DIR/zzz-houdini-plugin-sync-deactivate.sh $PREFIX/etc/conda/deactivate.d/ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # Recipe in rattler-build format. | ||
| # See https://prefix-dev.github.io/rattler-build/latest/ | ||
|
|
||
| context: | ||
| version_partial: "22.0" | ||
| version_minor: "368" | ||
| gcc_version: "gcc14.2" | ||
| version: ${{ version_partial }}.${{ version_minor }} | ||
|
|
||
| package: | ||
| name: houdini | ||
| version: ${{ version }} | ||
|
|
||
| source: | ||
| - if: linux | ||
| then: | ||
| url: file://archive_files/houdini-${{ version }}-linux_x86_64_${{ gcc_version }}.tar.gz | ||
| sha256: 8765335f090a8329768b415b64bc9fb80a0d9963b13f63455ad042e32d353616 | ||
| target_directory: installer | ||
|
|
||
| build: | ||
| number: 0 | ||
| script: | ||
| - if: unix | ||
| then: | ||
| - bash $RECIPE_DIR/build.sh | ||
| prefix_detection: | ||
| ignore_binary_files: true | ||
| dynamic_linking: | ||
| binary_relocation: false | ||
| missing_dso_allowlist: | ||
| - "**" | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The tests:
- script:
- houdini -h
- hython -hWorth adding (or narrowing the allowlist to the DSOs that are genuinely expected to be resolved at runtime rather than |
||
|
|
||
| requirements: | ||
| build: | ||
| - patchelf | ||
|
kenyrish-amazon marked this conversation as resolved.
|
||
|
|
||
| about: | ||
| homepage: https://www.sidefx.com/products/houdini/ | ||
| license: LicenseRef-SideFXEULA | ||
| summary: > | ||
| Houdini is built from the ground up to be a procedural system that empowers artists to work freely, | ||
| create multiple iterations and rapidly share workflows with colleagues. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new recipe is not added to the recipe index in
conda_recipes/README.md. That table lists Houdini 20.5, 21.0, and the Redshift/V-Ray plugin recipes, and the repository checklist requires updating the nearest category table when a sample is added. Suggested row, next to the 21.0 entry:Without it the sample is undiscoverable from the collection README.