Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 202 additions & 0 deletions conda_recipes/houdini-22.0/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,202 @@
# Houdini 22.0 Conda Recipe for AWS Deadline Cloud

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new recipe is not added to the recipe index in conda_recipes/README.md. That table lists Houdini 20.5, 21.0, and the Redshift/V-Ray plugin recipes, and the repository checklist requires updating the nearest category table when a sample is added. Suggested row, next to the 21.0 entry:

| [Houdini 22.0](houdini-22.0/) | Packaging Houdini 22.0 with Plugin Sync activation | Your jobs require Houdini 22 or frequently updated plugins |

Without it the sample is undiscoverable from the collection README.


## Overview

This directory contains a conda build recipe for Houdini 22.0.368, configured for use with AWS Deadline Cloud. The package runs Houdini rendering and processing jobs on Deadline Cloud service-managed fleets.

## Package Information

- **Application**: Houdini 22.0.368
- **Supported Platforms**: linux-64
- **Source**: SideFX Houdini downloads page
- **License**: SideFXEULA
- **Build Tool**: rattler-build

## Prerequisites

Before building this package, ensure you have:

1. **AWS Deadline Cloud infrastructure** set up with:
- A farm configured for package building. See https://github.com/aws-deadline/deadline-cloud-samples/tree/mainline/cloudformation/farm_templates/starter_farm for instructions to create a Farm.
- A queue for building packages. The submit command looks for a queue whose name starts with "Package". See https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/configure-jobs-s3-channel.html for instructions on creating one.
- Linux-64 fleet for building linux packages

2. **Deadline Cloud CLI** installed on your workstation

3. **SideFX account** for downloading Houdini installer

4. **Source archive** (see [Archive File Instructions](#archive-file-instructions) below)

## Archive File Instructions

### Linux

#### Download from SideFX
1. Download the `houdini-22.0.368-linux_x86_64_gcc14.2.tar.gz` from SideFX Houdini's downloads page
2. Clone this repository locally.
3. Place the downloaded file in the `conda_recipes/archive_files` directory

Note that Houdini 22.0 builds are compiled with GCC 14.2, whereas Houdini 21.0 and
earlier used GCC 11.2. Make sure you download the `gcc14.2` archive, or the build
will fail to find the installer.

## Plugin Integration

### Plugins

Houdini supports plugins through the use of package files. A package is a json file that tells Houdini where to find plugins.
[Houdini Plugin Reference](https://www.sidefx.com/docs/houdini/ref/plugins.html).

Create your package files in `$PREFIX/opt/houdini/packages` and point them to the location of your plugins. See our Redshift for
Houdini recipe as [an example](../houdini-redshift-2026/).

### Plugin Installation Paths

The conda recipe configures the following environment variables and paths for plugin discovery:

```bash
# Environment variables set by this package
export HOUDINI_LOCATION=$PREFIX/opt/houdini

# Plugin search paths (in order of precedence)
$PREFIX/opt/houdini/packages
```

### Creating Plugin Packages

1. **Plugin Package Structure**
```
my-houdini-plugin/
├── recipe/
│ ├── recipe.yaml
│ ├── build.sh
│ └── bld.bat
├── deadline-cloud.yaml
└── README.md
```

2. **Plugin Installation Script Example**

NOTE: Your plugin files can be anywhere as long as your package file points to their directory.

```bash
# In build.sh
mkdir -p $PREFIX/opt/houdini/packages
cp my-plugin.json $PREFIX/opt/houdini/packages/
cp -r plugin-files/ $PREFIX/opt/houdini/plugin/
```

3. **Plugin Dependencies**
- Add this Houdini package as a dependency in your plugin's `recipe.yaml`
- Specify version constraints: `houdini >=22.0,<22.5`

### Plugin Sync

This recipe includes Plugin Sync support, which allows customers to deliver plugins
to workers via S3 without building a separate conda package.

To use Plugin Sync, upload your plugin files and a Houdini package descriptor
(`.json` file) to the S3 path:

```
s3://<job-attachments-bucket>/<root-prefix>/plugins/linux/houdini/22.0/
```

The `.json` package descriptor should reference `$DEADLINE_CLOUD_HOUDINI_PLUGIN_SYNC_DIR`
for plugin file paths. At activation time, the conda package downloads plugins from S3
and copies `.json` files to `~/houdini22.0/packages/` for Houdini's native discovery.

See the [Plugin Sync documentation](https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/plugin-sync.html)
for more details.

## Application-Specific Requirements

### Licensing

See the [AWS Deadline Cloud licensing documentation](https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/license.html) for detailed guidance on license configuration. Houdini requires proper licensing configuration for rendering operations.

### Adaptor Compatibility

Jobs submitted with the Deadline Cloud Houdini submitter use the `houdini-openjd`
adaptor package. The adaptor's `houdini >=19.5,<22.5` version constraint already
covers Houdini 22.0, so no separate adaptor change is needed to run 22.0 jobs.

### System Requirements

- Linux x86_64 with GCC 14.2 compatibility
- Sufficient memory for scene processing
- Optional: a GPU to render with Karma XPU

## Adapting to Other Versions

### Version Update Checklist

To adapt this recipe for Houdini 21.0, 20.5 or 20.0:

1. **Update Version Information**
```yaml
# In recipe/recipe.yaml
context:
version_partial: "21.0"
version_minor: "596"
gcc_version: "gcc11.2" # 22.0 and later use gcc14.2

# In deadline-cloud.yaml
sourceArchiveFilename: houdini-[version]-linux_x86_64_[gcc_version].tar.gz
```

2. **Update Source Archives**
- Download new version archives from SideFX
- Update SHA256 hashes in `recipe.yaml`
- Update source filename in `deadline-cloud.yaml`

3. **Check Dependencies**
- Review and update dependency versions
- Houdini 22.0 needs `libatomic` from the system package manager, which 21.0 did not

4. **Update Build Scripts**
- Check for changes in installation directory structure
- Update file copy operations in build scripts
- Verify environment variable paths

5. **Update the Plugin Sync scripts**
- The activate and deactivate scripts hard-code the `houdini22.0` version in
both the S3 plugin prefix and the `~/houdiniXX.X/packages` directory

6. **Test Plugin Compatibility**
- Verify plugin paths haven't changed
- Test with existing plugin packages like Redshift
- Update plugin integration documentation

### Common Version Migration Issues

- **Path Changes**: Installation directories may change between versions
- **Compiler Version**: The archive filename encodes the GCC version, which changed from `gcc11.2` to `gcc14.2` in Houdini 22.0
- **Dependency Updates**: New versions may require different dependencies
- **Plugin API Changes**: Plugin interfaces may be incompatible between major versions
- **License Changes**: Licensing requirements may change

## Recipe Structure

```
houdini-22.0/
├── README.md # This file
├── deadline-cloud.yaml # Deadline Cloud configuration
└── recipe/
├── recipe.yaml # Rattler-build recipe
├── build.sh # Linux build script
├── zzz-houdini-plugin-sync-activate.sh # Plugin Sync activation script
└── zzz-houdini-plugin-sync-deactivate.sh # Plugin Sync deactivation script
```

## Resources

- **Houdini Documentation**: https://www.sidefx.com/docs/houdini/
- **AWS Deadline Cloud Developer Guide**: https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/
- **Rattler Build Documentation**: https://prefix-dev.github.io/rattler-build/
- **Plugin Development**: https://www.sidefx.com/docs/houdini/ref/plugins.html
- **Plugin Sync**: https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/plugin-sync.html

---

This recipe is configured for Houdini 22.0.368 on Linux x86_64 platforms.
10 changes: 10 additions & 0 deletions conda_recipes/houdini-22.0/deadline-cloud.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
condaPlatforms:
- platform: linux-64
defaultSubmit: true
sourceArchiveFilename: houdini-22.0.368-linux_x86_64_gcc14.2.tar.gz
sourceDownloadInstructions: Download the Houdini 22.0.368 Linux installer archive (not the launcher) from the SideFX downloads page.
buildTool: rattler-build
jobParameters:
# conda-forge is used to get patchelf on Linux
- name: CondaChannels
value: conda-forge
91 changes: 91 additions & 0 deletions conda_recipes/houdini-22.0/recipe/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#!/bin/bash
set -xeuo pipefail

mkdir -p $PREFIX/opt
cd $PREFIX/opt


# The Houdini installer expects `bc` to run, but does not fail when
# it is missing. Ensure that it is installed before running the installer
bc --help
# Example messages:
# houdini.install: line 1516: bc: command not found
# houdini.install: line 1517: bc: command not found
# houdini.install: line 1518: bc: command not found


# Install Houdini
INSTALLER=$SRC_DIR/installer/houdini.install
# date of the EULA agreement, not the current date
EULAdate=2021-10-13
$INSTALLER \
--auto-install \
--accept-EULA $EULAdate \
--no-install-engine-maya \
--no-install-engine-unity \
--no-install-menus \
--no-install-bin-symlink \
--no-install-hfs-symlink \
--no-install-license \
--no-install-hqueue-server \
--no-root-check \
--make-dir $PREFIX/opt/houdini

HOUDINI_DIR=$PREFIX/opt/houdini
# The Houdini version without the build number
HOUDINI_VERSION=${PKG_VERSION%.*}

# Remove the documentation, it's not needed on the farm
rm -r $HOUDINI_DIR/houdini/help

# Create symlinks
mkdir -p $PREFIX/bin
for BINARY in houdini houdini-bin houdinicore houdinifx \
hscript husk hython hbatch karma karma_cc mantra mantra-bin \
vmantra vmantra-bin; do
ln -r -s $HOUDINI_DIR/bin/$BINARY $PREFIX/bin/$BINARY
done

# Install Houdini dependencies from local package manager
mkdir -p $SRC_DIR/download
cd $SRC_DIR/download
dnf download --resolve -y alsa-lib fontconfig libXScrnSaver libxkbfile libatomic

for rpm_file in $(realpath $SRC_DIR/download/*.rpm); do
rpm2cpio "$rpm_file" | cpio -idm
done

# Copy .so's to Houdini installation
for so_file in $(find . -iname "*.so.*"); do
patchelf --add-rpath '$ORIGIN' $so_file
cp $so_file $HOUDINI_DIR/dsolib/.
done

# Script to set environment variables during activation
mkdir -p $PREFIX/etc/conda/activate.d
cat <<EOF > $PREFIX/etc/conda/activate.d/houdini-$PKG_VERSION-vars.sh
export "HOUDINI_LOCATION=\$CONDA_PREFIX/opt/houdini"
export "HOUDINI_VERSION=$HOUDINI_VERSION"
export "HOUDINI_BINARY_PATH=\$HOUDINI_LOCATION/bin"
export "HOUDINI_HOUDINI_PATH=\$HOUDINI_LOCATION/houdini"
export "HOUDINI_INCLUDE_PATH=\$HOUDINI_LOCATION/toolkit/include"
export "HOUDINI_LIBRARY_PATH=\$HOUDINI_LOCATION/dsolib"
export "HOUDINI_DONT_PURGE_SEARCH_PATH_CACHE_AFTER_STARTUP=true"
export "HOUDINI_DONT_PURGE_INDEX_FILE_CACHE_AFTER_STARTUP=true"
EOF

mkdir -p $PREFIX/etc/conda/deactivate.d
cat <<EOF > $PREFIX/etc/conda/deactivate.d/houdini-$PKG_VERSION-vars.sh
unset HOUDINI_LIBRARY_PATH
unset HOUDINI_INCLUDE_PATH
unset HOUDINI_HOUDINI_PATH
unset HOUDINI_BINARY_PATH
unset HOUDINI_VERSION
unset HOUDINI_LOCATION
unset HOUDINI_DONT_PURGE_SEARCH_PATH_CACHE_AFTER_STARTUP
unset HOUDINI_DONT_PURGE_INDEX_FILE_CACHE_AFTER_STARTUP
EOF

# Install Simple Plugin Sync scripts
cp $RECIPE_DIR/zzz-houdini-plugin-sync-activate.sh $PREFIX/etc/conda/activate.d/
cp $RECIPE_DIR/zzz-houdini-plugin-sync-deactivate.sh $PREFIX/etc/conda/deactivate.d/
43 changes: 43 additions & 0 deletions conda_recipes/houdini-22.0/recipe/recipe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Recipe in rattler-build format.
# See https://prefix-dev.github.io/rattler-build/latest/

context:
version_partial: "22.0"
version_minor: "368"
gcc_version: "gcc14.2"
version: ${{ version_partial }}.${{ version_minor }}

package:
name: houdini
version: ${{ version }}

source:
- if: linux
then:
url: file://archive_files/houdini-${{ version }}-linux_x86_64_${{ gcc_version }}.tar.gz
sha256: 8765335f090a8329768b415b64bc9fb80a0d9963b13f63455ad042e32d353616
target_directory: installer

build:
number: 0
script:
- if: unix
then:
- bash $RECIPE_DIR/build.sh
prefix_detection:
ignore_binary_files: true
dynamic_linking:
binary_relocation: false
missing_dso_allowlist:
- "**"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing_dso_allowlist: ["**"] disables every linkage check, and there is no tests: section, so a package with unresolvable shared libraries will build and publish cleanly and only fail on the farm. That risk is concrete here: 22.0 needed libatomic added to the dnf download list precisely because of a missing DSO.

The houdini-20.5 recipe guards against this with a smoke test that would have caught it:

tests:
  - script:
      - houdini -h
      - hython -h

Worth adding (or narrowing the allowlist to the DSOs that are genuinely expected to be resolved at runtime rather than **).


requirements:
build:
- patchelf
Comment thread
kenyrish-amazon marked this conversation as resolved.

about:
homepage: https://www.sidefx.com/products/houdini/
license: LicenseRef-SideFXEULA
summary: >
Houdini is built from the ground up to be a procedural system that empowers artists to work freely,
create multiple iterations and rapidly share workflows with colleagues.
Loading
Loading