Feature request: “editable / local source tree” workflow for ConfigureMake-based easyconfigs (dev builds without pushing to remote git)

[KUL-EES]

Summary

Researchers and developers often want to use EasyBuild to build the software they are actively developing (from a local working tree) and expose it as a module. Today, doing so typically requires pushing commits to a remote repository and then building from that remote source, which is excellent for reproducibility but cumbersome during active development.

I’d like to propose an “editable/local source tree” workflow for non-Python recipes (e.g., ConfigureMake, CMakeMake, etc.), comparable in spirit to the existing --editable behavior for PythonPackage/PythonBundle.

Current workflow and pain points

What happens today:

1. Developer makes local changes in a working tree.
2. To use EasyBuild they must:

• commit and push changes to a remote git repo,
• write/update an easyconfig with the correct commit hash,
• run EasyBuild, which then fetches from the remote repository.

Why this is painful for dev loops:

• It adds friction (“bureaucracy”) for every incremental change.
• As a result, developers often fall back to manual builds and ad-hoc modulefiles or load_modules scripts.

Why manual builds are problematic:

• The developer has to track provenance themselves: exact version/commit, toolchain, dependency versions, micro-architecture, build flags, etc.
• This bookkeeping is error-prone and requires ongoing discipline.
• The outcome is often less reproducible and harder to share with colleagues/users than a proper EasyBuild-built module.

Proposed feature: local “dev build” / editable source mode

Add an officially supported mode to build a package from the local working directory (or a specified local path), while still using EasyBuild for:

• dependency resolution/toolchain setup,
• consistent build environments,
• standardized install layout,
• module generation and metadata.

This could be exposed as either:

• a CLI option (e.g., --editable / --local-source / --from-workdir), and/or
• an easyconfig-level option (e.g., local_source = True), and/or
• a version convention (e.g., version == "dev"), though a dedicated toggle would likely be clearer/safer.

Expected behavior (minimum viable)

In “local dev build” mode:

1. Use the local working tree as the source instead of downloading/unpacking sources.
2. Build as usual (configure + make, etc.) under the EasyBuild-provided environment.
3. Install as usual into installdir.
4. Ensure module naming/versioning remains informative:

• optionally encode the current git hash/tag into versionsuffix (or similar).

Important: This mode is primarily for development convenience and should be clearly marked as less reproducible unless the local source state is captured.

Optional extension: “live” / fully editable install

Optionally, support a more “live” workflow:

• After installation, replace installed binaries (or selected artifacts) in installdir with symlinks back into the working tree, so developers can iterate with make in-place without re-running EasyBuild each time.
• In this mode, you may not want to set a versionsuffix to a commit hash, since the working tree can move.

This would be a separate, explicit option (because it further reduces reproducibility and could surprise users if enabled implicitly).

Proof of concept (hooks-based)

We implemented a proof-of-concept using EasyBuild hooks and have been using it successfully in a few internal projects.

POC hook file:
hooks.py

The POC approach:

• triggers “editable” behavior when version == "dev",
• optionally sets versionsuffix from git hash/tag,
• swaps builddir for a symlink to the working directory EasyBuild was launched from (optionally via fuse-overlayfs to keep the source tree clean),
• runs the standard build steps and installs into installdir,
• optionally symlinks installed artifacts back into the source tree for a “live” workflow.

We recognize the POC is somewhat crude, but the intent here is mainly to demonstrate feasibility and help discuss what a proper upstream design could look like.

Why this belongs in EasyBuild (in our view)

This feature would keep developers “inside” EasyBuild for dev iterations, improving:

• consistency of environments and dependency resolution,
• traceability and sharing (module-based),
• reduction of ad-hoc manual build setups that are hard to reproduce.