Record recommended tech stack for projects by euronion · Pull Request #269 · open-energy-transition/handbook

euronion · 2025-08-12T15:18:21Z

The OS coords group had the idea of documenting and harmonising our tech stack that we recommend for our projects, in order to reduce the number of tools we use.

Some ideas written down to start the discussion, list is not complete ofc.

Do we already have something similar?
What do you think about this initative?

Changes Proposed in This Pull Request

Checklist

I have checked the Deploy Preview link in the netlify bot's comment, and my changes look good
I tested my contribution locally, and it seems to work fine.
Code and workflow changes are sufficiently documented.
If new pages are added, maintainers are assigned for that document.
If the document falls under a controlled category, a controlled document banner is present.

netlify · 2025-08-12T15:18:52Z

✅ Deploy Preview for wonderful-mousse-057100 ready!

Name	Link
🔨 Latest commit	`b525f8a`
🔍 Latest deploy log	https://app.netlify.com/projects/wonderful-mousse-057100/deploys/6914bdf57d737b0008d81f8f
😎 Deploy Preview	https://deploy-preview-269--wonderful-mousse-057100.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

siddharth-krishna

Great idea, thanks! I started to collect some similar recommendations, e.g. repository settings, pre-commit hooks, in this presentation -- perhaps we can merge the recommendations here and launch it and ask everyone to add/update/comment?

…n/handbook into tech-stack

euronion · 2025-08-20T10:10:13Z

Great idea, thanks! I started to collect some similar recommendations, e.g. repository settings, pre-commit hooks, in this presentation -- perhaps we can merge the recommendations here and launch it and ask everyone to add/update/comment?

Good idea, I added some of the ideas from the slides!

I think it would be nice to also add examples for each point (and maybe in the future a template)?

brynpickering

Some things possibly missing

use in pre-commit: codespell, markdown linter, snakefmt, yaml linters, markdown link checkers
add a contributing guideline, a way to list contributors, an .editorconfig
I've found a CODEOWNERS file to be quite useful as you can get automatic review requests that differ by where in the repo changes have been made and you can enforce at least one codeowner to approve before merging PRs
something about yaml and data schema when working with configs and data?

brynpickering · 2025-11-17T14:21:16Z

+    * No force pushes
+    * Use pull requests for all changes
+    * Merge branches instead of rebasing
+    * Squash and merge pull requests to keep the commit history clean


I think this is good for python library projects, but in other project types (e.g. snakemake workflows)the full commit history can be quite instructive / can be used to resurrect superseded attempts at solving an issue when it later turns out that it was the correct approach.

The full PR branch history is preserved by GitHub, and I think this is the better place to archive this information, because it also has the context of why each change was necessary (review comments, CI failures, etc).

I would be okay with merge-merging PRs if everyone had iron discipline and kept their PR branches full of atomic, clean commits with descriptive messages, but too often we have (and it's much easier to have) commits like "fix, fix fix, typo, revert fix fix, ..."

It's been on my list for a while to make a handbook page/section on why I recommend squash-merging all PRs, but some other reasons include: easier to roll-back/revert a breaking change in production, easier to git bisect and find the source of a bug, and every commit on the main branch is a "valid" state of the project that works (or at least, CI passes).

The full PR branch history is preserved by GitHub

Is that the case even once a branch is deleted? I tend to keep repos clean by deleting branches on merging PRs but I assume at some point you can't roll back that deletion and the branch history is lost forever?

Yep!
Here's one of the first PRs on the benchmark repo: open-energy-transition/solver-benchmark#9
You can go through and view all the commits still, e.g.: open-energy-transition/solver-benchmark@f56d9ee

We should also add a recommendation to auto-delete PR branches on merge. :)

brynpickering · 2025-11-17T14:24:47Z

+    * No force pushes
+    * Use pull requests for all changes


Suggested change

* No force pushes

* Use pull requests for all changes

* Add [main branch protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/managing-a-branch-protection-rule) to include:

* no force pushes;

* at least one `approve` review before merge;

* override rights for project admins only;

* no pushes directly to `main` allowed (i.e. use pull requests for all changes).

brynpickering · 2025-11-17T14:25:31Z

+    * Squash and merge pull requests to keep the commit history clean
+* Continuous integration:
+    * For public repositories / open source projects: `github actions`
+    * For private repositories / internal projects: ?


If they are lightweight, github actions could still be possible (what's our total private repo allowance?)

We pay for GitHub actions per minute on private repos. It's still probably the recommended way, although I'd add that for forks such as pypsa-x it is a good idea to disable win/macOS actions since they cost 4-8x as much.

brynpickering · 2025-11-17T14:26:09Z

+    * `uv` for new projects
+    * always use virtual environments (better reproducibility, isolation, and dependency management)
+    * `uv init` to create a new project with default structure
+    * `uv add <package>` to add a package to the project
+    * `uv pip install <package>` to install a package in the virtual environment without adding it as a dependency
+    * `uv run <command>` to run a command in the virtual environment


pixi...? :P

brynpickering · 2025-11-17T14:27:30Z

+* Documentation:
+    * doc strings follow [numpydoc style](https://numpydoc.readthedocs.io/en/latest/format.html)
+    * Framework: `mkdocs-material`
+    * Hosting: `readthedocs`


Github pages can be a better option when there's the possibility of name clashes with existing RTD projects or when in a private repo. Not sure how best to recommend one or the other.

brynpickering · 2025-11-17T14:28:06Z

+    * doc strings follow [numpydoc style](https://numpydoc.readthedocs.io/en/latest/format.html)
+    * Framework: `mkdocs-material`
+    * Hosting: `readthedocs`
+* Repositories should use pre-commit framework: `pre-commit`


I would consider this to be part of continuous integration

brynpickering · 2025-11-17T14:28:45Z

+    * Public repositories: use `pre-commit.ci` for automatic checks (instead of integrating `pre-commit` into CI)
+    * Private repositories: use `pre-commit` in CI
+* Code style:
+    * `black`


maybe don't even mention black since ruff handles everything?

brynpickering · 2025-11-17T14:30:05Z

+    * doc strings follow [numpydoc style](https://numpydoc.readthedocs.io/en/latest/format.html)
+    * Framework: `mkdocs-material`
+    * Hosting: `readthedocs`
+* Repositories should use pre-commit framework: `pre-commit`


Link to your favourite implementation of .pre-commit-config.yaml so others can copy that across to their project if they like

siddharth-krishna

Thanks, Bryn for the review and suggestions! Johannes, whenever you think this is ready I think we can merge it in and iterate in successive PRs. 🚀

siddharth-krishna · 2025-11-24T13:12:00Z

+    * No force pushes
+    * Use pull requests for all changes
+    * Merge branches instead of rebasing
+    * Squash and merge pull requests to keep the commit history clean


The full PR branch history is preserved by GitHub, and I think this is the better place to archive this information, because it also has the context of why each change was necessary (review comments, CI failures, etc).

I would be okay with merge-merging PRs if everyone had iron discipline and kept their PR branches full of atomic, clean commits with descriptive messages, but too often we have (and it's much easier to have) commits like "fix, fix fix, typo, revert fix fix, ..."

It's been on my list for a while to make a handbook page/section on why I recommend squash-merging all PRs, but some other reasons include: easier to roll-back/revert a breaking change in production, easier to git bisect and find the source of a bug, and every commit on the main branch is a "valid" state of the project that works (or at least, CI passes).

siddharth-krishna · 2025-11-24T13:13:09Z

+    * Squash and merge pull requests to keep the commit history clean
+* Continuous integration:
+    * For public repositories / open source projects: `github actions`
+    * For private repositories / internal projects: ?


We pay for GitHub actions per minute on private repos. It's still probably the recommended way, although I'd add that for forks such as pypsa-x it is a good idea to disable win/macOS actions since they cost 4-8x as much.

siddharth-krishna · 2025-11-24T13:13:53Z

+* Continuous integration:
+    * For public repositories / open source projects: `github actions`
+    * For private repositories / internal projects: ?
+    * Use a pinned environment for the CI to ensure continuity and reproducibility


@brynpickering or @SermishaNarayana or whoever knows the latest/best way to do this could you please add a section below with instructions and link to it here? Huge thanks

Using a pinned environment? If we recommend using uv or pixi that would come as standard without the user really having to do anything.

As requested by Sid [here](open-energy-transition/oet-zib-manual#12 (comment)). Supersedes some content in #269 I haven't filled in the `uv` config recommendation section as I don't use it so don't know what would be recommended. I would be inclined to recommend `pixi` for all but pure Python packages, but realise that several use `uv` already and so didn't want to push for it too hard. The reason being that those working on Linux often miss the fact that they are utilising non-python libs that others will need. `pixi` allows those dependencies to be explicitly referenced in the project dependencies for better cross-platform compatibility. ## Changes Proposed in This Pull Request ## Checklist - [ ] I have checked the Deploy Preview link in the netlify bot's comment, and my changes look good - [ ] I tested my contribution locally, and it seems to work fine. - [ ] Code and workflow changes are sufficiently documented. - [ ] If new pages are added, maintainers are assigned for that document. - [ ] If the document falls under a controlled category, a controlled document banner is present. --------- Co-authored-by: Siddharth Krishna <siddharth-krishna@users.noreply.github.com>

brynpickering · 2026-06-02T12:27:24Z

@siddharth-krishna @euronion is this now stale or could we merge with minimal updates?

siddharth-krishna · 2026-06-05T14:10:25Z

I think this PR still has some really useful stuff! But it needs updating before merge:

Some of Bryn's comments are useful and should be actioned/included
Meanwhile we published https://open-energy-transition.github.io/handbook/docs/Engineering/CodingGuidelines -- this should align with that and refer to that instead of repeating things like how to work with PRs, etc
We also have https://open-energy-transition.github.io/handbook/docs/Engineering/DependencyManagement -- so again this page can just refer to that instead of going into uv, etc

Would anyone have the bandwidth to do the above? 🙏

Initial idea

8c594d3

siddharth-krishna reviewed Aug 13, 2025

View reviewed changes

tgilon reviewed Aug 13, 2025

View reviewed changes

Comment thread docs/Engineering/TechStack.mdx Outdated

finozzifa and others added 4 commits August 18, 2025 16:00

Merge branch 'main' into tech-stack

38917a0

typo

a674d66

Merge branch 'tech-stack' of https://github.com/open-energy-transitio…

84fe146

…n/handbook into tech-stack

doc: Integrate internal recommendations from slides

8134a01

euronion requested a review from brynpickering November 4, 2025 13:58

Merge branch 'main' into tech-stack

b525f8a

brynpickering reviewed Nov 17, 2025

View reviewed changes

siddharth-krishna approved these changes Nov 24, 2025

View reviewed changes

brynpickering mentioned this pull request Nov 24, 2025

Add dependency management page #298

Merged

5 tasks

-    * No force pushes
-    * Use pull requests for all changes
+    * Add [main branch protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/managing-a-branch-protection-rule) to include:
+      * no force pushes;
+      * at least one `approve` review before merge;
+      * override rights for project admins only;
+      * no pushes directly to `main` allowed (i.e. use pull requests for all changes).

Uh oh!

Conversation

euronion commented Aug 12, 2025

Changes Proposed in This Pull Request

Checklist

Uh oh!

netlify Bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for wonderful-mousse-057100 ready!

Uh oh!

siddharth-krishna left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

euronion commented Aug 20, 2025

Uh oh!

brynpickering left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

siddharth-krishna left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brynpickering commented Jun 2, 2026

Uh oh!

siddharth-krishna commented Jun 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

netlify Bot commented Aug 12, 2025 •

edited

Loading