-
Notifications
You must be signed in to change notification settings - Fork 11
Record recommended tech stack for projects #269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
8c594d3
38917a0
a674d66
84fe146
8134a01
b525f8a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,86 @@ | ||
| --- | ||
| sidebar_label: "Technology stack" | ||
| --- | ||
|
|
||
| import MaintainerTag from "@site/src/components/MaintainerTag"; | ||
| import ControlledDocBanner from "@site/src/components/ControlledDocBanner"; | ||
|
|
||
|
|
||
| <MaintainerTag maintainerEmails={["siddharth.krishna@openenergytransition.org","johannes.hampp@openenergytransition.org"]} /> | ||
|
|
||
| # Technology stack that we use | ||
|
|
||
| Recommended, not enforced. | ||
| Projects may deviate, especially in case of projects based on soft forks. | ||
|
|
||
|
|
||
| * Version control: | ||
| * `git` | ||
| * Host repositories on: `github` | ||
| * Repositories should belong to [Open Energy Transition GitHub organization](https://github.com/open-energy-transition) | ||
| * No force pushes | ||
| * Use pull requests for all changes | ||
| * Merge branches instead of rebasing | ||
| * Squash and merge pull requests to keep the commit history clean | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is good for python library projects, but in other project types (e.g. snakemake workflows)the full commit history can be quite instructive / can be used to resurrect superseded attempts at solving an issue when it later turns out that it was the correct approach.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The full PR branch history is preserved by GitHub, and I think this is the better place to archive this information, because it also has the context of why each change was necessary (review comments, CI failures, etc). I would be okay with merge-merging PRs if everyone had iron discipline and kept their PR branches full of atomic, clean commits with descriptive messages, but too often we have (and it's much easier to have) commits like "fix, fix fix, typo, revert fix fix, ..." It's been on my list for a while to make a handbook page/section on why I recommend squash-merging all PRs, but some other reasons include: easier to roll-back/revert a breaking change in production, easier to
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Is that the case even once a branch is deleted? I tend to keep repos clean by deleting branches on merging PRs but I assume at some point you can't roll back that deletion and the branch history is lost forever?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep! We should also add a recommendation to auto-delete PR branches on merge. :) |
||
| * Continuous integration: | ||
| * For public repositories / open source projects: `github actions` | ||
| * For private repositories / internal projects: ? | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If they are lightweight, github actions could still be possible (what's our total private repo allowance?)
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We pay for GitHub actions per minute on private repos. It's still probably the recommended way, although I'd add that for forks such as pypsa-x it is a good idea to disable win/macOS actions since they cost 4-8x as much. |
||
| * Use a pinned environment for the CI to ensure continuity and reproducibility | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @brynpickering or @SermishaNarayana or whoever knows the latest/best way to do this could you please add a section below with instructions and link to it here? Huge thanks
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Using a pinned environment? If we recommend using |
||
| * Programming languages: | ||
| * `python` | ||
| * whereever possible, including for static analysis, plotting, etc. | ||
| * `pandas`, `numpy`, `xarray` with `matplotlib`/`seaborn`/`hvplot` | ||
| * Package management: | ||
| * `uv` for new projects | ||
| * always use virtual environments (better reproducibility, isolation, and dependency management) | ||
| * `uv init` to create a new project with default structure | ||
| * `uv add <package>` to add a package to the project | ||
| * `uv pip install <package>` to install a package in the virtual environment without adding it as a dependency | ||
| * `uv run <command>` to run a command in the virtual environment | ||
|
Comment on lines
+34
to
+39
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
| * `source .venv/bin/activate` to activate the virtual environment (default location) | ||
| * `uv sync` to install all dependencies from `pyproject.toml` or `uv.lock` | ||
| * Share your environment with others using `pyproject.toml` and `uv.lock` | ||
| * some projects use `conda` because of specific dependencies, where installing the packages via `pip` requires additional steps to install the required binaries separately. | ||
| Installations with `conda` using `conda-forge` on the other hand proide the binaries automatically. | ||
| Can be especially important for projects that work with geospatial packages or on windows. | ||
| * Documentation: | ||
| * doc strings follow [numpydoc style](https://numpydoc.readthedocs.io/en/latest/format.html) | ||
| * Framework: `mkdocs-material` | ||
| * Hosting: `readthedocs` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Github pages can be a better option when there's the possibility of name clashes with existing RTD projects or when in a private repo. Not sure how best to recommend one or the other. |
||
| * Repositories should use pre-commit framework: `pre-commit` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would consider this to be part of continuous integration
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Link to your favourite implementation of |
||
| * To ensure code styles and checks are run consistently on PRs / commits | ||
| * Public repositories: use `pre-commit.ci` for automatic checks (instead of integrating `pre-commit` into CI) | ||
| * Private repositories: use `pre-commit` in CI | ||
| * Code style: | ||
| * `black` | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. maybe don't even mention |
||
| * `ruff` for implementation | ||
| * recommended to use inside `pre-commit` | ||
| * Code linting: | ||
| * `ruff` for implementation | ||
| * recommended to use inside `pre-commit` | ||
| * Static code analysis: | ||
| * `mypy` | ||
| * Use type annotations in code please, more infomation [here](https://docs.python.org/3/library/typing.html) and [example here](https://github.com/PyPSA/technology-data/blob/40ca61b9aaf8c3019fa264a35ca999e6c41ef562/scripts/compile_cost_assumptions_usa.py#L198) | ||
| * recommended to use inside `pre-commit` | ||
| * Code testing: | ||
| * `pytest` | ||
| * unit and integration tests | ||
| * recommended to use inside your CI | ||
| * use `pytest-cov` to measure code coverage | ||
| * for faster, parallel tests, use `pytest-xdist` with `pytest -n auto` | ||
| * Workflow management: | ||
| * `snakemake` for modeling and data processing workflows | ||
| * | ||
| * Data sharing: | ||
| * `Zenodo` for sharing data publicly with a DOI | ||
| * `Google Drive` for sharing data internally or temporarily with externals | ||
| * Licensing and copyright: | ||
| * Technical: Follow the [reuse](https://reuse.software/) specification | ||
| * Use `reuse annotate` to add license and copyright information and `reuse lint` to check for compliance of a repository | ||
| * recommended to use inside `pre-commit` | ||
| * Licenses: | ||
| * `MIT` for code | ||
| * `CC-BY-4.0` for documentation | ||
| * `CC-BY-4.0` for data | ||
| * `CC0-1.0` for metadata and general repository configuration files | ||
| * For details, see handbook entry on [Licensing and copyright](../Handbook/CopyrightAndAttribution.mdx) | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.