Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,4 +149,4 @@ jobs:
push: true
cache-from: type=gha
cache-to: type=gha,mode=max
no-cache: true # This line enables building without using any cache
no-cache: true # This line enables building without using any cache
102 changes: 102 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,107 @@
# Changelog

## [3.0.0](https://github.com/CIRADA-Tools/science-containers/compare/v2.0.2...v3.0.0) (2026-05-26)


### ⚠ BREAKING CHANGES

* update from prefect2 to prefect3
* Version 1, first production version

### Features

* add astroquery ([c52d88e](https://github.com/CIRADA-Tools/science-containers/commit/c52d88ee7338bbfe7d606513d72b4bc5c0e029cc))
* add bagpipes ([7b53ee4](https://github.com/CIRADA-Tools/science-containers/commit/7b53ee46fbb54e1ced34500aee784a6df9c8c379))
* add cartopy and psycopg2-binary for status page ([cb0ae8a](https://github.com/CIRADA-Tools/science-containers/commit/cb0ae8a039f16602ba2131c8fe0e38872e502d70))
* Add java to notebook version ([fc2404c](https://github.com/CIRADA-Tools/science-containers/commit/fc2404c0fa83b0a3dd09c93af690c2ea879d80fb))
* Add java to notebook version try2 ([526fd55](https://github.com/CIRADA-Tools/science-containers/commit/526fd5519cfc7e8900625b8198044c27176a9a0e))
* Add java to notebook version try3 ([0b4c80c](https://github.com/CIRADA-Tools/science-containers/commit/0b4c80c4dfaeeed6413882f0c89a5670acd2511b))
* Add java to notebook version try4 ([e80cbfb](https://github.com/CIRADA-Tools/science-containers/commit/e80cbfb6a5f4b7bd5d7fcad0aeebfaee54b71fd3))
* add MPI ([e8f1924](https://github.com/CIRADA-Tools/science-containers/commit/e8f19249b6708cf856fbb76e6a4a0bc930d91fa2))
* Add pyfftw ([2585c42](https://github.com/CIRADA-Tools/science-containers/commit/2585c42f891f3040b44ef40c36f0d57c4580d7db))
* add python-dotenv for POSSUMutils ([ff6e81a](https://github.com/CIRADA-Tools/science-containers/commit/ff6e81a78527882614b3990d6dcc93d58f7e103e))
* add singularity, apptainer and screen ([6c46c84](https://github.com/CIRADA-Tools/science-containers/commit/6c46c84f8044fe766ac79f9de8585187e6688486))
* add utilities for possum2caom2 ([a89e5ad](https://github.com/CIRADA-Tools/science-containers/commit/a89e5ad444100136c276893c05d3a0e2cb866536))
* **base Image:** debugged micromamba installation process ([4a6b0ac](https://github.com/CIRADA-Tools/science-containers/commit/4a6b0acc470da7ffc265459d4de443464ecbb607))
* **base:** add beautifulsoup module ([fb425ee](https://github.com/CIRADA-Tools/science-containers/commit/fb425ee41c9233b752e2f4e38e157505078a52a1))
* **base:** Add no cache to continuous development whenever possum2caom2 needs to be updated ([c097797](https://github.com/CIRADA-Tools/science-containers/commit/c0977976746d87a71c2cd8c0d3c2105ffb259562))
* **base:** add rclone ([3b82ecf](https://github.com/CIRADA-Tools/science-containers/commit/3b82ecf45035492727f79c5d2f147e50f06d4a6a))
* **base:** add sssd to prefect container ([3894c4f](https://github.com/CIRADA-Tools/science-containers/commit/3894c4f4eaf3995204825705f7b9a75f2762ea27))
* **base:** add tk to prefect container ([38c51c0](https://github.com/CIRADA-Tools/science-containers/commit/38c51c058044b482fb1c0f4632d979a51ac74d34))
* **base:** implement prefect in separate container ([73c1005](https://github.com/CIRADA-Tools/science-containers/commit/73c1005186e62c47f38ebdd60df6d9eea7d6ad02))
* **base:** remove possum_headless from CD and add POSSUM2CAOM2 software ([cdcb63b](https://github.com/CIRADA-Tools/science-containers/commit/cdcb63bf219310b441cca51107482bc92669bd67))
* **base:** try install prefect in headless docker container ([d25b7c5](https://github.com/CIRADA-Tools/science-containers/commit/d25b7c5c811adf6d69485d9c35d99bddc9a77509))
* **base:** Unify Dockerfile for notebook and headless. Add prefect to notebook, no jupyter in headless ([2c3c1d6](https://github.com/CIRADA-Tools/science-containers/commit/2c3c1d67b61b333adcf23c3aecb586087d38b4ef))
* **build:** add quotes around python versions ([a64ef9b](https://github.com/CIRADA-Tools/science-containers/commit/a64ef9be5945c9cba1e0634317aa6503d2e06392))
* **ci:** added pre-commit ci action ([2cc8b3d](https://github.com/CIRADA-Tools/science-containers/commit/2cc8b3d92aeca39e3605a38d2fe3e90fbe36c347))
* **core:** also push notebook images ([214cdfc](https://github.com/CIRADA-Tools/science-containers/commit/214cdfc71ad4959df3292877e83d2c49c8a0719a))
* **core:** install yappi and psrecord ([be51367](https://github.com/CIRADA-Tools/science-containers/commit/be51367c6cc1ca50bbb559cd8a5c0428a5f09bdb))
* **core:** remove POSSUM pipeline code copy ([92c10f4](https://github.com/CIRADA-Tools/science-containers/commit/92c10f4e22557a46591e059d40332658fa4a03fe))
* **docs:** added mkdocs ([8505ae8](https://github.com/CIRADA-Tools/science-containers/commit/8505ae80f38eefda829eba3b1de201326f80d095))
* downgrade astroquery so I can login headlessly ([f069f82](https://github.com/CIRADA-Tools/science-containers/commit/f069f8255896e7951f5862ad162f5831e6169481))
* force update caom2repo to 1.6.3 ([8a11a7b](https://github.com/CIRADA-Tools/science-containers/commit/8a11a7b866bcd6d5ab11913f2913b8f75ad3f614))
* **Github-Actions:** added a new deployment step with matrix capabilties ([1d2b711](https://github.com/CIRADA-Tools/science-containers/commit/1d2b71185d9e757a7fc7588c2e375704514d1522))
* **Github-Actions:** added deloyment action for base container ([32df4a2](https://github.com/CIRADA-Tools/science-containers/commit/32df4a2cb4183ba57996524f2735f45e5c9d9b20))
* install FITSverify into notebook ([da6a8c3](https://github.com/CIRADA-Tools/science-containers/commit/da6a8c3f5da44a61fd6831a02f4e9777d7bb1d32))
* install pyregion ([2d76a41](https://github.com/CIRADA-Tools/science-containers/commit/2d76a4123bf404f112267d8c2fe9bcd33cdda74e))
* remove pyfftw and install cadctap pyvo and vos for interacting with youCAT ([76a4e2e](https://github.com/CIRADA-Tools/science-containers/commit/76a4e2e42a1af4fcdeb5ee3efb3a508ed0118091))
* **setup:** added python setup config ([e740204](https://github.com/CIRADA-Tools/science-containers/commit/e740204bef364144902201fbeb8ed6431536c7e3))
* update from prefect2 to prefect3 ([a1eb503](https://github.com/CIRADA-Tools/science-containers/commit/a1eb503e2e9c885f05ac128870b67cbe1d740dd1))
* Version 1, first production version ([47bc23f](https://github.com/CIRADA-Tools/science-containers/commit/47bc23f1cc9925d0387a87911f9425233721c1d2))


### Bug Fixes

* add bagpipes back in to resolve dependencies ([49f4fcc](https://github.com/CIRADA-Tools/science-containers/commit/49f4fcc66834023798ad4cbf299a3aa2dc4ee6b5))
* Add java to docker try10 ([fa46979](https://github.com/CIRADA-Tools/science-containers/commit/fa469797a997fc8774f251078fd882ffb28bbcd3))
* Add java to docker try11 (just notebook) ([7763496](https://github.com/CIRADA-Tools/science-containers/commit/7763496801d42b66a41d04bce759ca6905da99e5))
* Add java to docker try12 (jre instead of jdk) ([cfbd323](https://github.com/CIRADA-Tools/science-containers/commit/cfbd323713a164c48aefe21fd9a34a87ad88e7ae))
* Add java to docker try6 ([748bbe4](https://github.com/CIRADA-Tools/science-containers/commit/748bbe4fb9291a41f8fe49ce5bcd59286892e1cd))
* Add java to docker try7 ([29b3cdb](https://github.com/CIRADA-Tools/science-containers/commit/29b3cdb42d1424edc9a819bfdb52a9439c51aaba))
* Add java to docker try8 ([6b0cf4c](https://github.com/CIRADA-Tools/science-containers/commit/6b0cf4cbb72e1e786d7673d1f995b59b1438e48c))
* Add java to docker try9 ([8f1b0a2](https://github.com/CIRADA-Tools/science-containers/commit/8f1b0a2745d68e455cb65b33121d4748fd855701))
* Add java to notebook version try5 ([dae0cd2](https://github.com/CIRADA-Tools/science-containers/commit/dae0cd23451c7cbcb7840919b3334f33aac68200))
* **base:** Also add possum2caom2 to notebook ([c73b877](https://github.com/CIRADA-Tools/science-containers/commit/c73b87777d1b18ef0f2d5ebf32604d233a65de32))
* **base:** CANFAR requires sssd ([ae21bf6](https://github.com/CIRADA-Tools/science-containers/commit/ae21bf6f982f9efb0ad34868f4571f358ca99365))
* **base:** move to py3.12 ([5e294cb](https://github.com/CIRADA-Tools/science-containers/commit/5e294cb2d4828ddb5dd96e4e5f34758adc65090c))
* **base:** Moving to py3.12 ([7d5dd5e](https://github.com/CIRADA-Tools/science-containers/commit/7d5dd5e782944585d86589a9a5419fafffade51f))
* **base:** rclone requires unzip ([4f8e87c](https://github.com/CIRADA-Tools/science-containers/commit/4f8e87c6e58cdc971cd6f0be760a6ccacdd2c8b3))
* **build:** fixed ref issue in docker build ([d971218](https://github.com/CIRADA-Tools/science-containers/commit/d971218673fd62ff139db6fd7fc2fa1aeea138f8))
* **CI/CD:** fixed issues with docs build ([1661518](https://github.com/CIRADA-Tools/science-containers/commit/1661518052492836200733459ed2aa4230024103))
* **core:** add gprof2dot ([db32043](https://github.com/CIRADA-Tools/science-containers/commit/db320436f9fa89ef0a8c45fb7cf0dd9ec831809e))
* **docs:** fixed the spelling of ubuntu ([ff272b3](https://github.com/CIRADA-Tools/science-containers/commit/ff272b3d0033a970d7d8dce8b4d3417732953edc))
* downgrade astroquery so I can login headlessly ([246ac72](https://github.com/CIRADA-Tools/science-containers/commit/246ac7231217eb6ee33ad2f4540f6c2709cd84f9))
* fix pymultinest installation ([0babf21](https://github.com/CIRADA-Tools/science-containers/commit/0babf2154974903b60c4c88fe42ba0059e44f527))
* force update containers to get new possum2caom2 version ([04fd6e0](https://github.com/CIRADA-Tools/science-containers/commit/04fd6e00ebaeb6651ffc9da8058bd13cb8e9d3fa))
* force update possum2caom ([6649940](https://github.com/CIRADA-Tools/science-containers/commit/6649940c216a6e308f1516171e0fbbeff3ebf456))
* Force update to latest possum2caom2 ([1be123d](https://github.com/CIRADA-Tools/science-containers/commit/1be123d16a110df459e20d8be6ec96eb5638c19d))
* **Github-Actions:** fixed github actions syntax ([beafcc6](https://github.com/CIRADA-Tools/science-containers/commit/beafcc65d8103fe09375a6206845b80549319935))
* **Github-Actions:** working on deployment ([d19c775](https://github.com/CIRADA-Tools/science-containers/commit/d19c775de05eeb51931e68f72930a9b89c441303))
* **Github-Actions:** working on deployment chain ([c14f04d](https://github.com/CIRADA-Tools/science-containers/commit/c14f04dbf0a46db26821f2aa9134c9b438665ae8))
* **possum_prefect:** rebuild to pick up healpy-based possum2caom2 (fixes 0923 vs 0922 tile names) ([9ceb2d9](https://github.com/CIRADA-Tools/science-containers/commit/9ceb2d90c4a8ebdf5bd7153d903bf32a4e8f23ee))
* remove pyfftw as it doesnt work with py312 ([959109a](https://github.com/CIRADA-Tools/science-containers/commit/959109a7dac278c833d2f89c50c6657b5a699357))
* remove singularity ([0c7be0c](https://github.com/CIRADA-Tools/science-containers/commit/0c7be0ce089083cc8f044a19d5699aff69ada41f))
* should have been psycopg2-binary' ([285a038](https://github.com/CIRADA-Tools/science-containers/commit/285a038997f054229519bc62154187cbec1fab37))
* standardise on prefect 3.4.25 for now ([9850716](https://github.com/CIRADA-Tools/science-containers/commit/9850716881eac482646704e37de66db14a701df0))
* this should finally install apptainer/singularity ([c9c8f99](https://github.com/CIRADA-Tools/science-containers/commit/c9c8f992129e18b18735871e7514bd302e9b3865))
* try fix with dev version ([906c5fe](https://github.com/CIRADA-Tools/science-containers/commit/906c5fe2372deb0a3b22bd338222e9c2cd439e17))
* try fix with mkdir ([557cfda](https://github.com/CIRADA-Tools/science-containers/commit/557cfdaff70a2065a4e500e59bbfc26573732bd7))
* update dockerfile to address bug ([5e2b4d9](https://github.com/CIRADA-Tools/science-containers/commit/5e2b4d92e62d14ceb0295a4bd8e7d710294cb06a))
* update readme ([952c6ac](https://github.com/CIRADA-Tools/science-containers/commit/952c6ac9cbc81ea2a3d1dc3d0ccffea574b6b076))
* Update README (and force building new version of possum2caom2) ([b275161](https://github.com/CIRADA-Tools/science-containers/commit/b2751614e650ea9557142e487e15a974fb7a7047))
* update README and force new build to update possum2caom2 ([6955d1f](https://github.com/CIRADA-Tools/science-containers/commit/6955d1fadca24e9715ff3985e8625511a9c25571))
* update README and force new build to update possum2caom2 v0.1.4 ([80fd30c](https://github.com/CIRADA-Tools/science-containers/commit/80fd30cae06fcf7cb1646f5b7ebc1471c7f59f9f))
* Working version of fitsverify in both containers ([6e5d348](https://github.com/CIRADA-Tools/science-containers/commit/6e5d34832dc636b11c5d2c0853e9707e4eb85749))


### Documentation

* **docs:** initial port from science-containers ([af7abaf](https://github.com/CIRADA-Tools/science-containers/commit/af7abaf9276c51b2b6e6a748f0ffc9a179202932))
* **docs:** root README.md ([68ea7d3](https://github.com/CIRADA-Tools/science-containers/commit/68ea7d3a20e80dac9ffe0c689b7e4e11a2083dff))
* **docs:** root README.md ([c31227c](https://github.com/CIRADA-Tools/science-containers/commit/c31227c998cafbd7f4fe1af103989ccc3c1abe20))
* **rework:** index rework ([0072015](https://github.com/CIRADA-Tools/science-containers/commit/00720150466de4de4d2097a405f839e0ff48d764))
* **rework:** rework ([5bc217c](https://github.com/CIRADA-Tools/science-containers/commit/5bc217cc11b4ab72b156d8f96f8d35995b4a8d2b))

## [2.0.2](https://github.com/ErikOsinga/science-containers/compare/v2.0.1...v2.0.2) (2026-02-06)


Expand Down
10 changes: 4 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This repository hosts Docker container configurations for the VLASS* and POSSUM pipelines, optimized for deployment on the CANFAR Science Platform but more broadly employable as well. These containers are designed to facilitate scientific computing tasks in a reproducible and scalable manner. The repository is based on the original repository found [here](https://github.com/opencadc/science-containers/).

For POSSUM there are currently two Dockerfiles, which have identical software, except that the 'notebook' version installs jupyterlab, while the 'headless' version does not.
For POSSUM there are currently two Dockerfiles, which have identical software, except that the 'notebook' version installs jupyterlab, while the 'headless' version does not.


## Usage
Expand Down Expand Up @@ -58,12 +58,10 @@ Version 1.14 and up have a downgraded version of astroquery to allow headless lo

Version 1.13 and above will contain singularity/apptainer and screen

Version 1.7.5 Contains possum2caom2 as of 2024-08-20, which has the ingest set-up for the 3D pipeline.
Version 1.7.5 Contains possum2caom2 as of 2024-08-20, which has the ingest set-up for the 3D pipeline.

Version 1.7 now contains fitsverify, part of libcfitsio-bin. Version 1.7.4 should contain a new version of possum2caom2 (caom2 to version 2.6.2 and caom2utils to version 1.7.3, 0.1.6 of possum2caom2) which (hopefully) fixes ingesting 3D FDFs and RMSF into CADC.
Version 1.7 now contains fitsverify, part of libcfitsio-bin. Version 1.7.4 should contain a new version of possum2caom2 (caom2 to version 2.6.2 and caom2utils to version 1.7.3, 0.1.6 of possum2caom2) which (hopefully) fixes ingesting 3D FDFs and RMSF into CADC.

Version 1.6 (2024-07-16) now contains java (jre) and cadctap pyvo and vos as well. Note that pyfftw does not work with py3.12 as of 2024-07-23 but there is a separate container available on CANFAR which runs py3.11. Additionally there is also a container available that contains a 2022 version of WSclean (both not maintained on this GitHub page though).
Version 1.6 (2024-07-16) now contains java (jre) and cadctap pyvo and vos as well. Note that pyfftw does not work with py3.12 as of 2024-07-23 but there is a separate container available on CANFAR which runs py3.11. Additionally there is also a container available that contains a 2022 version of WSclean (both not maintained on this GitHub page though).

Version 1.0 (2024-05-16) is now here! The POSSUM polarimetry processing is now in full-swing and working (so far), so it is time for version 1.0.


2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "science-containers"
version = "2.0.2"
version = "3.0.0"
description = "Science Containers for CANFAR"
authors = ["CANFAR <support@canfar.net>"]
license = "GNU Affero GPL v3.0"
Expand Down
2 changes: 1 addition & 1 deletion science-containers/Dockerfiles/base/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ RUN apt-get update --yes --quiet --fix-missing \
tcsh \
&& apt-get clean --yes \
&& apt-get autoremove --purge --quiet --yes \
&& rm -rf /var/lib/apt/lists/* /var/tmp/*
&& rm -rf /var/lib/apt/lists/* /var/tmp/*

ADD nsswitch.conf /etc/

Expand Down
10 changes: 5 additions & 5 deletions science-containers/Dockerfiles/possum_notebook/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM prefecthq/prefect:3.4.25-python3.12-kubernetes
# see available versions at
# see available versions at
# https://hub.docker.com/r/prefecthq/prefect/


Expand Down Expand Up @@ -36,9 +36,9 @@ RUN apt-get update --yes --quiet --fix-missing \
screen \
&& apt-get clean --yes \
&& apt-get autoremove --purge --quiet --yes \
&& rm -rf /var/lib/apt/lists/* /var/tmp/*
&& rm -rf /var/lib/apt/lists/* /var/tmp/*

# install rclone
# install rclone
RUN sudo -v ; curl https://rclone.org/install.sh | sudo bash

ADD nsswitch.conf /etc/
Expand Down Expand Up @@ -97,7 +97,7 @@ RUN pip install numpy \
prefect \
pyregion \
tqdm

# Install other python packages with pip
RUN pip install RM-Tools \
RMTable \
Expand Down Expand Up @@ -176,5 +176,5 @@ RUN git clone https://github.com/${OPENCADC_REPO}/caom2tools.git && \
RUN pip install git+https://github.com/${OPENCADC_REPO}/caom2pipe@${OPENCADC_BRANCH}#egg=caom2pipe
RUN pip install git+https://github.com/${OPENCADC_REPO}/possum2caom2@${OPENCADC_BRANCH}#egg=possum2caom2

# Get jupyter in the headless version
# Get jupyter in the headless version
RUN pip install jupyterlab
10 changes: 5 additions & 5 deletions science-containers/Dockerfiles/possum_prefect/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
FROM prefecthq/prefect:3.4.25-python3.12-kubernetes
# see available versions at
# see available versions at
# https://hub.docker.com/r/prefecthq/prefect/

LABEL maintainer="Erik Osinga <erik.osinga@utoronto.ca>"
Expand Down Expand Up @@ -32,9 +32,9 @@ RUN apt-get update --yes --quiet --fix-missing \
default-jre \
&& apt-get clean --yes \
&& apt-get autoremove --purge --quiet --yes \
&& rm -rf /var/lib/apt/lists/* /var/tmp/*
&& rm -rf /var/lib/apt/lists/* /var/tmp/*

# install rclone
# install rclone
RUN sudo -v ; curl https://rclone.org/install.sh | sudo bash

ADD nsswitch.conf /etc/
Expand All @@ -49,7 +49,7 @@ RUN pip install numpy \
prefect \
pyregion \
tqdm

# Install other python packages with pip
RUN pip install RM-Tools \
RMTable \
Expand Down Expand Up @@ -126,5 +126,5 @@ RUN pip install git+https://github.com/${OPENCADC_REPO}/caom2pipe@${OPENCADC_BRA
# 2026-05-26: rebuild to pick up healpy-based possum2caom2 (fixes 0923 vs 0922 tile coordinate names)
RUN pip install git+https://github.com/${OPENCADC_REPO}/possum2caom2@${OPENCADC_BRANCH}#egg=possum2caom2

# dont get jupyter in the headless version
# dont get jupyter in the headless version
### RUN pip install jupyterlab
Loading