Failed bioconductor package installations do not cause docker builds to stop

[pmoris]

I've ran into a number of issues trying to install bioconductor packages in a Docker image. Specifically, the two approaches that I've foud for installing bioconductor packages (R -e 'BiocManager::install("package") and /usr/local/lib/R/site-library/littler/examples/installBioc.r) do not raise a non-zero exit code and thus do not cause Docker builds to fail when something goes wrong (e.g. package is unavailable in the repo, mis-spelled name, or missing dependencies).

I've raised this issue over at the rocker repository as well, with a lot more background information (rocker-org/rocker-versioned2#292), but I believe that it would be beneficial to add some warnings or clarification on the intended method to extend the bioconductor images to the documentation provided here: https://www.bioconductor.org/help/docker/

Any other advice or insight into how I can better handle these installations is of course highly appreciated as well.

[nturaga]

Hi @pmoris

Happy new year.

First, I'm not sure about the installBioc.r script; I've personally never looked at it or used it. So for the sake of reproducibility of the issue at hand, let's focus on BiocManager::install().

cc - @LiNk-NY regarding the BiocManager::install() issue. This isn't much of a problem with this docker image if I understand correctly, it's more to do with the functionality of BiocManager::install() itself.

Correct me if I'm wrong.

[mtmorgan]

A work-around might be to promote warnings to error, and then return an error status code, like

R -e 'options(warn = 2, error = quote(quit(status=10))); BiocManager::install("foo")'

(?quit suggests status 10:255 are appropriate for scripting errors). Promoting warning to error handles the case, like the one here, where a warning is generated that the package is not available. How do you detect failed package installations from install.packages()?

[pmoris]

Yikes, it seems I missed these notifications, so apologies for the tardy reply.

I delved back into this topic recently and came up with a solution that seems to serve my needs. In a nutshell, I've modified littler's installBioc.r to throw warnings whenever the installation of a package goes awry (due to missing system libraries, misspelled names, conflicts, etc.).

This allows me to use littler's install2.r script for installing packages from CRAN (or RSPM) and installBioc.r for Bioconductor packages. Both are easy CLI calls during a docker build and both will now interrupt the docker build when something goes wrong, rather than silently failing and ending up with a functional docker image that is missing some of the intended packages.

I've submitted this altered version of the script as a PR to the littler repository: eddelbuettel/littler#103

If you think it's useful, perhaps this approach (or something similar to it) could be used (or mentioned) by the docker guidelines for bioconductor? Because the current approach that is suggested here (https://www.bioconductor.org/help/docker/) leads to the problem I described.

@mtmorgan : thanks for the example! I ended up using your suggestion to install BiocManager itself in my dockerfiles, because I'd like to pass a bioconductor version to it explicitly, and this did not seem possible via install2.r.

EDIT: I forgot to address your question:

How do you detect failed package installations from install.packages()?

I haven't figured out a way to do so, and the recommended practice seems to be to use littler's install2.r script (see https://rocker-project.org/use/extending.html#e---error, https://stackoverflow.com/a/54305993 and https://stackoverflow.com/questions/49129223/let-docker-image-build-fail-when-r-package-installation-returns-error). Hence why I turned in that direction for a BioC equivalent.