TL;DR
Replace
|
future::plan(future::multisession, workers = parallel::detectCores() - 1) |
with
with(future::plan(future::multisession), local = TRUE)
Details
Hello, I'm running reverse-dependency checks on future when checking FracFixR and I got the following error;
Error: processing vignette 'FracFixR-intro.Rmd' failed with diagnostics:
Cannot create 255 parallel PSOCK nodes. Each node needs one connection, but there are only 123 connections left out
of the maximum 128 available on this R installation. To increase this limit in R (>= 4.4.0), use command-line option
'--max-connections=N' when launching R.
--- failed re-building ‘FracFixR-intro.Rmd’
This errors is due to detectCores() in:
|
future::plan(future::multisession, workers = parallel::detectCores() - 1) |
detectCores() is too harsh, because it doesn't respect what compute systems have given the current users, e.g. HPC compute clusters, servers shared with multiple users. It actually is also against the CRAN policy, which only allows two parallel workers. Then it is better to use parallelly::availableCores(), because that respects all of the above automatically. So, the first change would be:
future::plan(future::multisession, workers = parallelly::availableCores() - 1)
Continuing, because there are systems with a single code, the above might result in workers = 0 - detectCores() is worse - you might end up with workers = NA. So, the safest is to use:
future::plan(future::multisession, workers = parallelly::availableCores(omit = 1))
You can read all about the problems and pitfalls with parallel::detectCores(), and how parallelly::availableCores() addresses them, in my blog post Please Avoid detectCores() in your R Packages.
Next, putting one CPU core aside is not really needed these days and something that stems from the old days. So, I recommend using:
future::plan(future::multisession, workers = parallelly::availableCores())
which is what most people use. Now, since workers = parallelly::availableCores() is the default, you can simplify that to:
future::plan(future::multisession)
Almost done. There's one more thing. As explained in https://future.futureverse.org/articles/future-7-for-package-developers.html, packages and functions should only change future::plan() temporarily. To achieve this, use:
with(future::plan(future::multisession), local = TRUE)
That will cause your changes to only apply in that function and undo the settings back to what the user might have set elsewhere.
TL;DR
Replace
FracFixR/CRAN/R/fracfixr.r
Line 119 in 7a69238
with
Details
Hello, I'm running reverse-dependency checks on future when checking FracFixR and I got the following error;
This errors is due to
detectCores()in:FracFixR/CRAN/R/fracfixr.r
Line 119 in 7a69238
detectCores()is too harsh, because it doesn't respect what compute systems have given the current users, e.g. HPC compute clusters, servers shared with multiple users. It actually is also against the CRAN policy, which only allows two parallel workers. Then it is better to useparallelly::availableCores(), because that respects all of the above automatically. So, the first change would be:Continuing, because there are systems with a single code, the above might result in
workers = 0-detectCores()is worse - you might end up withworkers = NA. So, the safest is to use:You can read all about the problems and pitfalls with
parallel::detectCores(), and howparallelly::availableCores()addresses them, in my blog post Please Avoid detectCores() in your R Packages.Next, putting one CPU core aside is not really needed these days and something that stems from the old days. So, I recommend using:
which is what most people use. Now, since
workers = parallelly::availableCores()is the default, you can simplify that to:Almost done. There's one more thing. As explained in https://future.futureverse.org/articles/future-7-for-package-developers.html, packages and functions should only change
future::plan()temporarily. To achieve this, use:That will cause your changes to only apply in that function and undo the settings back to what the user might have set elsewhere.