Currently running a pipeline which imports a remote file will download the files on every run.
It would be good to cache the imports so that subsequent runs don't need to pull down the files again.
It'd be good to also provide an option to run with what it has even if some files fail as currently it's impossible to run local tasks/pipelines if an import fails.
Currently running a pipeline which imports a remote file will download the files on every run.
It would be good to cache the imports so that subsequent runs don't need to pull down the files again.
It'd be good to also provide an option to run with what it has even if some files fail as currently it's impossible to run local tasks/pipelines if an import fails.