Area
Architecture and Design
Motivation
Performance — speed or resource usage
Problem
Adapt is only able to have one processor running at a time, which can make processing large datasets very time consuming.
Proposed solution
Historical mode would greatly benefit from being able to run multiple processors at once and the ability to merge the outputs. This would allow for large datasets to be run in parts, at the same time instead of having to do everything out of a single node. The merging of repository folders would make it easier to manage running multiple processors and not having to worry about all pointing to the same output, which may not even be possible if there are multiple writing to the same file at once.
Alternatives considered
No response
Additional context
No response
Checklist
Area
Architecture and Design
Motivation
Performance — speed or resource usage
Problem
Adapt is only able to have one processor running at a time, which can make processing large datasets very time consuming.
Proposed solution
Historical mode would greatly benefit from being able to run multiple processors at once and the ability to merge the outputs. This would allow for large datasets to be run in parts, at the same time instead of having to do everything out of a single node. The merging of repository folders would make it easier to manage running multiple processors and not having to worry about all pointing to the same output, which may not even be possible if there are multiple writing to the same file at once.
Alternatives considered
No response
Additional context
No response
Checklist