Copy input and output of provers to RAM#71
Draft
bclement-ocp wants to merge 2 commits into
Draft
Conversation
Contributor
Author
|
Note: I am seeing some crashes with these changes that I thought I had eliminated, marking the PR as draft until I figure it out. |
This changes the way Benchpress collects the prover's outputs from using a pipe to using temporary files that are read once the prover is done. On Unix systems, the temporary files default to being stored in `/dev/shm`; on other platforms, the default temporary directory is used. This ensures that the prover never gets stuck waiting for benchpress to read from pipes, but more importantly minimises context switches between the prover and benchpress.
This ensures that we don't benchmark the time it takes to load the problem from disk, and helps minimise noise in situations of high disk contention, and other noise from the filesystem cache. We are still limited by the RAM bandwidth but… what can you do. This is only effective on Unix (more precisely, on Unix systems where `/dev/shm` is actually in RAM); on Windows the default temporary directory will be used. To enable the same behavior on Windows, pass a RAM disk as `--ramdisk` argument. To disable the behavior on Linux, you can pass a directory that is not in RAM to the `--ramdisk` argument. The files will still be copied.
bclement-ocp
force-pushed
the
bclement/ramdisk
branch
from
edc1f99 to
d5b0260
Compare
c-cube
force-pushed
the
main
branch
5 times, most recently
from
d895531 to
88a7818
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the RAM disk part of #69, split off from #69
It includes the changes from #69 because there are multiple changes in
Misc.mland that would cause conflicts otherwise; hence, it should be merged after #69. The other two commits can be reviewed independently.