SWAR is great, but this is a huge problem.
How to use multiple drives for storage.
My current technique is to set
Destination_directory: D:
maximum_plots: (drive size/108gb)
When this drive is full, I shutdown and edit the YAML for a new drive…
This wastes time since the stagger has to start all over. It wastes nvme drive durability since I’m deleting temp files before restart.
Do I even need to delete them?
Is there any way to automatically switch to the next destination_drive without shutting down, editing, and restarting?
It’s possible to start multiple jobs… with each job getting some resources, but I think this negates the benefits of staggering since the resources are no longer shared…
yes, delete temp files of plots you stopped, otherwise your temp drive will just fill up with them.
You can actually insert a list instead of a single folder as the destinations drive into the config.
So if you run 4 plots you can have every job go to a different destination drive. This doesn’t completely solve your problem but makes it much less frequent.
Just don’t ask me the YAML format to do that, no idea
But it is in the description that it is possible
Have a look at Ploto, it automatically checks for drive space and wont start a new plot if there’s a lack of space. I really like SWAR but it’s lacking features that Ploto has. Trade off is no API, no max phase 1 or start early. I’ve not used it yet but will be running it once i change out NVME’s this weekend.
