2nd Temp directory: What does it *do*?

After reading the docs, it looks like it will do stage 1-2 in TEMP1 and (optionally) 3-4 in TEMP2.
So if I have disks/folders

  • myssd/
    • myssd/temp1
    • myssd/temp2
  • myrust/
    • myrust/plots

and don’t do anything advanced, it will plot all 4 stages in myssd/temp1. Which is fine. If I point “Select 2nd Temporary Directory” at myssd/temp2, did I just

  • double my plotting space usage
  • create unnecessary copying of files between folders
  • Make it easier to clean up a crashed plotting process

How about if I pointed “2nd temporary directory” at myrust/plots?

2 Likes

Yeah I’ve been confused about this as well, but the impression I get is that it allows you to overlap / parallelize I/O a bit more and optimize overall plot time?

It moves plotting over to the 2nd location after stage 1&2. Can be marginally useful to move from fastest SSD to a slower SSD to free up your fastest SSD for a new plot – but your mileage may vary.

It’s worth mentioning that if you do this and your 2nd tmp dir is on the same drive where you want the final plot to be on, then you should set the 2nd tmp dir to the same dir where the final plot should be. This way you avoid copying the final tmp file to final dir after phase 4. Instead it will just get renamed, which is instant.

I find that secondary temp is most useful if you set it to be your final destination. There space is less of a concern.

(above info was recovered from Reddit, where information goes to be… lost forever once it inevitably scrolls off the homepage… :smirk:)

It feels a bit like a micro-optimization to me, but I’d love to learn more from anyone who is an expert on it!

3 Likes

I’m interested in exploring this, but I haven’t run any plots with it yet. I bought a 2 TB nvme SSD for temp space but also had a 1 TB sata ssd laying around that doesn’t have enterprise level endurance. Instead of being limited to 6 plots at a time, I’m thinking I could maybe get away with 8 or more if I could offload some lower io work to the sata drive.

2 Likes

I am running a test with temp1 and temp2 on 2 different drive.
Running 4 plots in total. Lets see how are the results! :wink:

I tried this last night on my NUC. Temp drive on the NVMe, 2nd temp drive on the SSD where the OS is. I ended up with a complete plot that was still .plot.2.tmp and 3 other incomplete plots. I was surprised to see so many files at once. I was under the impression with an hour stagger there would be only one file on the 2nd temp drive at a time. I don’t think it helped, but I don’t have hard numbers.

1 Like

Thanks for testing! It all helps with my current best guess of “don’t mess with it, pros only.”

1 Like

This is an old thread but it doesn’t seem like the topic was really hashed out and I can’t really find another discussion of it in the forum. So first, has anyone used the 2nd temp drive successfully? Here are my assumptions:

  1. Phases 1 & 2 are the I/O intensive phases and should be ran on a high performance drive. Like a quality NVME.

  2. Phases 3 & 4 are less I/O intensive and can be offloaded to another temp drive so they don’t have to suffer from the phase 1/2 traffic.

I have a system that will be online tomorrow night that I think has the situation to test this. I have a system that is currently running with 1 NVME. This second system will be exactly identical to the first but I have a nice Samsung SATA SSD that I could put in it as the second temp drive. Then I will be able to clearly see any performance gains or losses because I can compare directly with the first system output.

My biggest question is if a SATA SSD is good enough (performance wise) to be the second temp folder location? Or does it need to be an NVME also?

3 Likes

Hi everyone,

Temp 1 is used most intensively by Phases 1 & 2 and require a very high IOPS value for parallel plotting.
Specifying a different Temp 2 on another SSD which can be SATA frees up the IOPS of the primary NVMe for the first 2 phases!
I had good results when I split temp1 and temp 2 on the cloud! :wink:
doing ~20 or so parallel plots with about 8 hours per plot.

2 Likes

Assigning a 2nd temp drive does not affect the activity in phases 1 or 2. I just watched an entire plot last night with a 2nd temp drive assigned and the only thing written to temp2 during phases 1 & 2 is an empty tmp file. Temp2 isn’t touched with activity until phase 3. At that time there is still heavy traffic on temp1 while it performs the sorting but as it sorts each table it writes the results to that tmp file on temp2. Then when it rolls over to phase 4, it is done with temp1. There are files left over, but they slowly disappear as phase 4 progresses. Phase 4 and the final copy all take place from temp2.

You are very right!
However when you are doing a lot of parallel plotting, it helps to free up the bandwidth of Phase 3 & 4 from the temp1.

1 Like

My takeaway: no, I should not be plotting with temp1 on SSD and temp2 on rust.

Yes some people may care, but at my current plots/day, I’m not in that level by a long shot.

I started using a 1TB SSD for my temp 1 and transfering to an old 1tb HDD as final destination while I waited for the rest of my components to arrive and was finishing plots in 10 ~ 14 hrs.

Now I’m using a 1TB NVMe for Temp 1 and 1TB ssd for Temp 2, then transfering to my 8TB HDD as the final destination, doing ~20 plots in parallel with 2.5hr offset and I’m finishing 32k plots in 6 to 7 hrs.

I figured that it’s probably helping my NVMe last a bit longer at the expense of the cheaper SSD.

You are a saint - I was just posting to ask how this works and the auto-search brought me here. Thanks!

Here is a thread where I have been doing experiments if you are looking for more.

2 Likes

Can you elaborate on how you are plotting in the cloud, please?

Using EC2 servers on AWS!

Can the 2nd temp be 120GB because I have a spare 120GB SSD from an old laptop laying about. Or does it have to be 256GB?

I believe it would fit one plot… Barely. I believe it is similar disk usage as finished plot. But never really watched as I use 1tb so didn’t need to.

Side note for others, temp 2 allows me( using) Swar "Start early phase to be 3 instead of 4(although I don’t do it initially on stage 3. I also delay 120mins after stage 3 for "start early " (enough time to clear/compress enough of the phase 1+2 files off of temp1 before I start ramming another plot on temp1).

It also spreads I/O and tbw across more drives.