Wide range of plotting times when running 4 parallel

Chia Plotting
1

Here’s some output of 4 plots I ran parallel and staggered about an hour each.

Total time = 25198.229 seconds. CPU (99.270%) Wed May  5 14:44:48 2021 (7.0 hours)
Total time = 24477.250 seconds. CPU (99.970%) Wed May  5 14:55:28 2021 (6.8 hours)
Total time = 35446.726 seconds. CPU (69.090%) Wed May  5 18:36:14 2021 (9.8 hours)
Total time = 35056.360 seconds. CPU (70.390%) Wed May  5 18:52:26 2021 (9.7 hours)

Here’s the command I’m using to kick them off.

chia plots create -k 32 -b 5000 -r 2 -t Z:/ -d Q:/ -n 5

Isn’t it odd that the last would take almost three hours more?

Also, I’m having trouble finding the sweet spot for RAM. I read online you shouldn’t go > 5000, but I have 64GB total to allocate. Can I go up to 10g per process? Or would that have negative speed effects?

I’m running an i7-9700K @ 5.0ghz, 64GB 3200mhz ram, 1TB M.2 temp and copying them to a USB 3.1 external.

Cheers.

2

as you add more, you’re straining the system more. So your later plots will be slower. Though, I’m a bit surprised the 2nd plot was faster. But for me, I get about 7hr plots single. running 5 at a time, I’m up to 8.3hrs (4 to nvme, 1 to ssd).

default ram should be fine. It’s IO and CPU.

Also, how much ram do you actually have? You said 64GB to allocate, but list 16GB in your system. 5000x4 would eat up your ram if it’s 16… and that’d probably account for the big jump in time

3

I have 64, that was a goof. How much should you allocate before it’s counterproductive?

4

I’ve found even if you allocate a ton of extra RAM to the plotter, it doesn’t seem to actually USE it all. For example, I built a 10th gen intel 6c/12t plotter and I put 64gb in it which was a huge waste… so I figured might as well give the plotter extra memory, like so:

  1. plot 4t/16gb → nvme c drive
  2. plot 4t/16gb → nvme c drive (30 min stagger)
  3. plot 4t/16gb → sata ssd d drive (1 hour stagger)
  4. plot 4t/16gb → nvme c drive (4 hour stagger)
  5. plot 4t/16gb → sata ssd d drive (5 hour stagger)

It never even gets close to using all the RAM… from what I can tell…

image
image986×801 121 KB

… and from previous benchmarks, the plotter just can’t make much use of the extra RAM, it’s pointless. You’ll get maybe 1-2% faster plot, and those benefits are with merely an extra gigabyte or two of memory.

1 Like
5

@codinghorror thanks for sharing that. How much RAM do you allocate per process?

6

As it says above, 4t/16gb for each one?

7

Sorry, I got confused since I read on the subreddit for Chia going above 5-6GB will actually slow you down. Also, doesn’t the Chia wiki say 2 threads is optimal?

8

*Just a little side note: memory consumption has been SIGNIFICANTLY reduced since v1.0.4. My usual allocation since about a month before block one was -b 7700 and this now consumes what appears to be half of what it previously did per plot. My data backs up your findings Jeff!

1 Like
9

It is possible that going above that – adding too much RAM to a plot – will slow you down now!

That was not true in earlier versions but you bring up an excellent point. I need to scale back the mem on these 16gb ones.

Can you find a recent citation for the “too much RAM slows down plotting” statement @wwwroth ? I want to make sure. I do think it’s possible in the latest plotter for sure, because the memory requirements are so much leaner than they were in the initial releases.

10

This I can also debunk. Throw whatever number you want at it, after a big enough number it just nets zero improvement

1 Like
11

Ok phew. Good to hear my friend!

I think adding an extra GB or two helps speed plotting maybe 2% and that’s literally it.

1 Like
12

TLDR: No difference!

I’m the guy who was running those crazy :crazy_face:… hmm what if I allocate 32GB to this one plot (of my 128GB) just to see what happens. It does look “cool” in the output though! :laughing:

1 Like
13

The benefit of having 64gb of ram in my experience is pairing it with a high core count cpu like Ryzens 3900/5900 or threadrippers.

For example a 3900x can make use of 64gb of memory using default memory size while running 12-17 plots concurrently.

1 Like
14

Yeah you’d need a VERY high core count to make use of 64gb. It’s not even 4gb per plot, right? so 64gb can easily handle 16 simultaneous plots. And with my rule of thumb of “no more than 4 plots per nvme SSD” that’s also 4 NVME drives, which isn’t terribly hard to achieve, considering the mobo will typically have two full-bandwidth NVME slots.

1 Like