Trying something out tonight now that I have my rig running nice and stable. I have a 1TB NVME in my NUC, so right now, I can run 2 plots at once (I staggered them by an hour for now). Hopefully with 1.0.4 coming out soon, I’ll be able to get 3 going at once.
Alright, I’ve now run 6 plots since starting the test, 4 on internal NVME, 2 on external Sata HDD (Over USB 3.1). All were ran with 2 threads on an i5 NUC, and 4GiB RAM each. I ran with about an hour of stagger between each plot, with a few queued per each, and all were written to a separate external usb 3.1 5TB hard drive.
All tests were ran on Chia 1.0.3. I was running 3 plots at any given time.
So as you can see, while the HDD plots were about 40% slower than the NVME plots, they still weren’t that bad overall. And since I can run an HDD plot while I’m running the NVME plots, it ultimately allows me to squeeze out a few more plots per day.
So, my next test I’m running now, is to run 4 plots at once, 2 threads each, with 3250MiB RAM, staggered about an hour or so each, 2 running on the NVME at once, then 2 on HDDs (the pair of 12TB drives).
There should be just enough RAM (16gb total on system) to handle this. But if this test works, I should be able to realistically get 10 plots a day out of this little NUC, and possibly more when 1.0.4 comes out since I think I’ll be able to do 3 on the NVME at once at that point.
Oh yeah so running at 4 at once definitely ruined the plotting times.
Internal NVME
Plot 1: 10h14m
Plot 2: 10h20m
External SATA HDD
Plot 1: 17h36m
Plot 2: 17h37m
Now admittedly, I didn’t stagger these all that well, so could have probably squeezed out some better performance. But since this was considerably worse than my previous run, I’m going to play it safe and go back to 2 plots on the NVME at once, staggered, and 1 plot on the HDD. All with 2 threads, and the default 4.6GiB of RAM each.
I’ll run another test once 1.0.4 is officially launched
I definitely think plotting in parallel, with a bit of stagger in the beginning to help separate the heavy threading, is the way to go.
I’m currently running 3 plots at once, on the 1tb NVME, with Chia 1.0.4, staggered an hour behind each other. Looking like an average of 7.5hrs per plot that way, we’ll see how it does after a few more iterations.
Oh I actually just pulled the beta from their repo and fired it up, seemed stable enough from what was being said in Keybase. Running fine so far about 10 plots in
I tried with 4 plots to start, to see if everything is going well and the total time have been between 17h22m and 18h32m for each plot… I staggered them by an hour and used 3 threads and 4GiB RAM each.
Now I am trying again to plot just 2 plots with 8GiB RAM and 4 threads each, but I don’t have much hope to see big improvement…
Once these plots are finished I will try to plot with same config, but with an external HDD, to compare it.
Through my rough Spanish, yours seems like a good drive. For what it’s worth, I use 2 threads and 3.4gib of ram per plot, staggered now at 1.5hours each. Running pretty consistently at 7.5 hours.
I just tried starting a test with 3 nvme plots in parallel, staggered, and then 1 hdd plot as well. We’ll see how it goes, I’ll report back!
Thanks! Yours seems have higher writing speed, but anyways I think my times are really slow. I’m thinking my nvme drive or my nuc are not working well…
With more RAM and threads and only 2 plots it improved a little bit, but nothing significant:
Plot1: 16h11m
Plot2: 16h42m
Now I’m plotting 2 more using an external HDD, staggered 1.5h, 8GiB RAM 4 threads. Let’s see what happens…
The network is growing really fast and my plotting speeds are frustrating… I have 24TB really to fill but it’s going to take a lot of time
Yeah I hear ya, I have 5.5tb filled so far, 24 more to go as well (assuming I don’t buy more drives). I’m pretty reliably getting 9 plots per 24 hours, so I’ll nearly fill my drives by the time transactions open up.
You should be able to plot twice as fast as that, something is off. Try using dstat and the other tools to watch the write throughput of your drive. But even then, it’s weird the speeds you’re getting.
I’m curious what your external drive speeds will be.
Maybe 9 plot/24h is not too much comparing to many of the chia farmers, but to me seems pretty good plotting speed!
With my build definitely is happening something, the NUC recently shut down without any reason. I look at the logs and after almost 30h still in phase 1 the two plots using the external HDD as temp drive…
With the same external drive and a 9 years old computer (i7 2700k and 8gb RAM DDR3 1600) took me about 25h to plot 1, it’s so weird.
I used some monitoring tools and seems everything is working fine except the writing speeds, which are extremely slow… I don’t have much experience, but I think everything is mounted correctly… So, due to this and the unexpected shutdown, Amazon will replace me the NUC for a new one.
I also wrote to @storage_jm form Chia Decentral and thanks to his advice I’m replacing the NVME drive too for a Corsair MP600 which should improve by a lot the plotting speed.
Will report my new speeds as soon as possible (hopefully this Wednesday) with the new NUC and drive!
I’m running another test right now, using a simpler way to measure throughput, I’ll share my results here tomorrow along with the method for measuring it all. I might whip up a script to process log files and spit out pretty stats too, we’ll see.
Can wait to see that results and script!
I’m using a simple script that returns Total time per plot.
The new NUC and NVME drive work pretty cool!!
Finally I bought the Seagate Fircuda 520 1TB, because the heat sink of the Corsair was too big for the NUC.
This are the logs of the old build:
Total time = 58605.715 seconds. CPU (55.410%) Sat Apr 10 12:46:48 2021
Total time = 59537.603 seconds. CPU (53.870%) Sun Apr 11 10:59:23 2021
Total time = 58117.708 seconds. CPU (55.080%) Mon Apr 12 09:36:26 2021
Total time = 71287.644 seconds. CPU (46.610%) Tue Apr 13 05:39:24 2021
Total time = 63844.642 seconds. CPU (51.740%) Fri Apr 9 15:30:10 2021
Total time = 68287.839 seconds. CPU (48.550%) Sun Apr 11 14:55:14 2021
Total time = 63223.090 seconds. CPU (51.770%) Mon Apr 12 12:01:31 2021
Total time = 70469.802 seconds. CPU (47.090%) Tue Apr 13 07:49:22 2021
Total time = 67813.585 seconds. CPU (48.630%) Fri Apr 9 17:36:19 2021
Total time = 66445.564 seconds. CPU (49.300%) Mon Apr 12 13:55:14 2021
Total time = 70986.216 seconds. CPU (46.990%) Tue Apr 13 09:58:40 2021
The new NUC and NVME:
Total time = 24083.250 seconds. CPU (110.530%) Tue Apr 13 21:24:02 2021
Total time = 24630.669 seconds. CPU (110.150%) Wed Apr 14 04:21:59 2021
Total time = 24369.612 seconds. CPU (110.680%) Wed Apr 14 11:15:39 2021
Total time = 25111.424 seconds. CPU (108.810%) Tue Apr 13 23:41:10 2021
Total time = 24946.530 seconds. CPU (108.750%) Wed Apr 14 06:44:24 2021
Total time = 24633.652 seconds. CPU (110.200%) Tue Apr 13 22:33:13 2021
Total time = 25520.944 seconds. CPU (108.820%) Wed Apr 14 05:46:02 2021
For now I will fill try to fill as soon as possible the drives I have. I have improved a little bit the speeds, about 22k sec per plot average plotting 3 in parallel. Also, I’ve just started to simultaneously plot another one using the external HDD, to compare results. Will post here.
I want to learn more about the pools, but maybe, depending on how things are going I will buy some more drives to join the pool once it’s launched! Probably I can add 1-2 HDDs of 10-14tb. I will try to find some good deals.
Yeah that’s my plan with the pool, I’ll have my own approximately 50tb of plots, and I’ll setup another external drive case with about 40-60tb of new space that I’ll plot into the pool. Will be interesting to see how the coin prices shake out starting in a few weeks leading up to pooling availability, will influence how much we all invest further I bet.
Out of curiosity, are you running with bitfield disabled (-e) or leaving it enabled in your plots?
