You should be able to get around 5 to 6 hours running ~8 in parallel. But this assumes a well tuned Linux install running plotman.
i7-3770K @ 3.5 GHz, 4core 8thread, 32GB 1333 RAM, 1TB WD 3D SSD SATA 6G, 300 min delay, 6000 ram, 3 threads, Farm is 18TB WD Elements USB 3.0 (6G)
Total time = 32798.003 seconds. CPU (121.170%) Sat May 15 08:09:23 2021 Total time = 34124.709 seconds. CPU (118.860%) Sat May 15 13:31:28 2021 Total time = 33591.081 seconds. CPU (123.240%) Sat May 15 18:22:35 2021
Using GUI 1.1.5
My times have varied from 8 to 14 hours with different ram, threads and delays. I have a list of 50 if you are interested.
Hi there, my modest contribution
Intel I9 9900K (All 5ghz WC) 8c/16t, 6 parallels Plots, 32go Ram, 3300 per plot
- 3 plotting drive
- 1 Intel P3700 Nvme 800go
- 2 intel S4610 Sata 920go
planning to fill my 89tb farm in 40 days, waiting for the pool update to start, only testing and tuning for now.
I use only 12threads for now, my computer is doing other things than plotting.
I’m going more on sustainability and durability than speed and throwing nvme away (Massey Ferguson > Ferrari )
I’m getting 18 to 20 plots a day, the variation is due to the intel nvme beeing way faster than the two sata ssd, and sometimes “overlaping” their phase 1
As far as durability goes, my goal is to fill my 89tb of drive with plot and then planing to move to sas datacenter grade mechanical hdd for plotting, but massively parallel (with hpe server, rack format) Seagate XE14 seemed to be the ideal way to go.
Maybe selling the nvme / ssd’s when done, the price already go way above the regular market price and they may farm a lot more plot than I need… we’ll see what the future may be !
A post was split to a new topic: What does the percent of CPU time mean in total time?
Dell Precision Workstation
2 x Intel Xeon Silver 4215R CPU
128 GB RAM
8 x 1 TB Samsung 980 Pro NVMe
30 x K32 Plots Parallel = 24 hours.
Hi Boba, it’s a CPU architecture limit for both Zen 2 and Zen 3 series. AMD confirmed this back in 2019 that each CCD/IOD link is 32B/cycle for read and 16B/cycle for write. Chia plotting is super write-intensive. For DDR-3600, raw clock 1800MHz * 16B = 28.8Gb/s per CCD. Once your parallel jobs exceed this bandwidth, plot time start to slow down. The only solution is to make DDR4 freq as fast as possible! eg. DDR4-4000
Using PCIE4.0 ssd don’t help much either. Based on current Infinity Fabric design, the DDR4 cannot bring enough write data to the CPU for processing. Intel CPU wins in this part… Ouch
Yes this is consistent with some other graphs I’ve been seeing… Intel CPUs do unusually well. Thank you for the concise summary!
- Total time =
13894.995seconds. CPU (140.040%) Fri May 21 21:31:14 2021
- Total time =
13547.966seconds. CPU (144.550%) Fri May 21 19:31:49 2021
Averaging just under 4 hours total per plot.
These are some new timings after maxing the CPU and Ram which allowed me to get out of phase 1 quicker and reduce the delay:
i7-3770K @ 3.89 GHz, 4core 8thread, 32GB 1600 RAM
288 min delay, 8000 ram, 4 threads
Total time = 26725.885 seconds. CPU (126.290%) Sat May 22 21:11:35 2021 Total time = 28478.226 seconds. CPU (130.540%) Sat May 22 16:52:47 2021 Total time = 28067.965 seconds. CPU (124.580%) Sat May 22 11:57:57 2021
Interesting, i tried other ways with a similar HW…
cpu i5 3470 4core/4threars 16GB ram, 2xsata SSD 1T, 2x USB 3.0 HDD 10T
Using 1 single Plot, 3 threads, 3.3 GB ram, ssd1+hdd1
Total time = 33090.422 seconds. CPU (100.920%) Sat May 15 01:59:06 2021
= ± 2,6 plots daily
Using 2 parallel Plot, 2 threads, 3GB ram, ssd1+hdd1
Total time = 2x 55306.401 seconds. CPU (61.850%) Mon May 17 06:06:26 2021
= ± 3,1 plots daily
Using 2 parallel Plot, 1 threads, 3GB ram, ssd1+hdd1
Total time = 2x 64640.961 seconds. CPU (54.610%) Sat May 22 03:27:48 2021
Using 2 parallel Plot, 1 threads, 3GB ram, ssd2+hdd2
Total time = 2x 70897.936 seconds. CPU (50.360%) Sat May 22 05:13:26 2021
= ± 5,1 plots daily
Using 2 parallel Plot, 2 threads, 3GB ram, ssd1+hdd1
Total time = 2x 60079.563 seconds. CPU (59.060%) Fri May 21 03:15:34 2021
Using 2 parallel Plot, 2 threads, 3GB ram, ssd2+hdd2
Total time = 2x 67687.852 seconds. CPU (53.010%) Fri May 21 05:23:05 2021
= ± 5,4 plots daily
For this way is i7 much better.
Saying Phase1 is multi threaded is a bit misleading. More than 50% of the time the algorithm is still just single tread (and CPU bound) in that phase. Phase 2 is completely single thread and CPU bound. This is why fast single thread speed is highly important, and I also strongly suspect cores rather than threads will matter a lot as the hyperthread will be too affected by the already maxed out base thread.
Out of curiosity @Yae , there are some Xeon CPUs on the market without hyperthreading.
Would in your opinion a 8 core CPU be faster than a 4 core / 8 threads. ( When core speed, cache etc are all the same)?
8 cores is absolutely faster than 4+4 hypertreaded. There have been xeons without HT, but that was long ago. You can always disable HT in the bios.
i also just updated my ram with same speed on a ryzen 3900x but i now seem to have a bit longer times from 7hours 50mins to a 10-11 hours
with the gui plotting with 1x mp510 nvme 920gb and 1x 2tb crucial p5
before i used the mp510 and did queue 3 plots in the 7 hours and now
with storage pool the mp510 and the crucial trying 5-10 plots 3t and 4gb ram and getting 10 hours + with 30 mins stagger and also have 48gb ram of 3200 but Windows clock them at 2133 I think, not sure why it slowed down so much must be ram related i think like you guys said i ll try to overclock it and see if it gets any faster, or you guys have any better idea ?
- i7-11700 8c/16t
- Asus Z590-F
- ADATA SX8200 PRO 1TB for the first temporary (tweaked XFS)
- ADATA SX8200 PRO 512GB for the second temporary (tweaked XFS)
- RAM 32GB 3200MHs
- running on Fedora 34
chia plots create -k 32 -r 4 -t /temp1/ -2 /temp2/ -d /plots/
Time for phase 1 = 4942.052 seconds. CPU (186.810%)
Time for phase 2 = 2194.180 seconds. CPU (89.190%)
Time for phase 3 = 4652.688 seconds. CPU (95.740%)
Time for phase 4 = 322.057 seconds. CPU (98.430%)
Total time = 12110.979 seconds. CPU (131.790%)
I setup my new hardware and made fits plot on it. Working ssd is bottleneck for now, I’ve ordered Intel P4610 and waiting for next week it will arrive, so I believe I can improve those numbers. Till I waiting I will try some optimizations and measurements.
Maybe a stupid question, but you sure xmp is enabled?
Yes I’m with ASUS so it’s DOCP and its enabled for some reason though the whole pc is now really really slow I’m thinking of breaking the storage pool couldn’t even open a winrar file in normal time everything is super slow I might even format…
Ryzen 9 5950X 4 Threads / 6750 RAM
TOTAL RAM 96GB (2666Mhz)
2x Samsung 980 PRO 2TB (Full Performance Mode Enabled via Magician software).
Running 14 plots at the same time, average 8h 30 minutes each plot.
30817.074 seconds. CPU (128.790%) Thu May 27 21:59:44 2021
Doing 39/40 plots per day using swar manager.
For Queue 1
enable_cpu_affinity: true cpu_affinity: [ 1, 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23, 25, 27, 29, 31 ]
For Queue 2
enable_cpu_affinity: true cpu_affinity: [ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30 ]
I have noticed that the timings are really bad on regular plotting with 14 plots. By enabling cpu affinity this way allows each plot queue to have its logical processors. In regular Chia Client all plots from both queues at the start of phase 1 jump straight into CPU0/1/2/3/4/ overloading first logical processors.
Without CPU affinity using regular chia client.
11 - 12 hours was with the regular Chia client,
8 hours is with the swar manager.
Your total plotting time is very quick, how many plots could you do per day, how is the setting you do, RAM, Threats, Delay Time, or you do Queue?
It’s faster with some optinizations.
I did not test full performance yet. The reasons is: (1) ADATA SX8200 PRO is not so good for large writings, speed degrades and brakes performance, I’ll get P4610 soon and will optimize for it; (2) my hdd is full of plots, so I will run hard when pool support released; (3) now I working on process measurements to know what is happening and how it could be optimized.
Theoretical target is 80PPD. Next week, I hope, would show results.