BB disk on this machine is slow (under Ubuntu). when running BB it sees all CPU threads are utilized fully. I/O wait time kills the speed. I did thread -1 and it made no help on I/O wait time.
Maybe that I/O wait time suggests that your NVMe is just slow and is killing plotting times? That would also impact MM. Again, a screenshot of your resource monitor CPU utilization would help understand what your cores are doing (under MM).
Ubuntu has CPU memory network usage monitor build in. I have not figured out HDD SSD monitor.
Having other data, you may be able to infer how SSD performs. Also, you can run:
hdparm -t /dev/sdX
hdparm -t --direct /dev/sdX
To get the baseline.
Update
Added memory to 192 GB. First MM plot flies. 33 minutes. Second and later plots are slow, 45-60 minutes. I added -w into the command line. Write to destination after plot complete, before next plot. Problem solved. Every plot is 33 minutes.
But there is a -w time added between plots.
This phenomenon does not happen in Windows.
The Z840 (2x2699v3) running Linux and Bladebit(512GB Ram)
can plot in 8 minutes. But it needs BIOS setting ISOC=disable
as that improves performance by 250%.
Also BIOS setting “Home Snoop Mode” increases performance a bit.
You could try ISOC=disable setting and let us know if this also improves your
plot time with Windows+Madmax?
More Info:
Ultimate Upgrade for this machine to E5-2699v3:
2x (E5-2699v3 @2.30GHz, 145W, 45MB Cache, 18 Cores, 36 Hyperthreads), 1TB LRDIMM @2133MHz
Total: 72 Threads
Average Time per Plot: 8.1 minutes
Average plots per day 177 PPD
Best plot time: 8.00 minutes (479.8s)
Notes:
- This machine/CPU needs ISOC=disable in BIOS (~20minutes per plot with default ISOC=enable).
Perhaps this is somehow related to HCC (High Core count) die, as ISOC was not so
relevant for other v3 CPUs with less cores (MCC= Medium Core count). - Running 2 bladebits parallel has good performance also with ISOC=enable.
However, for this configuration there is no throughput gain for 2*bladebit parallel. - Reported result is for Home Snoop Mode (default Early snoop is a bit slower)
2 Likes
My t7610 is also running Ubuntu, and I don’t need -w switch. Maybe the difference is that I don’t use MM to write to the final destination, but rather use a script file to do it. However, this doesn’t explain the difference.
The only thing that I can think of is that your HD is NTFS formatted, and that means that the write speeds are really long (maybe exceeding plotting speeds?), plus Ubuntu NTFS support is done in the user space, thus potentially eating a lot of CPU cycles; therefore, slowing down MM.
1 Like
Is that setting related to v3 Xeons? I have Dell t7610 with 2x e5-2695 v2 but cannot find that setting in the BIOS (unless Dell calls it differently). I also don’t see “Home Snoop Mode.”
On this box, my BB runs are about 10+% slower than running 2x MM (Ubuntu 22.04).
Yes, this is specific to Xeon V3.
HDD is formatted in Ubuntu. Not NTFS.
My HP Z620 = Dell T5610. Crippled dual CPU system. Don’t know how 192 GB memory are assigned between two CPU. During MM, nearly all 192 GB memory are used (110 GB assigned to ram disk). when writing to destination disk during the early stage of next plot, I see “swap” is adding up. It does slow down everything.
Maybe the best solution for me is do -w to a 3-4 HDD RAID 0 destination. 5 minutes write to destination and move on to the next plot.
I turn off swap when plotting. MM has RAM assigned to it (110 GB), and it draws small amount of working RAM (few GB), so there is no need for swap. On the other hand, whether MM or mv (from script), those are not really writing to the destination, but rather to the RAM cache, and the kernel takes care of the final xfr. This is where the extra RAM usually goes to. This caching works nice when writing to the dst is much faster than writing to HD. Usually, you will see problems with writing to HD toward the last 1TB or so, sometimes really crawling (from what I saw before).
Also, as you don’t have enough RAM to run 2 instances of MM, maybe you should check whether BIOS has some NUMA settings and play with that. It may be that for just one instance, NUMA in BIOS needs to be disabled.
God in the future when plots complete in seconds we’re going to look back at this thread and laugh.
Reminds me of when I used a map to navigate when driving.
2 Likes
Is that ever likely to happen?
Doesnt that just open the netwotk to that type of attack?
Forcing k size to increase , causing longer plot times?
1 Like
To my knowledge there is the ability to make plots on demand using current tech just the cost for the tech would outweigh the benefits. Coming innovations may reduce the difference in cost but they don’t make it profitable yet.
Even if we have to replot to K33 if it happens fast I don’t think we will be that stressed so don’t worry.
1 Like
You kind of evaded my question, or didnt grasp what i was getting at.
Is it really ever likely that the general public without great expense can plot a plot that chia will accept in seconds?
Because it seemed that is the scenario you were writing about.
But as i wrote.
Meaning your scenario is no longer plausible.
2 Likes
No, because it is still cheaper to then store the plot than discard it and create a new one. Edit:in 2022
On ten year old consumer hardware it takes hours to make plots, so maybe in 10 years it will take seconds. In ten years let’s come back and laugh at our predictions
Will ssd finally be cheaper per TB than hdd?
1 Like
The point of making one in a cpl of seconds would be to wait for the proof request, then create a plot that fits, allowing you to get a reward.
You would need ridiculous amounts of hard drive stored plots to compete at that level.
1 Like
No. I forgot the exact amount but being able to plot JIT is worth around 400TB or something. People can do so now there’s just not enough incentive to balance the costs. There likely won’t be for 5+ years unless hardware gets a lot cheaper and more energy efficient somehow
Wheres the maths to back that up?
Because managing to craft a rewardable plot every challenge must be worth alot more than 400TiB.
It must be, or else 400 TiB would be capable of doing so ( getting a reward every challenge ) and it certainly cant.
I am unsure what " JIT " is, are you sure were talking about the same thing.
Edit.
JIT , im guessing just in time
Yh, lets see the maths.