Optimizing plotting on an AMD Ryzen 5950x (or any other 16c/32t CPU)

Should probably take the plunge and go Linux with plotman. Still on windows kicking off the chia plot commands manually and staggering them with the sleep function.

1 Like

Has anyone thought of disabling OS mitigations against speculative attacks?

With Zen 3 they are still largely software based and you can disable them, in Linux at least. It might not seem worthwhile but this is the kind of workload that could benefit. It did for me on Broadwell, but that’s an old architecture. I’m curious what you would get to be honest.

For newest kernels there’s mitigations=off flag. With Windows on Zen 3 this can’t be meaningfully disabled.

1 Like

For Windows 10, select the console window I’m plotting in, ctrl+a, ctrl+c to copy all the console info, then ctrl+v to paste into notepad, then press ctrl+f to search for “total time”. Like so:

(that’s from the Phantom Canyon NUC, so 11th gen Intel)

I also have the console windows set to very large scrollback numbers so it retains a lot of console history, in the console preferences. Keep It Simple, y’all!

2 Likes

I have a terminal window opened for each drive and a tab for each parallel plot. Works pretty well, but I’d like to automate the process of sifting through logs.

Sorry for the boomer screenshot, I don’t have anything on this rig besides chia.

3 Likes

LOL @ “boomer screenshot” :rofl:

My general principle is, I don’t want to spend too much time over-optimizing anything… it can be such a timesink. But! I hope we can glean some good principles and guidelines for others with 16c/32t CPUs (or more?)

… because I have to say, there is inherent risk in doing that much plotting all on the same box. If you need to shut it down, or it crashes, it takes out a lot of plotting all at once. Plus there’s a massive amount of interaction betweeen {x} competing plotting processes, all touching RAM, all touching disk, all touching various CPUs…

2 Likes

Or you accidentally flip the PSU switch while moving your monitor since your rig isn’t in a case… That was a mistake :joy:.

Definitely risky keeping all your eggs in one basket. For example, I was testing a new plotting strategy the other night before bed and woke up to 0 new plots. I exceeded the RAM accidentally and they all started crawling. Wasted 10 hours.

3 Likes

there is inherent risk in doing that much plotting all on the same box. If you need to shut it down, or it crashes, it takes out a lot of plotting all at once

This is a great point, and one we should probably bring up more often as more people are planning their starting setups. In fact, I’m struggling with this now. I have a second PCIe to NVMe adapter that arrived but I haven’t installed because I don’t want to shutdown a system that is performing well, despite the data posted above that indicates my system is IO starved.

I have no doubt that your plan to throw more systems at plotting is a better approach than eeking out every last percent of performance from any single system. I would take that route if I could. Sadly I don’t have the physical space for multiple systems, so for me optimization is somewhat of a necessity. It can also be a fun mini-game.

3 Likes

Totally! I am not casting aspersions!

I just don’t want people to get FOMO over not getting absolutely perfect optimal plotting performance out of their box. That’s all :sweat_smile:

I think my priority is making sure people avoid common pitfalls… like saturating I/O, or running a crazy number of plots in parallel. So much trouble could be avoided if people started small and scaled up over time as they gathered data, rather than YOLO-ing it into “Imma make 50 plots/day RIGHT NOW!”

3 Likes

What boards are you guys using that enables using all 4 x PCIe 4.0 slots on the Asus Hyper M.2 card?

I’ve a Strix 570-E Gaming, 64Gb ram and 5950X but cannot use more then 2 slots of the card with my current setup. I thought I should be able too, but the Bios (latest) only sees two of the 4 x nvme.

The Hyper card is in the PCIex16-1 slot, a 3070 GPU in #2 slot, and a riser connected 2nd GPU in the PCIx1-1 slot. Both nvme slots on the board are populated with PCIe 3.0 nvme cards.
Raid and Bifurcation are enabled in the Bios.

Ideally I’d like to see all 6 nvme’s (Win10)

3 Likes

It’s covered in excruciating detail in this ASUS support page

https://www.asus.com/support/FAQ/1037507/

So if you are using Asus mobo with the Asus card you are absolutely golden. Just read closely and correlate the model number with the chart.

For other manufacturer mobos, the setting is in your BIOS under PCI bifurcation… be on latest BIOS, of course.

Thanks, but that’s what I orginally based my buying descision on. Asus board, Asus hyper card. All seemed good to go -

But as I stated, bios doesn’t not see the last 2 nvme’s.

1 Like

Well, I have the ASUS hyper 4 port card and the ASUS Tuf Gaming X570 in my config it HAS to go in the first PCI slot, and all 4 NVME drives are definitely recognized as long as I set the BIOS bifurcation setting … I mean, I can prove it, here they are: 1 2 3 4 5 6

Hi, i have 3900x but i can’t make more than 20 plots/day, my setting for plotman: 7 plots parallel, 45minutes stagger, 2 plots in phase 1, threads=8, r=6500
One of the possibilities that i wounder has effect on the performance is the limitation frequency of my ddr4 ram, my 64gb ram can support 3600mhz but i couldn’t enable it in the DOCP mode (asus tuf x570), so it runs at 2666 mhz, how do you think?

1 Like

Yeah, my gut says RAM speed isn’t a big factor in Chia plotting, considering the low RAM requirements overall. I’ve also noticed both of my 5950x systems default to REALLY low memory speeds out of the box even though I bought “3200Mhz” memory, it has that printed right on the box the memory came in! It’s so annoying. Turns out this is not true, memory speed does matter!

1 Like

Afaik, they all do this, you need to set it in the bios

3 Likes

Yeah it’s SUPER annoying. Plus, the system that cyberpowerPC built me didn’t even work at the rated memory speed! They sold me “3000 Mhz” memory but every single BIOS memory level other than 2400 Mhz resulted in bluescreens or outright failure to boot… sigh… I had to replace that “teamgroup” RAM with name brand 3200Mhz stuff, even then I got scared and set it to 3000Mhz in the bios…but that did work!

2 Likes

Very confusing, as per this Reddit thread.

Haha, ok, might have been a little optimistic / naive. Still new to plotting and have now come to the conclusion you need to plot for a substantial amount of hours for it to ‘settle’.

So, tried running 4x4, 3x4, 2x3…4x4 and 3x4 just clogged up phase 3 and slowed down the rest significantly. Pulled the runs probably too early before they settled down but wasn’t looking too good. 4x4 had 1hr extra on ph.3 over ph.1, 3x4 ph.1/3 looked pretty even (good sign), 2x3 worked alright (but under utilization), 2 plots/2hrs. Nothing special but decent steady plotting.

Changed a couple of days ago to a straight staggered run, think i had max of 3 in phase 1 which meant i hovered between 7-8 plots at any 1 time (3 in ph.3), 4gb/45min. Churned them out at 1 plot every 50-52 minutes, roughly 28/day which i was pretty happy with. Roughly 25.5-25.7k/s per plot.

1.1.5 dropped so good excuse to change it up a little again. Now changed to max of 4 in ph.1 and increased stagger from 45 to 52 (think i might have to adjust this back down) but dont think it’s ran long enough yet to get all plots in the que at once, sat at 7 plots total running right now but times are pretty steady at 22.7k/s per plot. Probably needs a few more hours to expand the que, and most likely drop the stagger down a bit.

Gen3 is 100% my limiting factor (IMO) and squeezing more than 28ish per day would be nit-picking. Conclusion, single Gen3 is probably more than adequate for 5800x/32GB with a healthy 2.8TiB per day, i’d be quite happy with that. Even with 6750mb allocated, max is only 31GB with 7 plots running.

5950x absolutely needs Gen4 drives and most likely 2 of them to take advantage of the core count, paired with 64GB.

On the topic of CCX usage, the way Ryzen works will always use both CCX providing the 2nd has at least one core stronger than the weakest on CCX1. Only way around that would be to use process lasso but can’t see that working with plotting. I do favour Win/Swar though as it lets me run CTR which is far quicker to setup than PBO/CO and keeps my clock speed nice and high along with epic L2/L3 cache speed.

RAM tuning i’ve not checked yet, just put 64gb 3600CL16 into the rig the other day and have been plotting so still running DOCP. Couple of rockets arriving this week so i’ll take the rig offline for a day, tune ram and get the rockets in.

I’ll most likely start with 14 (max for temp), 3t (42 total so within x1.5 / 48), 4500mb and see how it goes.

Rockets seem popular, no SLC, decent specs on the box but i’m keen to see how a pair of these compare to a P5510 (works out roughly the same price depending on deal found) so that’s the next step. I don’t think aiming for running 5950x to the max at 2t is worth it as that would need 2xP5510 3.84 and 128GB ram, it’s cheaper just to add another plotter which will also plot more than the extra gained from maxing out 5950x. 5900x is probably the sweet spot with 64/2xGen4, 5800x 32+/Gen3+.

4 Likes

This thread is amazing and the people involved are really into it

I’m already really interested in going the ryzen route after trying plotting with my 4yo system

2 Likes

I just put together my 5950X rig last night along on ubuntu and also have a couple other rigs (5900X and 3900X) also. I will post some data here as soon as they run for a bit and I find the best balance in load for the 5950X system.

3 Likes