How would you set this HW up?

Hi, new to chia, but have mined other coins in the past (ETH,Litecoin etc). I work for a small IT company and have a bit of old gear to throw at this.

I currently have 2 rigs plotting. Neither is great, and they were thrown together quickly. Was wondering what tips people have for maximizing plotting on them.

Rig one.
Dell Optiplex
32GB PC-2400T
256GB NVME (OS - ubuntu 20.04)
I have a stack of old 256 SATA SSDS (mostly samsung evos).
Currently with this rig, I am setting up two raid 0’s with a pair of sata ssds and plotting one plot on each set. Eventually I may get a better, high endurance NVME for it and plot there too. But currently just using what I have on hand, and will swap out SSDS as they fail.

Rig two:
Older HP rack server.
2x Xeon E5-2650 (8 core/16 thread)
6x 15k 300GB SAS drives
Currently all drives are in a RAID5. I have two more bays, but I need drive trays, then I can stuff something in those.
Also Ubuntu 20.04.
I plan on rebuilding this - wondering what the best drive configuration would be? Each single with OS somewhere else like a flash drive?

Farming plots are stored on a 12 bay synology stuffed with old 3TB SATA HDDs. I have two other NAS, for a total (depending on RAID and other office storage needs) of about 24-35TB storage without even trying.

Pointers to links for similar builds or any other tips are much appreciated.

I wont try to help you with the server as I am not the guy for that but I can help with the Optiplex.

I plot on a 7020 small form factor with 32GB RAM, a 512GB SSD for the OS and page file, a 2TB (1.81 usable) EVO 970 Plus NVMe for my main plotting drive, and a 2TB SSD as a secondary plotting drive. I plot to USB 3.0 HDD arrays.

Your have two intertwined concerns. The first is having enough plotting drive space for the number of parallel plots you are doing, and second is not doing too many in parallel. You can burn your cpu up if your try to push this machine too hard.

I have tuned my machine for maximum plots per day while not overly taxing the CPU, and have come up with a plot production rate I am very pleased with.

Before you start plotting you want to go into your system BIOS and turn the fans from auto to always on. Your CPU will be running at or near 100% for hours at a time and your NVME can use help staying cool, as well.

First you need 2TB of plotting drive. NVMe is by far the fastest, but SSD will do. You could RAID 0 a bunch of your SSDs to achieve 2TB, but you are probably better off buying a PCIe 3.0, 2TB NVMe.

You can do without a secondary plotting drive. It speeds the process a little but is not necessary. If you do want a secondary plotting drive, RAID 0 of your SSDs is a good choice for this job.

If you are still with me I will go into why I (and I believe you) should be plotting 3 K33s in parallel on your machine and completing them in under 18 hours.

1 Like

My Dell is an optiplex 7050 small tower. It only has a few sata ports, and I added a SATA card and some SATA power splitters, routed out the back so I can set drives on my desk.
I plan on getting a good NVME eventually, but I want to make best use of what I have already.
Might I ask why you recommend K33? I am plotting K32s in 7-8 hours currently. 2 in parallel, one on each RAID 0. My understanding is that K33 takes longer with no current benefit.

As a side note, I made 2 RAID 0s of 2 drives each instead of a larger one, so that if a drive fails and an array drops, the other one keeps plotting and I do not lose that one. Then I can grab another drive from the stack and build a new RAID 0 (and repeat until I am out of drives). These are all used drives from upgraded systems that needed more capacity that I had sitting in a drawer.

The little optiplex does not have much power to spare. I used one power split but would hesitate to use more. This is one of the reasons I got the NVMe as it uses power from the PCIe slot it lives above.

You might want to consider some external power and/or a plug in array. I believe you also only have one PCIe slot. Assuming you use that for an NVMe, everything else must be USB. Fortunately, you have a lot of USB, lol.

As to the K33s, we had that conversation including my own personal test findings in:

Bottom line is that a K33 has the same value as 2 K32s, has a higher chance of a much longer viability (K32s will become useless at some point) and, contrary to popular belief, it takes less time to plot one K33 than 2 K32s parallel. This may or may not be true on high end machines where the plotting drive is the bottleneck. Our bottleneck is the CPU, and running one long process turns out to be easier on our CPUs than two shorter ones at the same time.

If you install a 2TB NVMe you should expect slightly better production than I.

I churn out 3 parallel K33s every 17.5 hours. This is 4 K33s/24hrs and equal to plotting 8 K32s every 24hrs. You are getting in that range already, without the NVMe.

If you do plot K33s the way you are setup it would be brilliant. No parallel means that you can plot single K33s until you fill the target HDD without having to constantly restart plotting every 7-8 hours. Without the parallel you may also see a significant production increase, no NVME required.

I suggest running a single K33 in your current setup and checking your numbers.

As to your RAID 0s, all good, as long as you have enough for the plot you are doing. 1 K33 is slightly larger than 2 K32s both during plot and after it is laid down. It is a small margin so you should not need to enlarge the RAID 0 plotting array from your current setup unless you are very unlucky. There is actually a little margin under the numbers given for plotting size needed.

1 Like

Hi Brad, I have two dell servers, one is an R515 with 12 x 4TB SAS, this is mainly a farmer but it has enough spare CPU to be doing 4 parallel plots most of the time. I plot to the SAS drives selecting for example drive E as the final destination, drive F as temp1 and drive G as temp 2. Once E is filled I will move down a drive letter.

The other machine is an R720 dual E5-2650v2 with 8x 3TB SAS similar to your HP, but I got a deal on a stack of ram so I have 320GB. This means rather than parallel plot, which I was doing first I tried via GUI in windows 10, then via PowerShell CLI plotter to 7 of the drives as 7 separate processes, plotting to each drive as temp and destination, then moving the plots off to the R515. This took around 16 hours for 7 plots so not a very impressive throughput. I tried adding in 6 more USB drives and plotting to all 13 at once, but this failed twice, so I am not sure if plotting directly to USB HDDs is worth it, but farming from them should be - yet to determine though. 13 at once doesn’t run the processor much over 40%, so more would be possible with a backplane or enclosure supporting another 10-20 SAS drives to plot to in parallel.

Swar plot manager helps to manage all of this, but with staggered parallel plots running 24/7, it’s a PITA if you need to reboot or maintain a machine as you will stop multiple plots mid-way through and waste them.

So…Now I am trying with madmax plotter - the fury road windows version to see how fast it can plot 1 plot at a time throwing everything at it (though I’ve only tried one so far and it didn’t max out my CPU), plotting to partially ramdrive, though I am also going to try 100% to ramdrive. Even if this isn’t faster, the ram will not degrade like an nvme or ssd would so this is a big plus for me.

Update - plotting 1 K32 plot using Furyroad/Madmax using 120 GB ramdrive and SAS as temp drives took 10000 seconds (2.7hours) allocating 28 threads, though the processor was only above 50% (a little) during phase 1 and 3, so I think I could stagger two parallel processes maybe get 2 plots in 3.7 hours? I have enough RAM.
I also tried entirely ram drive and used 280GB ram with two temp folders on the ramdrive. It succeeded but took 12700 seconds, so there is an advantage to using two temp drives, it seems rather than one very fast one. Many people have been raid 0 striping two SAS drives together to get a bit more speed. I will try this too, if I give up on parallel plotting to 7+ drives and limit it to 2 in parallel, then raid 0 pairs of drives as temp 1’s will be better than 8 x single drives.
I will also try this direct to SAS also to see if it makes a difference as I am not totally convinced ramdrive is all that worth it. Maybe my RAM isn’t all that fast (DDR3) or well utilised.
I’ll just blog different permutations here.
Ramdrive is worth it. Direct to SAS it was 4 times slower for the first table. So I abandoned that.
Now trying stagger using ramdrive for temp2 and SAS for temp1.

Tried 2 plots staggered, using Madmax/fury road 0.0.4.
Second one began after phase 1 completed on first one - 3000 seconds.
First plot finished in 13394 sec (3.7 hours).
Second plot took 14000sec (3.9 hours)
So I could make 12 plots per day with this software, with fewer disks spinning, and the ability to interrupt the system some way through.
I am wondering if my slow(ish) ram is the issue and why I cannot get the CPU to really max out.

1 Like