Ordered new Plotter Build - $5168 (UPDATED)

rkalla · May 4, 2021, 2:43pm

Before anything, I want to thank all the engaged folks on this forums - been reading through 100s of posts to fill in a painful amount of blanks in my mind about how Chia plotting/farming works. No matter how small a question or esoteric build, it helped me understand different aspects of the Chia ecosystem.

In the name of transparency and sharing, I wanted to write up my experience with Chia so far to contribute back to this ecosystem of information.

Finding Chia to 1st build was something like 2 weeks; the sense of urgency around plotting seemed palpable - I missed the boat on ETH back in the day and Cardano SPO’s more recently by sitting on my hands for a month and didn’t want to do that again here.

My original goal was to build a machine for $1500-2000, stick it in a closet and forget it existed.

The more I read, especially around the growth rate of the network, the more it seemed like I either needed to go in a decent amount or not at all and just wait and buy the currency as soon as I could.

I would say the majority of the “going in a decent amount” was primarily because of the cost of disks spiking - I know EVERYONE here is annoyed by that. Especially when you see 18TB drives for sale for $350 a few months ago and now they are almost $700… uggghhhh

I have been prepping different kinds of builds on Newegg since Friday - Xeon Workstation builds, AMD consumer builds, Intel consumer builds, Frakenbuilds (re-using old cases, drives, GPUs I have here at home), etc.

Last night I finally felt a gut feeling of “Just f-ing do it” and locked in my Intel consumer build, the components are:

UPDATED based on Feedback…

Intel Core i9-10850K
ASUS TUF GAMING Z590-PLUS
[ADDED] EVGA SuperNOVA 850 GT (Thx @vandy)
- [REMOVED] Corsair RMX Series RM650x
Phanteks Enthoo Pro TG PH-ES614PTG_BK
Corsair Vengeance LPX 64GB (2x 32GB) DDR4 3200
(BOOT) SAMSUNG 860 PRO SSD 256GB SATA
[ADDED] (TMP) 2x Samsung 980 Pro 1TB (Thx @Quindor)
- [REMOVED] (TMP) 2x Seagate Firecuda 520 1TB
(PLOT) 6x Western Digital 14TB Ultrastar DC HC530 SATA

I think that’s really the meat and potatoes of the build - the CPU has an integrated GPU, so no standalone card there.

The other TMP SSDs I was looking at because they also had a really high TBW rating were the (Patriot VPN100 M.2 2280 1TB).

I should have all the parts here by Friday and get this together over the weekend. I’ll keep this thread updated with any findings if folks are interested.

EDIT: I should have stated - how I settled on the total drive size (for the initial build) - SUPER unscientific, but in these early days it’s hard to be much more informed than this - I fired up https://chiacalculator.com/ and scrolled until the total farmable TiB resulted in a monthly income, that when halved I was still happy with.

Mentally I am preparing myself that XCH falls to 1/10th of it’s current value and STILL ok with that - I think that’s what eventually got me to the place of just biting the bullet and diving in.

cultiv · May 4, 2021, 3:49pm

Thank you for sharing so extensively and THIS is what I would hope most people do, proper investigations. I tend to skip all the threads looking for easy answers (there are none) indicating that they never did any research.

So: well done, love that you’re sharing your findings and being an example for the rest of the members of this forum! :

ChiaKing · May 4, 2021, 3:54pm

Great Post. Using WD Ultrastar DC HC530 SATA hard drive how significantly lower would the speeds be compared to SSD or Nmve? I’m curious on what made you settled to HC530 for plotting.

Thank you.

rkalla · May 4, 2021, 3:57pm

Great question - I’ll be plotting to the drives marked ‘TMP’ above (the 2x NVME) and then immediately moving off to the 530s.

I don’t know that I would have wanted to plot on a spindle drive to be honest. I’m ok replacing SSDs every 6 months or so - I hope I can move up to the 2x TB SSDs are replacements for these when they bomb out.

The 530s should be good for quite a while (read heavy)

tstarr_stl · May 4, 2021, 3:57pm

I have a 10th gen intel in a build of mine, but just a heads up - the perk of using the z590 boards is being able to use the 4th gen nvme, HOWEVER the 10th series intel CPUs do NOT support them, so you’ll be capped at gen 3 speeds. Just calling it out in case you had a 490 board in mind with similar connectivity otherwise that would save you a few bucks.

Question - Are you planning on putting your 2 TMP drives in a raid0? I’ve seen one person post advising against it, but the majority of posts mention people ARE doing that (I have). Curious the route you’re taking there.

rkalla · May 4, 2021, 4:02pm

HOWEVER the 10th series intel CPUs do NOT support them, so you’ll be capped at gen 3 speeds.

SUPER good call out - you are spot on. I was willing to look the other way on this, but can’t say it wouldn’t have been nicer to run at full speed.

Question - Are you planning on putting your 2 TMP drives in a raid0? I’ve seen one person post advising against it, but the majority of posts mention people ARE doing that (I have). Curious the route you’re taking there.

Yes - one of the reasons I wanted that board (no RAID hardware planned… yet?) - the only reason to be against RAID 0 is data loss - when anything fails, it all fails.
For plotting that doesn’t seem like a big deal to me if I’m losing at most say 20mins of plotting time if a drive fails and I lose the progress in that plot - everything getting moved off to the storage disks once done. Also during plotting the name of the game is getting the plots DONE and off for farming as quickly as possible - so NVME-SSDs and RAID 0 it is

You DEFINITELY don’t want to run RAID 0 on your storage array - then you straight up stand to lose all the plots stored in the array.

The way I have it setup here is that I would stand to lose, at worst case:

Any in-progress plotting happening on the TMP drives.
Any completed plots stored on the drive that fails.

But the failure wouldn’t propagate beyond that.

The one case where I might start backing off the ‘RAID 0 for plotting is fine’ position is if I wanted to dump serious $$ into plotting disks and I have 4TB+ NVME drives… in that case, losing all that in-progress data (assuming lots of parallel tasks) could really get painful but I’m expecting that reality is a few years down the road when that stuff gets much cheaper.

Quindor · May 4, 2021, 4:08pm

Nice looking build but I do want to warn you, a lot of topics don’t highlight the importance of the SSD/NVMe correctly. They focus on manufacturer stated TBW numbers and tend to point towards high TBW drives.

While that inherently isn’t wrong it also doesn’t paint the correct picture in my opinion. TBW on itself is a good number to know but without knowing for what kind of workload this is for, it’s basically meaningless… I know that’s a bold statement, especially in this community!
It’s basically about the Write Amplification Factor (writes a drive actually does to a NAND cell to perform an action) that was used while calculating this number and generally a manufacturer doesn’t state what was used. So in practice it could be that subjecting the SSDs to a “chia” workload a 600TBW drive could actually survive longer then a 1700TBW drive in real usage without writing 200TB or so to a drive with that exact workload, we won’t actually know.

With that said, it’s still not a bad number to look at but for your build another value of the NVMe drive will be more important and that’s the sustained write capabilities of the drive. As can be seen in this graph by TomsHardware

the Seagate FireCuda 520 1TB has an SLC write cache (all TLC drives do) but then drops off to quite low levels of around 500MB/sec.

What this means in practice is that once you start to load the drive with more then let’s say 2 or maybe 3 plots in parallel performance will tank since the drive no longer has time to recover. It will try to perform your read and write requests while also managing it’s background processes but basically, grinding the whole setup to a halt. This is where plot times of “12 Hours” and such come from.

Said in a different way, a 970Evo Plus 1TB will be able to handle 3x as many plots or execute the same amount of plots (if there wouldn’t be a CPU limit) 3 times faster. With a good (non-limited) setup completing plot times under 4 hours is certainly possible, but with a few in parallel 4 to 5 hours is reasonable. The selected NVMe drives will not be able to sustain that in this case and I’m afraid that this will limit/cripple your setup more then you’d expect for the setup, having plenty of CPU and memory available.

Now I’m not saying I don’t like the drives, I actually like the Seagate brand for most stuff, and they do actually advertise their SSDs with durability and such, but performance wise I believe you will be disappointed. If part of the build is wanting to plot as fast as the hardware can handle, consider adding another pair of 1TB NVMe SSDs and running these in RAID0 with MDADM with XFS on top of them (personally tested XFS wins from EXT4 and heavily tuned ZFS). RAID0 will help with burst behavior and also you get double the cache basically. There are 15$ M2 to PCIe with heatsink little riser boards you can buy so you can easily plug in 2 extra NVMe drives. With that I believe you will be able to max out your processor and plot as fast as you can. Without, you’ll be limited to at or below half of what this processor will be capable of to get decent plotting speeds.

In the end the plots per 24Hrs is the only thing that counts, on a 5900x you should for instance be able to achieve ~50 plots per day if you tune everything right and then run into the CPU bottleneck, but you will require very high sustained NVMe/SSD performance.

vandy · May 4, 2021, 4:09pm

HEY MAN! welcome aboard!

Chia’s become a sensation, like a legit boy band one. You’re build looks well researched and thoughtful, only one caution is that power supply. I just built 8 “mega” plotters from scracth for a client this weekend and the only power supply I’m trusting going forward is evga. I’d sub what you ordered for something this caliber:
https://www.newegg.com/evga-supernova-850-gt-220-gt-0850-y1-850w/p/N82E16817438199

What OS are you considering running. Note: that anything other than Ubuntu 20 server will have performance implications! Windows will give you about a 20% drop in plotting performance

Quindor · May 4, 2021, 4:10pm

Agreed, well tuned (enabled discard in mount point!) MDADM RAID0 with XFS on it also performs best in my setups vs 2x single NVME because bursts can be much higher and you double DRAM and SLC cache.

rkalla · May 4, 2021, 4:19pm

Thank you SO MUCH for such a detailed response - I had no idea the falloff on the Seagate was so poor - I have a Samsung 980 Pro in my primary desktop that I love quite a bit so already a fan, but this makes me even more of a fan.

I see your point about blinding following TBW - certainly caching/write amplification will play a big role that I’m ignoring here.

UPDATE - just ordered 2x 980 Pros and requested cancellation on the FireCuda’s - really appreciate the timely and detailed heads up to get the build on point man… seriously means a lot.

If part of the build is wanting to plot as fast as the hardware can handle, consider adding another pair of 1TB NVMe SSDs and running these in RAID0 with MDADM with XFS on top of them (personally tested XFS wins from EXT4 and heavily tuned ZFS).

Is the favorite way to do this, the ASUS NVME PCIE RAID card I see mentioned a bunch and is currently sold out or are there other ways people like to set this up from a hardware perspective?

The mobo has 2 slots which I’ll use for now.

Thanks for the tip on XFS!

rkalla · May 4, 2021, 4:23pm

vandy, brother I know your posts WELL - your cataloging of the builds was one of the biggest enablers/inspirations for me.

I am already an EVGA fan (good PSU experience in my current desktop) - so to see you backing this one it’s a no brain - literally shot over to Amazon, requested a return on the unopened 650 and next-day ordered the 220-GT-0850-Y1 as specified.

I’ve seen your builds - if you are saying “X” is the right PSU, then it’s the right PSU

Really appreciate you weighing in here with the assist - means a lot dude.

Quindor · May 4, 2021, 4:25pm

Sure, glad I could help, I think the 980 Pro 1TB will do A LOT better then the FireCuda for this purpose, certainly fill up that CPU and memory and getting you fast plots. . On the PCIe cards, since you aren’t using any PCI slots, you have plenty of bandwidth and room available there, get 2 of these:

Although, as said, I believe those 2x 980Pro 1TB will serve you just fine already.

REVISED/UPDATE
And the TBW, this is a Corsair Force MP510 “B” version, performing well, but only 600TBW rated. But the numbers:

So 86TB written is 4%? Then linearly that would be 2150TBW for this kind of workload. But that’s speculating a bit since I’m not 100% sure on those values yet and I also do not know if “percentage used” is linear or not.

In the end, let’s see how long this stuff lasts. I feel that if you are building a plotter you are going to run it until it breaks anyway.

p.s. Last thing, get some heatsinks for those things and a bit of airflow running over them! These drives can work sustained, but only if properly cooled (which isn’t a normal desktop load).

rkalla · May 4, 2021, 4:29pm

Damn… that screenshot illustrates your “TBW isn’t everything” point pretty vividly - that’s interesting. If it keeps trending linearly that will be VERY interesting.

Good tip on headsinks and cooling - I’ll need to take that more seriously.

Voodoo · May 4, 2021, 4:44pm

Nice, I was looking for a graph like that but somehow I couldn’t find a lot on the sustained write speeds.
This is very helpful.
Do you happen to know any other sources that compare other ssd’s like this?

In my first build I was too focused on the high numbers of pcie gen 4, but actually it really doesn’t seem to matter much if you use gen 3 or 4, the sustained speed is internal to the drive regardless of the protocol.

Quindor · May 4, 2021, 4:47pm

Tom’s hardware generally includes these graphs, but techpowerup also does include them, here is for the 980 Pro 1TB again:

rkalla · May 4, 2021, 4:50pm

This actually helped make me feel better because I’m running them with a CPU that doesn’t support Gen4 speeds… so it looks like I’ll mostly be hovering around 75% of Gen3 speeds.

Thx for the heads up on Tom’s providing this typically.

JustinLloyd · May 4, 2021, 6:18pm

If you are not running Gen4, the 970 Pro might be a better option. Lower speed overall than the 980 Pro. It’s pure MLC so no cliff. In fact, even if I was picking a Gen4 motherboard, if I were buying consumer drives purely to plot, I’d get the 970 Pro.

alphatio · May 4, 2021, 6:27pm

HDD sold in USA is cheaper than those in the manufacturing country!
It’s shipped across the World and so funny that it’s still cheaper more than 20%, with a lot more option to choose from.

JustinLloyd · May 4, 2021, 6:51pm

Taxes, province-to-province tariffs, local channel supply pressure, competitors, customer acquisition costs, customer retention costs, warranties, strength of consumer protection laws, and also, plain human greed. Every little piece contributes to the overall price in your geographic region.

alphatio · May 4, 2021, 7:15pm

I build another plotting rig using external SSD. Based on my reading, it should able to sustain similar speed using USB3.2 gen2x2.
The good thing is there is no TBW in warranty term! and it’s 5 year warranty.