I am undecided at the moment on how to develop my home Chia farm.
Right now I am farming and plotting on a single PC, but I have rapidly exhausted all USB ports, internal PCIe slots and sata ports.
I am seeing two possible development ways:
- Purchasing a NAS and expanding by adding additional drives and eventually additional NAS as required
- Piling up Raspberry Pis with a USB hub and several USB drives
What do you think? Are there better solutions?
I’m going to homebrew a JBOD utilizing my server power supply and breakout board from GPU mining. I can also use the PCI-E risers to power the HBA cards so I don’t need a MB or anything else.
NAS is imo unnecessarily expensive. Cheaper option is using usb (jbod) drive bays, about 300 for an 8 bay enclosure. Think thats a better prive than you can get a 8 bay NAS. It need a host comuter though, but you can hook up many to one pc.
The cheapest option I think is using old servers with a lot of drive bays, or sas JBOD enclosures hooked up to a hba card with external sas connectors. but this is a little bit less easy to source and setup than an usb external bay
edit, yes DIY is always cheapest
I’m on the NAS route… I have 4 that are just about full, they run harvesters directly… 5th one is empty but not for long. Planning to move this solo farm (400TB) to a data center this week (already got my cabinet set up with shelves & a firewall)… Then I’ll do some a small pooled farm from home with a collection of external drives on usb.
8 slots for $300? You can get EMC Jbods on eBay $500 easy, 15 slots. Cheaper sometimes, but not always with rails and drive sleds.
There’s some very good advice on this topic at Chia Decentral:
I’ve been working on this same project for the last month or so! I’m almost finished 3d printing my “v2” JBOD hangers/slides so the drives can slide in long-ways from the sides of these wire racks. I’ll be able to fit 30 drives per shelf - in the picture you can see 20 drives per shelf.
Show off your rigs! - Chia Plotting - Chia Forum
Sneak peak of v2 - you can just pull the drive straight out:
A NAS will still be usefull should you decide to stop Chia farming.
For ‘long haul’ Chia, you want to focus on the system that will give you the least headaches and the least maintenance. This should be set and forget. You preferably want a system that will be able to re-plot failed drives (yes, you will have those), an OS that mostly takes care of itself, and some decent monitoring.
You want the least components possible, preferably just a single machine farming all your storage. Anything you have more than one of will just give you headaches.
People are over-obsessed with cost and power. Penny wise, pound foolish. You are counting on Chia hitting the majors as all other scenarios can’t ever make your money back. So don’t get stupid on brick-a-brack complex balls of mud. Keeping systems running smooth for years is not evident. Keep it simple. For re-plotting a replacement drive you are not in a rush. We passed the time that that mattered many Exabytes ago. So a basic value machine will do.
That’s awesome! I’m doing close to the same with my wire GPU rack
I think I’m going to do a frame setup instead of hanging. Hanging is probably great for isolation vibration though…
If you’re going to spend $300 for an 8 bay enclosure I’d suggest looking at the Fractal case that will hold 18 hdd’s. Can be had for $200, but you have to buy the extra trays to be able to add all the drives. Pick up a MB that will support an hba card to attach 16 drives internally. Can be used as a good base of operations to add additional storage. My plotter is a separate machine.
I have 3x SAS Expander, each supporting 32xSAS drives - each in a plywood DIY enclosure - so 96 drives - i’ll have 2xH310 HBA in the farmer and use 3 outputs to the SAS expander and 1 connector to 4 internal disks. A single Farmer - Ryzen 3600 running Ubuntu - should get away with 8GB but i’l likley run with 32GB for other uses
A single 1200W HP Server PSU + a handful of DC->DC converters for 5v (not sure how many i need yet until i measure power usage) to power all 3 enclosures. The SAS Expanders will be running off old mining 16 pin/USB bases. I’ll try to power them down partially, but not spin down fully - at least not until i know if a 7 second response is fast enough to respond to requests (unsure at this time as they are warning of a 5 second target)
This will take me up to 290+TB with 3TB drives or higher with 4+ TB drives
Yeah it is a bit Heath Robinson but it is cheap and “fun”
I have a few older Rack servers that i will be using with 10K SAS to replace my plots with poolable plots and retire the SSD whilst they are still alive
Watched an interesting YT video today where they mentioned there was some talk of the low response time threshold being to allow more time for MEV
Wow…plenty of nice ideas
Thinking aloud… Mining motherboard (thinking about the ASUS B250 Mining Expert with 21 pcie slots), with the appropriate usb risers, each one with a SAS card with 2 outputs (8 SATA drives per card)
That would make 21*8 = 168 SATA drives connected to the same motherboard
Do you think it could work?
I don’t know about the particular mining card but i would GUESS they are PCIE3-1 lane - about 1000MB/s - i think trying to get 8 drives on that is asking for a perfect star alignment - i would test with 4xSATA per card first and ensure the cards running at PCIE3
How are you going to power those SATA disks . A typical disk will draw 0.5a-1a off the 5v line (and something similar off 12v) . Most PSU are really concentrating their outputs at 12v - a typical 600-800w PSU will only output 15-20a regardless of its oveall rating which caps you at 15-20 drives depending on the quality of the PSU (5v is needed elsewhere as well and you have to allow for surge currents) To go further you have to worry about supplementing the 5v line or using multiple PSU
Also, depending on your CPU - i would expect to max out in the 30-40 plots/day range - so many disks are just completely unnecessary - unless you are just planning to use it for farming and not plotting in which case neither CPU or overall throughput of the disks should be an issue
Good luck with the wiring of that one
I am planning to use the setup only for farming. I believe that plotting needs a dedicated machine
I was planning to test the rig with an Asrock H110 BTC (I used this mobo on other GPU rigs). It’s similar to the ASUS but it has only 13 PCIe slots.
As you say, there is the danger of the PCIE3-1 lane becoming a bottleneck, so I will progressively increase the amount of hard disks per port.
I was thinking about using additional PSUs, separated from the main motherboard. Thank you for pointing out that Hard Disks draw power also on the 5v. I checked the specs of the PSUs that I have at home and effectively the main power output is on the 12v lines. I will have to pay attention to that.
For the wiring… it’s going effectively to be a challenge (probably not as challenging as wiring up GPU rigs as you have to be extremely careful about how you distribute power).
I am just going to get in a few DC->DC Buck converters - they are £10-12 for 20A but these are “chinese” ratings with a total dissipation limit of 1-200w You can use several conventional PC PSU (linking 1 pin in the MB connector means they will power up) but i doubt they will be any better, cost more and much bulkier - i have picked up a couple of basic Ammeter modules (used for radio control) and i will have 1 on each DIY enclosure to keep one eye on current draw and make sure the modules run cool enough with my cheapo infra red thermometer
You have to be happy to blow yourself up though … lol
I have one of these to test - a STEP DOWN only - cheaper and more efficient than a step up/down)
These look interesting - you may not need a server PSU as they are switched mode that you run off mains - just make sure you get one well over spec for your needs - in my case I am looking at a 5v, 40A-50A which should be ok for 32xSAS drives quoted at 0.7A each (so around 20A) - the actual draw will peak higher
You can also buy 12v modules but they are not quite so desirable if you have to have server PSU around with connectors already. If i didn’t i would consider a 12v/5v pairs suitably over-sized for each DIY enclosure. I would expect a good server PSU to be more efficient and better quality (but only 12v output) as they are built to last forever so i will be sticking to the server PSU for 12v - i have just not decided between 2x20A 5V Step Down Buck (you can use them if parallel if they are carefully adjusted to match outputs) running off the server PSU or 1x40-50A switched power supply as above
Are you sharing the file for these? Great idea
Thanks! I haven’t quite finished the design for the backplane hanger but very close now.
Here’s another preview. This is 2 shelves’ worth of hangers printed and installed along with the first several drives. Should fit 60 drives! I’ll be moving over all the rest of them from my main rig once I finish the backplane.
Nice! What material are you printing in?
I’m planning to do something similar with hanging disks - and was going to 3D print brackets, but then discovered that the ‘vents’ on these rack shelves are spaced pretty perfectly for bolting disks directly:
I’m just using PLA+ (has a little higher melting point, but not much). I don’t imagine I’ll have much problems with heat - I’ve been watching these drives for the last month and they seem to average 35-40° C so I’m not to worried about melting plastic since PLA won’t start getting soft until ~100°. I’ll also have fans blowing on them up from the bottom.