Wow, very impressive.
What’s your typical HDD price per TB for your current rig? <20? 25/TB? It’s definitely a big farm and budget will be a factor how much HDDs you can add.
Wow, very impressive.
Yes I have. You can get a Cougar or EVGA or Be Quiet or Phantex 850w and 80+ rated PSU for around $100. You would only ever need 2 of them + some molex to SATA adapters and some SATA splitter cables.
Your converters cost half that price of a PSU, but anyone looking to replicate this could need more than 2 to do it properly and IMO safely. In your picture, the PSUs show 75W. Also, the PSU you linked to is a DELTA/HP server PSU. The cost isn’t worth it at $155 used on Amazon.
Thank you! I use 4tb enterprise drive and I get them for about $10/tb. Actually my last order was about $12/tb including shipping.
For sure! When I ordered 2 of them just earlier this week (Monday I believe) they were $55/each:
I got my first one on eBay for around the same price. It looks like the price probably fluctuates on Amazon depending on stock, but if you keep watch you should find them for less than $60. Also the PSU isn’t 75w, you may have meant the 12v-5v converter? Those are probably overpowered actually, they each can handle 75w of 5v. Each drive takes less than a
watt amp of 5v, so plenty of overhead there! One worry I always have with “normal” ATX power supplies is the 5v rail. A lot of them are underpowered for this type of application, so making my own gives me plenty of overhead!
That wasn’t the PSU, thats the DC-DC converter for the 5V rail.
Oh, you’ll need more than that for 100s for drives. Even with the 1200w PSU he’d be only able to power 80 drives safely.
WARNING BRO MATH ALERT (Read with caution):
Seagate EXOS x16 Power Requirements (according to label):
12V x 0.72A = 8.64W
5V x 0.9A = 4.5W
8.64 + 4.5 = 13.14W (Per drive)
1200W / 13.14W = 91.3 drives per PSU.
Add in a 10% margin for safety/efficiency: 82.2 Drives per 1200W PSU
And I don’t think these calculations take into consideration the power spikes created at spin-up.
Earlier dude mentioned in OG post
Yeah, this sounds about right, I would go with industrial power supplies for something like this.
Then again, there’s more than one way to skin a cat. If you can get ATX power supplies for cheap or have them lying around, there’s no reason not to use them.
One quick addition that I think I mentioned earlier - I should stop saying “1200w” because even though it is rated for 1200w, that’s if you are using 220v instead of 110v. That’s another benefit of server PSUs actually - they can use 220v if you have it available. I do not, unfortunately, at least not yet!
At 110v, it is only rated at 900w max.
I’m being a total nerd and looking up the specs for the Phanteks 850W 80+ PSU and surprisingly the 5V rail only supplies 20A. I remember some people mentioned they ran out of headroom on the 5V rail but now I see why. This would power 22 drives max.
- Phanteks, 5V, 20A (22 drives)
- EVGA, 5V, 24A max. (26 drives)
- BeQuiet, 5V, 25A max. (27 drives)
It would appear that with ATX psu’s the biggest limiting factor is actually the 5V rail and not the overall wattage.
This is exactly my experience! To be fair - there is nothing stopping you from using the same 12v-5v converters on those ATX psu’s and getting a little more 5v juice out of the 12v rail.
What would be the best way to connect multiple ATX PSUs to power a large number of drives. Should all the PSU grounds(DC negatives) be connected? I am just thinking what would be needed to run like 100 of such drives in a similar setup but with 20 drives attached per PSU → total 5 PSUs
Although that would probably work, I wouldn’t trust normal ATX power supplies to share current correctly unless they specifically mention that as a feature. This is yet another benefit of server PSUs - they assume they are just one of at least two because of the hot-swappable requirement. So server PSUs have a special pin that they share so they can correctly balance and share the load.
Probably better just to leave them independent of each other. You can wire them so they all turn on at the same time at least, using something like this: Amazon.com: 24Pin (20+4) Dual PSU Power Supply Cable Splitter Adapter 1 Feet 18AWG for ATX Motherboard Extension Kit: Industrial & Scientific Although you might not want them to all turn on at once - see my posts earlier in this thread about spin-up power on experiences.
@enderTown Did the 12 port sata connector work out? I noticed that its a pcie 1x(which severely limits the bandwidth). Were you able to successfully farm? And how are you transferring plots to the drives? Is the server they are connected to plotting or are you manually transferring drives?. I had to manually transfer the drives for my setup as usb2 was very slow for copying over drives, but good enough for farming.(Currently running 48 drives over usb2)
I hope to test it later today assuming my extra RAM arrives in the mail! I ordered 128gb RAM for the server and I’d rather only turn it off and disassemble it once to add the RAM and the 12 port card lol.
Two interesting things: even though the description says 1x, the card actually looks like it fits a 4x slot. It could still be 1x, but that would certainly be weird when they could just save money by using a 1x connector instead of a 4x. So I’m definitely interested to plug it in and see if the description maybe was just a mistake? Here’s a pic - if it was just 1x, it wouldn’t have the extra connections:
Secondly, I was totally right in that other thread - look at those 4 empty spots for more ports! This could have been a SIXTEEN port card!! Maybe they offer it but I couldn’t find it anywhere…
Lastly, I think you’d be pleasantly surprised with even 1x PCI-e speeds. Here is a screenshot showing my disk throughput while copying 3 plots to 3 hard drives through a four port 1x PCI-e SATA card - not only that, all 3 hard drives are on the same port multiplier so they are all sharing the same SATA channel on that 1x card:
That is plenty fast to copy plots and farm!!
This is a breakout board made by parallel miner parallelminer.com - however SATA power requires BOTH a 12v connection for the disk motor AND 5v for data. PCIe 6pin connectors only have a 12v positive and ground but lack the 5v connection.
I’ve been in contact with the folks at ParallelMiner and it appears they’re working on a new breakout that will support custom cables that will support 10x sata devices per “connector” on the breakout. This is definitely possible since basically all HP / Dell / server PSU’s in general feature both a 5v and 12v rail. The question is if the 5v rail has enough current to power all the connected devices. In many cases the 5v rail in server PSU’s is only intended to power USB devices (which usually are never used or not used in more than 2-4 usb ports on the server itself).
That said, drop the ParallelMiner guys a line and let them know you’d like to see a Chia breakout board. If we continue to bug them this “prototype” they’ve mentioned might become something they actually sell.
Server power supplies generally only output 12v, then use a PSU backplane with voltage regulation onboard. Even in this thread, the OP is using 12v PSUs with 15A 5V buck converters, which is probably all parallelminer is going to do.
Thanks for the heads up, but I think this is something @enderTown would be more interested in.
Personally, I’m more interested in shucking a dozen usb HDDs and a raspberry pi into a wooden crate and leaving it on a shelf somewhere. That’s as about as sophisticated I’m gonna get with chia I think lol.
Industrial grade mega-farms aren’t really of interest to a small-time micro-farmer like me…
That being said I did see this in costco earlier today, if anyone in the UK is looking for a similar rack…:
But othewise, projects like the OP is more like hardware porn for nerds lol.
That’s cool, it would be nice to have a single breakout board for both 12v and 5v! But as @gryan315 says, all of the server PSUs I’ve seen only output 12v and then the 5v conversion happens somewhere else. So they’ll need to integrate a 5v rail into the breakout board - but that would save me a bit of wiring!
Getting something with caster wheels is the way to go! Mine didn’t come with any and I just added them, which was not fun. Pro-tip: Add them from the start instead of when the thing weighs 200kg lol
I’m up to 90 SATA drives all connected to a single farmer now. The bottom shelf is full with 50 disks. I’ve added 2 more 1200 watt server PSUs - now each shelf of 50 disks will each have their own 1200w PSU. Also, I’ve moved to the laundry room (as evidenced by the watchful eyes of my “fire suppression system” shown on the wall behind the rack ) where I have access to the 220v dryer socket, which now powers the 1200w server PSUs. I’ve also got a little portable AC unit in there to keep reasonable temps - it isn’t working very hard now, but will be soon when I move my ETH rigs in there.
How is the noise and vibration at 90 drives?
As for noise, I certainly can’t hear them over the 2 huge box fans (not pictured) blowing air straight into the front But with the laundry room door closed, I can only hear the low hum if I’m in the hallway outside the laundry room.
As for vibration, I don’t have a good way to measure scientifically, but the “feel test” still feels “good.” This is where I place one hand on one wire shelf and another hand on another wire shelf. I can feel very light unique vibrations in each shelf. To me, a not-scientist, this says that each shelf is doing a good job insulating the other from mutual vibrations. Any (cheap) ideas for better testing of this??
Here’s a better view of the 3d parts I designed so you can get a better look at the spring mechanism. Pardon the super old HDD model - at least the screw holes haven’t changed
The wire rack goes between the top and bottom piece so they are stackable.
Well, if the shelves aren’t rattling, and the entire rig isn’t dancing across the room I’d call it good.
The 3D printed dampeners look like they’re doing a good job too. I like the design.