Currently plotting and farming Chia from 1 dedicated rig.
I’m potentially planning to expand my setup in an effort to continue plotting provided I can get my hands on HDD space.
If I want to expand to potentially 0.5 - 1 PB storage what would be the best option available?
Buy multiple USB hub’s and connect all the USB’s? I’m concerned about the USB’s falling asleep and also the amount of power consumption, wires, cables and etc…which might become a huge pile of mess once we reach around 10 - 20+ units of varying sizes. Moreover, I believe there is also a limitation to the amount of USB’s I would be able to connect to my existing PC.
I have 0 experience with using a SERVER/DAS/NAS etc… or anything beyond a consumer grade laptop/PC I’m a huge noob but I would like to look into this option because from what I understand refurbished devices could help reduce my overall cost but I have 0 experience with and keen to learn. But is latency going to become an issue? Also, how would I integrate this with my existing system? If I buy a refurbished server do I need to buy any additional adapters? Components and etc…is it just a plug and play and how would I get it setup?
Any other ideas for expansion possibilities?
I basically would like to invest in a device which just operates as huge hard disk
As Alternative expansion option Wasabi/crowdstorage/backblaze
But outside of that
Getting 1pb at home with not much experience definitly might be difficult(and possibly expensive power costs wise, hardware and maybe getting an electrician to make a new outlet which can provide the required amps without blowing a fuse), also invest in a good ups, a power failure may break all you file systems
For the operate as one huge disk option, check out union file systems they combine multiple independent Filesystems into one folder basically
They also work across networks if you mount the other systems (in the case for multiple servers)
At that size I would also recommend going for 10gbit ethernet
Be very careful investing in something that treats a bunch of disks as one disk. Using raid 0 will cause you to lose ALL of your plots on ALL of the disks if one fails.
At the very least you are going to be looking at expanding your current computer using RAID cards flashed to IT mode. That will let you hook up a lot of hard drives and keep speeds reasonable.
This 100x… any RAID array will be a waste of either space by using up for parity or a risk of losing large amounts of data to a drive in your array crashing… or both. If you use a RAID card, just set it up in IT mode and treat each drive as it’s own drive.
I should clarify when I meant 1 big hard disk I was referring to JBOD’s and not implying the RAID 0 or any RAID over the external HDD’s, this was my mistake.
Thank you for the suggestions and I was going through some of the cloud storage options and honestly at this point in time based on the Chia price and etc the cloud storage doesn’t seem viable personally from a cost perspective.
I’m going to look into the server option and might start off with a refurbished rack and see what happens from there. I’m not planning on scaling up to 0.5 pb immediately also, since this is just for farming I should be fine with Windows Server right? Or would there be any added benefit for farming on other server operating systems?
Not to say these are the same things but lol I just started Ubuntu last week for the first time in my life and I was introduced to Python few weeks ago.
Never typed in anything in Terminal/CMD in my life, already went through 1-ram failure (defect), multiple OS installations so yeah I’ve definitely seen my share of pain which is still nowhere close to the shitshow these things can become.
While it is overwhelming at times I try to keep at it, hopefully it works out
If you have rack space then l’d go for Dell R720XD (12 drives 3.5 config) or Dell T620 (go for same 12 drive config) if you don’t have a rack or a space or you want quieter operation.
You can get 6TB refurb/open box drives for those units about £90-105+VAT here. 6TB seems the best TB per buck.
As for setting them up. I’ll be 2x6TB pairing so they are fast enough to copy plots on to quick but not all eggs in one basket. 72TB Per 2U or Tower.
Run a Linux distro like Ubuntu if you are comfortable. Ubuntu Studio 21 seems pretty user friendly.