Hey - you either stepped out of the wrong door of the Tardis - or you are in the wrong thread … lol
Thank you fore this amazing reply. I think after reading what you’ve said about the old Xeons, I thought I might look around a bit for something newer, like a ryzen 3900. the xeons are just too old in my area.
If you are looking at Ryzen, get the right motherboard - Look at something LIKE the Aorus B450 AORUS PRO - with 2xM.2 NVME slots and 2xPCIe 16x slots you have plenty of options - I got mine off Amazon warehouse. The 3600, 3700X, 3900X are great without breaking the bank -higher spec parts are great but the costs escalates very quickly.
I reckon …
2x1TB Sabrent (8 parallel) would run great on a Ryzen 5 3600/3700X/32GB,
3x1TB NVME (12 parallel) great on 3700X/48GB
2x2TB NVME (16-18 parallel) the 3900X/64GB
… above that the price escalates quickly and part availability becomes an issue
All of these are more expensive than a pair of Xeon - if you can get the Xeon at the right price
Anything predating Nehalem on the Intel side is definitely too old as it will nt support bitfield in Chia
Thanks Ianj, again!
On this, would the 2x1TB sabrent run in RAID0?
2x1TB Sabrent (8 parallel) would run great on a Ryzen 5 3600/3700X/32GB <— I am leaning strongly towards this
Corvin
Yeah 1x1TB RAID 0 (software raid) - am not sure if it is faster but it gives that little extra space to complete 8 in parallel (vs 2x4). I haven’t done the space calculation exactly but there is usually plenty of space. I wouldn’t try and push 9 parallel plots because it might overflow unless you have perfect straddle. I have a 3rd Sabrent 1TB but have sufficient throughput to fill my disks in about a week so will keep that one as pristine - there is loads of CPU headroom on any of the 3600/3700X/3900X for 8 parallel but i might look up to 3700X for 3x1TB. I happened to have the 3900X for another project so i used that. chia plotting is quite CPU intensive so the 3900X might be faster by virtue of its speed - i was just looking at when the CPU might be too slow
Wow you really know your Xeons, would you be able to help me out with what I could expect on my system? Its a Dell r720 with 2x 2670v2 (10c/20t each), 96GB ram, and 8x 900GB 10K SAS drives.
I assume i can plot 1 on each SAS drive with 4 threads, but im just wondering when I would kick off the second plot on each drive? Would it be just after phase 1…or will this just slow me down and I should stick to completing 1 per drive. As its a 900GB drive and Phase 1 is the most intensive I was hoping I could do 2 in parallel per drive and drive kick out close to 24/day around.
If it is to 8x10K SAS drives then you should only make ONE parallel plot to each drive - so 8 parallel plots - the 2x2670v2 CPU can easily do that, almost regardless of how many in Phase 1 (when the Threads are used) - you mainly need to minimise the bottleneck writing to the destination. I am acquiring a handful of SAS 10K myself soon in order to release my more expensive SSD (I am 75% of way through my disks - don’t intend on getting more at this time - i just want to replot my collection to make it pool compatible) so i might be able to comment more specifically but 4 threads would seem appropriate.
I am guessing the plots will take 13-14 hours each so stagger them appropriately (say 80-90 mins)
If you get the chance to add more SAS disks those CPU/memory should be good for 20(easily) -25+ (diminishing returns) parallel plots
Thanks @ianj ! This is one of the biggest questions I’ve had and havent been able to get a solid answer.
So to confirm my understanding on a 10k SAS drive I can plot multiple as long as the previous one is well out of phase 1? And as long as I’m not overloading the destination drive so making sure no plots (across the drives) start within whatever the copy time is (say 20 minutes).
e.g Plot 1 per drive with a 90 min stagger, along with making sure each drive is staggered by 20 mins to offset copying time.
What would you say the max I could have going per 900gb drive is? 2 with a stagger or could I push to 3 with the 90min stagger.
Yeah this is the reason im sticking with the SAS drives, my desktop has a 2TB nvme in it but I dont want to completely use it up but at the same time want to get this system set up so that its ready to plot for pools.
And in saying that i didn’t realize this system could hit 20 plots in parallel easily without diminishing returns!! Im really underutilizing this then by only doing it. Ive been thinking about getting a DAS (as my r720 only has 8x3.5" bays) for storage but i suppose I’ll have to rethink that and use some of that space for more 10k drives in that case. If i could hit 40 plots a day without too much thinking that would be awesome, with a stagger assume it would be a little more.
1 parallel per temporary drive - DO NOT OVERLAY parallel plots on a single drive - DO NOT set them up in RAID 0 in the hope of getting more - it doesn’t help much - 1 plot per temporary drive at any 1 time - you can target multiple plots per TARGET drive as long as they are staggered enough to let one complete before the next finishes
To get 40 plots a day without SSD would require upwards of 24+ SAS hard drives as the plot time would roll off as you ran more in parallel - they would NEVER drop below 12h and you could only ever run about 30 in parallel because of the memory use
You can’t just scale up linearly as various resources will slow you on the way
1 Like
Alright thank you for clearing that up im on board the train now, so I can just use a plotting manager to organize this then by telling it only ONE ACTIVE PLOT per drive and only start the next when its done ALSO make sure that no two plots start within 20 minutes of each other (taking into account copying to destination).
To get 40 plots a day without SSD would require upwards of 24+ SAS hard drives as the plot time would roll off as you ran more in parallel - they would NEVER drop below 12h and you could only ever run about 30 in parallel because of the memory use
Gotcha so depending on whether my DAS is 12x3.5" drives or 24x2.5" drives I can leverage around 19/20 of the drives to do the plotting and use the remainder as storage. I suppose then in this case having 900GB drives is a bit of a waste and 600GB’s would have be just fine if purchasing drives.
Can I also do less in parallel, say 15-18 in parallel and also farm via the same machine?
Clearly you are in the UK buying drives from systemsuppliesindusties … lol
My thoughts were similar - the advantage of the 900GB is that you can top them off when you are finished plotting - more efficient than 600GB filling all the slots. The problem with an enclosure plotting upwards of 24 drives might be aggregate throughput - not a problem for farming but plotting am not so sure you would scale so easily - again staggering would be crucial to mitigate concurrent temporary CONTROLLER congestion
hahah i am in the UK but not heard of systemssuppliesindustries before. Ive got a = server already with the 900gb drives from earlier. And the DAS is via my employer who doesnt mind selling some old equipment vs recycling but just havent decided between the 2.5 or 3.5 yet enclosure yet. Need to take into account what models they want to get rid of and noise/power usage. But as far as extra drive I’ll probably go through a refurbisher in the UK, I know of a few.
My thoughts were similar - the advantage of the 900GB is that you can top them off when you are finished plotting - more efficient than 600GB filling all the slots
You know what thats a really good point, will stick with 900GB’s then also whenever i do purchase. The price difference also makes this a non-issue as the 10k 900GB drives are £15/each so a bit of a no brainer.
The problem with an enclosure plotting upwards of 24 drives might be aggregate throughput
Perhaps you’re right there, having many on the enclosure would saturate the connection quite quickly. Guess its something only testing can confirm but with a large enough stagger hopefully it would cut down the congestion.
Hello ianj.
I too have purchased a second-hand HP Proliant DL380P with 2 Xeon E5 2.7 GHz processors, 384 GB RAM and 25 x 10K SAS 600 GB HDD.
I am using Windows 10 Pro.
Do you know how many plots in parallel could I get a day without using NVMe or SSD?
Thank you.
I’m thinking ram is your limiting factor.
In all seriousness, which E5 do you have? The E5-2680 with a base of 2.7?
Yeah you should be able to pull off 20 in parallel easy, with two of those. Staggered of course. Would expect between 30 and 40 a day. Plot to each SAS HDD you have and you will be golden. Not a lot of IO slowdown.
1 Like
At 20 in parallel you will require decent CPU - passmark is a good indication
I have an 2xE5-2670v1 doing 20 in parallel - about 13-14h (around 33 a day) using 2xioDrive - but it is flat out at 70+% CPU - 2x2670v2 is still affordable but about 25% faster - so 35-40 is in reach but again flat out - 20 in parallel really need 64GB - the extra RAM won’t do any harm as it will buffer disk writes somewhat
As long as you plot 1 per drive and use SAS 10-15k you should not be far behind
1 Like
Nice, my questions were really around how to plot with xeons with an SAS and you perfectly answered. I’m using what I have so already, got the 2670v2 with 96gb ram in the garage. Can upgrade the ram if needed but I think first is getting more then 8 SAS drives going in another enclosure.
The passmark scores are a great, i didnt realize they correlated like that, its good to know that 35-40 is in reach. Guess I’m just limited right now by my number of drives…and i suppose 96GB ram is good for 30 plots or so.
Hi.
Thanks for the reply.
I have 2 of these :
Still in the process of installing and updating Win 10 Pro drivers.
Will test and probably come back with more questions.
I have an super old Intel Xeon E5-2420 @ 1.90GHz with a passmark score of an impressive 5176 ! and the server has 16GB memory.
The only thing i upgraded is a temp 2TB NVME SSD and a 480GB sata temp2 drive.
I can plot 8 per day at the moment, and i hope with some further tuning i can get it up to 10 
CPU is a real bottleneck, with two parallel plots in phase 1 with 90mins stagger it takes about 5.5 hours for phase one, with 3 plots in phase 1 it takes about 7+ hours to complete phase 1. I hope to find a good sweetspot with max TiB output per day.
Curious, how do you get to 8 plots per day with those kind of times per plot?
Are you just running 8 at a time and they finish in a day?
I just got a HP Z230 with E3-1246v3 and started to try and figure out how to best run it.
My plan is actually to just plot direct to hdd on that one.
I tried two plots last night. 1 on and external USB, 1 on a 2.5" Hitachi 500GB
USB took 34000 seconds, hdd 54000 second but the hdd plot was running with bitfield disabled because it said in the docs that this might be good for plotting to hdd. Right now they are running again both with bitfield enabled.
I would like to run 6 plots on it at a time, but I’m having a hard time finding memory for it at a good price. Plenty of registered ddr3 out there but unregistered…not so much