Hi building my first plotting rig and wanted to use old server parts. Can someone give me some advice on how these specs look.
CPU: Intel XEON E5-2697 V4 ( 18 cores, 36 threads)
Motherboard: GIGABYTE GA-X99-UD4P
Ram: 128GB DDR4-2133
Temp Drive: 4x Intel SSD S4500 1.92 TB
Final Drive: 4x WD 18TB HDD
Will I be able to plot 32 plots in parallel? 8 Per SSD and Final Drive? Should I try to find a P4510 instead of the S4500? Will the plot speed be greatly bottlenecked because the S4500 uses sata?
Hey there, welcome! Have you read the Chia Github wiki? A bunch of your questions are answered there: Home · Chia-Network/chia-blockchain Wiki · GitHub
For example, in the Wiki you’ll find you need 256GB of temp storage per plot. So you will only be able to plot 7 at a time on each SSD.
I suggest you get a small cheap SSD for your operating system to run on. Reserve the temp drives for the plotting.
Yeah I briefly looked but I was curious about the read and write speed of Sata vs u.2. Will it hinder the plot speed? Can sata even do 7 at a time? Also can I get 8 per SSD if I stagger them?
I really urge you to do more than briefly read the Wiki. There’s a lot of useful info in there.
Yeah it’s a bit frustrating since there are no quick answers.
It depends @nwincoin - you will need to do some solid research and experimenting to get the best setup for your situation. If there was an easy solution, people would be selling it pre-packaged already.
Like on an actual farm, you don’t grow anything without breaking a sweat! 
I too use Broadwell cores and SATA in addition to NVMe (whose bandwidth never gets saturated). 2TB of temp space per SATA lane is exactly the right spot to keep it saturated on a Broadwell machine! Load doesn’t require U.2 NVMe unless you have more than 2TB in one piece. I also see P4510 having lower PBW endurance than an S4500, though U.2 interface may have a better resale value.
I found performance slightly diminishing beyond 1.6 simultaneous plots per core, so I’d expect max throughput to be at 29 parallel plots, but 32 will work too, using about all memory you have at default settings.
You can fit more jobs than you appear to have a space for with 50-90 minute staggering (needs careful tuning but you’re probably good to start with 1 hour, should you run out of space just delete a job, the rest will resume). The quarter terabyte per job isn’t always occupied, usage growing up to that size in phases 1 and 2 and decreasing after that. Dividing number of jobs by 5 gives about the space I see occupied at any given time.
What’s your average time per plot, 18 hours?
17 hours per plot on average. About exactly 90 i.e. 9TB a day with 40 cores.
I use LRDIMM memory which trades higher density for extra CAS latency. Things are slightly faster with RDIMM and yet faster with UDIMM but gains are under an hour so I don’t care.
Wow thanks for all the replies guys, I hit a hiccup on the delivery of my cpu so I’m still waiting on it.
@roybot for sure, Tons of great info there but I didn’t exactly see what I was looking for but maybe I’ll take a day to go through it all more
Thoroughly.
@cultiv yeah I love your approach, once I get all the parts in, I am Excited to experiment and contribute more to the forum on my findings.
@login-taken your answer was what I was looking for, thanks for sharing. I’m gonna experiment once I get the rig all built, but I did buy some u.2 just in case.
Don’t update BIOS beyond F22 version if you can. Disabling Spectre/Meltdown mitigations saved me some hours.
In Windows it’s complicated, needs a special driver to disable microcode update by the OS, then a script, test with InSpectre or MDS-tool. In Linux there’s just a kernel flag mitigations=off.
BIOS microcode summary, v.23c includes mitigations, v.22 dates from before vulnerability exposure; your CPUID is 406f01
╔═══════════════════════════════════════════╗
║ MC Extractor v1.53.0 r186 ║
╚═══════════════════════════════════════════╝
X99UD4P.F20 (1/3)
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║ Intel ║
╟───┬───────┬────────────────────┬──────────┬────────────┬──────┬────────┬──────────┬──────╢
║ # │ CPUID │ Platform ID │ Revision │ Date │ Type │ Size │ Offset │ Last ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 1 │ 406F1 │ EF (0,1,2,3,5,6,7) │ B000010 │ 2015-12-03 │ PRD │ 0x5000 │ 0xEDEFE0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 2 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 14 │ 2015-07-02 │ PRD │ 0x7C00 │ 0xEE3FE0 │ Yes ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 3 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 10 │ 2015-05-01 │ PRD │ 0x6800 │ 0xEEBBE0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 4 │ 306F2 │ 6F (0,1,2,3,5,6) │ 29 │ 2014-09-03 │ PRD │ 0x7000 │ 0xEF23E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 5 │ 306F1 │ EF (0,1,2,3,5,6,7) │ 80000013 │ 2013-10-02 │ PRE │ 0x8800 │ 0xEF93E0 │ No ║
╚═══╧═══════╧════════════════════╧══════════╧════════════╧══════╧════════╧══════════╧══════╝
X99UD4P.F22 (2/3)
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║ Intel ║
╟───┬───────┬────────────────────┬──────────┬────────────┬──────┬────────┬──────────┬──────╢
║ # │ CPUID │ Platform ID │ Revision │ Date │ Type │ Size │ Offset │ Last ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 1 │ 306F2 │ 6F (0,1,2,3,5,6) │ 2D │ 2014-11-21 │ PRD │ 0x7000 │ 0xD215C0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 2 │ 306F1 │ EF (0,1,2,3,5,6,7) │ 80000013 │ 2013-10-02 │ PRE │ 0x8800 │ 0xD285C0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 3 │ 406F1 │ EF (0,1,2,3,5,6,7) │ B00001D │ 2016-06-06 │ PRD │ 0x6400 │ 0xD30DC0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 4 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 10 │ 2015-05-01 │ PRD │ 0x6800 │ 0xD371C0 │ No ║
╚═══╧═══════╧════════════════════╧══════════╧════════════╧══════╧════════╧══════════╧══════╝
X99UD4P.23c (3/3)
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║ Intel ║
╟───┬───────┬────────────────────┬──────────┬────────────┬──────┬────────┬──────────┬──────╢
║ # │ CPUID │ Platform ID │ Revision │ Date │ Type │ Size │ Offset │ Last ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 1 │ 306F2 │ 6F (0,1,2,3,5,6) │ 3D │ 2018-04-20 │ PRD │ 0x8400 │ 0xD215E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 2 │ 306F1 │ EF (0,1,2,3,5,6,7) │ 80000013 │ 2013-10-02 │ PRE │ 0x8800 │ 0xD299E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 3 │ 406F1 │ EF (0,1,2,3,5,6,7) │ B00002E │ 2018-04-19 │ PRD │ 0x7000 │ 0xD321E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 4 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 10 │ 2015-05-01 │ PRD │ 0x6800 │ 0xD391E0 │ No ║
╚═══╧═══════╧════════════════════╧══════════╧════════════╧══════╧════════╧══════════╧══════╝
Press enter to exit
Oooh this is interesting. Which specific BIOS settings?
I didn’t do it by toggling a setting, as architecture release predates the problem. The BIOS updates contain new microcode, it has to be downgraded.
For newer computers performance impact is more negligible. There rarely is a setting, and just for AMD it seems, due to their handling of STIBP mitigation. HP call this setting Hardware enabled Spectre Variant 2 Mitigation which is not entirely true. Seems it can also be disabled in Linux with a kernel mitigations=off parameter, but not in Windows.