Hi building my first plotting rig and wanted to use old server parts. Can someone give me some advice on how these specs look.
CPU: Intel XEON E5-2697 V4 ( 18 cores, 36 threads)
Motherboard: GIGABYTE GA-X99-UD4P
Ram: 128GB DDR4-2133
Temp Drive: 4x Intel SSD S4500 1.92 TB
Final Drive: 4x WD 18TB HDD
Will I be able to plot 32 plots in parallel? 8 Per SSD and Final Drive? Should I try to find a P4510 instead of the S4500? Will the plot speed be greatly bottlenecked because the S4500 uses sata?
Hey there, welcome! Have you read the Chia Github wiki? A bunch of your questions are answered there: Home · Chia-Network/chia-blockchain Wiki · GitHub
For example, in the Wiki you’ll find you need 256GB of temp storage per plot. So you will only be able to plot 7 at a time on each SSD.
I suggest you get a small cheap SSD for your operating system to run on. Reserve the temp drives for the plotting.
Yeah I briefly looked but I was curious about the read and write speed of Sata vs u.2. Will it hinder the plot speed? Can sata even do 7 at a time? Also can I get 8 per SSD if I stagger them?
I really urge you to do more than briefly read the Wiki. There’s a lot of useful info in there.
Yeah it’s a bit frustrating since there are no quick answers.
It depends @nwincoin - you will need to do some solid research and experimenting to get the best setup for your situation. If there was an easy solution, people would be selling it pre-packaged already.
Like on an actual farm, you don’t grow anything without breaking a sweat! 
I too use Broadwell cores and SATA in addition to NVMe (whose bandwidth never gets saturated). 2TB of temp space per SATA lane is exactly the right spot to keep it saturated on a Broadwell machine! Load doesn’t require U.2 NVMe unless you have more than 2TB in one piece. I also see P4510 having lower PBW endurance than an S4500, though U.2 interface may have a better resale value.
I found performance slightly diminishing beyond 1.6 simultaneous plots per core, so I’d expect max throughput to be at 29 parallel plots, but 32 will work too, using about all memory you have at default settings.
You can fit more jobs than you appear to have a space for with 50-90 minute staggering (needs careful tuning but you’re probably good to start with 1 hour, should you run out of space just delete a job, the rest will resume). The quarter terabyte per job isn’t always occupied, usage growing up to that size in phases 1 and 2 and decreasing after that. Dividing number of jobs by 5 gives about the space I see occupied at any given time.
What’s your average time per plot, 18 hours?
17 hours per plot on average. About exactly 90 i.e. 9TB a day with 40 cores.
I use LRDIMM memory which trades higher density for extra CAS latency. Things are slightly faster with RDIMM and yet faster with UDIMM but gains are under an hour so I don’t care.
Wow thanks for all the replies guys, I hit a hiccup on the delivery of my cpu so I’m still waiting on it.
@roybot for sure, Tons of great info there but I didn’t exactly see what I was looking for but maybe I’ll take a day to go through it all more
Thoroughly.
@cultiv yeah I love your approach, once I get all the parts in, I am Excited to experiment and contribute more to the forum on my findings.
@login-taken your answer was what I was looking for, thanks for sharing. I’m gonna experiment once I get the rig all built, but I did buy some u.2 just in case.
Don’t update BIOS beyond F22 version if you can. Disabling Spectre/Meltdown mitigations saved me some hours.
In Windows it’s complicated, needs a special driver to disable microcode update by the OS, then a script, test with InSpectre or MDS-tool. In Linux there’s just a kernel flag mitigations=off.
BIOS microcode summary, v.23c includes mitigations, v.22 dates from before vulnerability exposure; your CPUID is 406f01
╔═══════════════════════════════════════════╗
║ MC Extractor v1.53.0 r186 ║
╚═══════════════════════════════════════════╝
X99UD4P.F20 (1/3)
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║ Intel ║
╟───┬───────┬────────────────────┬──────────┬────────────┬──────┬────────┬──────────┬──────╢
║ # │ CPUID │ Platform ID │ Revision │ Date │ Type │ Size │ Offset │ Last ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 1 │ 406F1 │ EF (0,1,2,3,5,6,7) │ B000010 │ 2015-12-03 │ PRD │ 0x5000 │ 0xEDEFE0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 2 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 14 │ 2015-07-02 │ PRD │ 0x7C00 │ 0xEE3FE0 │ Yes ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 3 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 10 │ 2015-05-01 │ PRD │ 0x6800 │ 0xEEBBE0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 4 │ 306F2 │ 6F (0,1,2,3,5,6) │ 29 │ 2014-09-03 │ PRD │ 0x7000 │ 0xEF23E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 5 │ 306F1 │ EF (0,1,2,3,5,6,7) │ 80000013 │ 2013-10-02 │ PRE │ 0x8800 │ 0xEF93E0 │ No ║
╚═══╧═══════╧════════════════════╧══════════╧════════════╧══════╧════════╧══════════╧══════╝
X99UD4P.F22 (2/3)
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║ Intel ║
╟───┬───────┬────────────────────┬──────────┬────────────┬──────┬────────┬──────────┬──────╢
║ # │ CPUID │ Platform ID │ Revision │ Date │ Type │ Size │ Offset │ Last ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 1 │ 306F2 │ 6F (0,1,2,3,5,6) │ 2D │ 2014-11-21 │ PRD │ 0x7000 │ 0xD215C0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 2 │ 306F1 │ EF (0,1,2,3,5,6,7) │ 80000013 │ 2013-10-02 │ PRE │ 0x8800 │ 0xD285C0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 3 │ 406F1 │ EF (0,1,2,3,5,6,7) │ B00001D │ 2016-06-06 │ PRD │ 0x6400 │ 0xD30DC0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 4 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 10 │ 2015-05-01 │ PRD │ 0x6800 │ 0xD371C0 │ No ║
╚═══╧═══════╧════════════════════╧══════════╧════════════╧══════╧════════╧══════════╧══════╝
X99UD4P.23c (3/3)
╔══════════════════════════════════════════════════════════════════════════════════════════╗
║ Intel ║
╟───┬───────┬────────────────────┬──────────┬────────────┬──────┬────────┬──────────┬──────╢
║ # │ CPUID │ Platform ID │ Revision │ Date │ Type │ Size │ Offset │ Last ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 1 │ 306F2 │ 6F (0,1,2,3,5,6) │ 3D │ 2018-04-20 │ PRD │ 0x8400 │ 0xD215E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 2 │ 306F1 │ EF (0,1,2,3,5,6,7) │ 80000013 │ 2013-10-02 │ PRE │ 0x8800 │ 0xD299E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 3 │ 406F1 │ EF (0,1,2,3,5,6,7) │ B00002E │ 2018-04-19 │ PRD │ 0x7000 │ 0xD321E0 │ No ║
╟───┼───────┼────────────────────┼──────────┼────────────┼──────┼────────┼──────────┼──────╢
║ 4 │ 406F0 │ EF (0,1,2,3,5,6,7) │ 10 │ 2015-05-01 │ PRD │ 0x6800 │ 0xD391E0 │ No ║
╚═══╧═══════╧════════════════════╧══════════╧════════════╧══════╧════════╧══════════╧══════╝
Press enter to exit
Oooh this is interesting. Which specific BIOS settings?
I didn’t do it by toggling a setting, as architecture release predates the problem. The BIOS updates contain new microcode, it has to be downgraded.
For newer computers performance impact is more negligible. There rarely is a setting, and just for AMD it seems, due to their handling of STIBP mitigation. HP call this setting Hardware enabled Spectre Variant 2 Mitigation which is not entirely true. Seems it can also be disabled in Linux with a kernel mitigations=off parameter, but not in Windows.
TL/DR: Got a dual cpu motherboard with two Xeon E5-2686 v4, I want to max out the ram to 512gb (8x64gb) ECC DDR4 2400 ram. Question: I found a 8x64gb kit of ram, but it is LRDIMM. Can I use 512gb of LRDIMMs as opposed to UDIMMS or RDIMMS, or any any combination of the 3?
Hi everyone, complete beginner here (i’ve got about 12 plots up now and it’s taking forever).
I have been plotting (m1 mac 16gb 512gb, 1tb micron thunderbolt 3 SSD, two USB 3.0 hhds for plot storage) and downloading some purchased plots in frustration at how slow it was going for a couple weeks now, and while the downloaded ones come in nice and smooth with no problems, but the gui does not like plotting while there is so much downloading in the background. I often come home from work to a frozen computer, tones of time lost, or my node becoming out so sync for hours or days.
So, I was going to get an old computer just for harvesting and downloading, but I was to intrigued by some of this stuff popping up here from China (I live in Taiwan). and while searching I came across some motherboards that include dual xeon processors, three nvme slots, tons of pcie lanes, usb, sata, ect (X99 motherboard dual xeons). I thought that it could be quite interesting so I bought it online last night. Hopefully it will be here in a day or two. I chose the xeon E5-2686 v4 cpus (they only had certain ones on had in stock) with 18 cores with 36 threads each, and I thought the price was quite reasonable. According to the seller, there are 8 ram slots that max out at 32gb each, for a grand total of 256gb. I just accepted what she said and paid the extra to upgrade the ram to the max.
I as a bit disappointed because I was hoping I could get 512 gb of ram so I could try faster plotting methods. So after doing some searching I discovered the ram sticks Ddo come in larger sizes, namely 64 and 128gb. Then I checked the manufacturer which says it does support 8x64gb of ram. After tons of searching, the cheapest DDR4 64gb ram modules I could find were on Amazon, about 600 USD for 8x64gb.
Now at this point, I am confused about udimm, rdimm, and lrdimm memory and which ones I can, can’t, or shouldn’t use or mix. They seller and manufacturer don’t mention this.
I did come across this data sheet from Intel specially for this series of xeon CPUs, and they list many ram modules that and brands going up to 128gb. At the top it says:
LRDIMM SR/DR DDR4-
2400 MT/s 1.2V 1DIMM per channel; 3 slots per channel; 4 channels
2400 MT/s 1.2v 2DIMMs per channel; 3 slots per channel; 4 channels
1800 MT/s 1.2v 3DIMMs per channel; 3 slots per channel; 4 channels
From the Xeon 2200 family specs, I also find a summary of which ECC memory types are supported with a summary:
ECC (error-correcting code) memory is a type of system memory that can detect and correct common kinds of internal data corruption. Generally, the three ECC memory types are as follows:
*** UDIMM - Unbuffered Dual-Inline-Memory-Modules**
*** RDIMM - Registered Dual-Inline-Memory-Modules**
*** LRDIMM - Load-Reduced Dual-Inline-Memory-Modules**
So am wondering, can I use all of one type? Can I mix and match? Do they have to be even or in certain slots? I am looking at buying this the kit form Amazon, it’s the last one and it seems cheap compared to anything else I can find, and the last one in stock, so I was hoping for advice on if it would work or not.
It is a kit of 8x64gb 4Rx4 PC4-19200 Load Reduced ECC LRDIMM 288-Pin DDR4 2400 server memory.
I got worried because of the LRDIMM part.
Sorry for the long post, any advice would be greatly appreciated!
My man, this thread is 2 yesrs old…
Id advise reading on compressed plotting, things have moved on. its done with cpu, or gpu which is much faster, and you dont need a great pc.
One option to start with.
Or theres other options.
Id read before buying stuff.
Use the search button.
I personally don’t mix ram anymore like I did in the 1990’s when memory was scarce - it’s not worth having jumbled latency or brands at this point. I’m not a server guy though so I can’t advise on the different ECC types and compatibility.
But your new board arrives in a couple of days? I think you will be pleasantly surprised with what you can do with 256GB of ram and Xeon, especially if you get a GPU involved as Bones is suggesting. Have faith, you can make something decent happen with this gear.
Right, that’s why I was asking about the ram. I wanted to fully plot in ram, I just don’t understand the different types of server ram and if I could have ALL 8 sticks as LRDIMMs. I read it depends on the CPU and that you can have some UDIMMs. I’m also using Xeons like the op, so I thought someone could give me a quick yes or no on that ram. The dual CPUs will be used for other purposes than plotting fyi.
From my basic knowledge, it should be ok so I just went ahead and got the 512gb of ram, Amazon is always good at accepting returns anyways. 
I’ll work on the GPUs next, probably also from Amazon as they aren’t as easy or cheap to come by here. I’m trying to keep it to all used parts, but so far just the ram has been a problem. I could only find one or two random sticks for 200+ USD each, impossible to find 8 of the same.
You are IMO unlikely to get a fast response if at all from those who had posted in here.
Id have posted in the relevant up to date thread.
Hence why i though you were old plotting.
Well that and it taking you 14 days to create 12 plots.
Hi! I recently built this setup:
Take a look at this video:
The problem you are facing is Memory rank compatibility.