WTF 2x Intel Xeon 12 cores slow! What am I doing wrong?

let us know how it goes once it arrives

Will do. Hopefully by the beginning of next week.

currently
the one with the x5690ā€™s
its a dell t710 mark 2mobo
8 bay backplane
gen 2 pcie
it has 6.5 total free slots
I have all ā€œpowered downā€ except the 16x lanes slot, basically{fine tuned in bios}
1 980 evo pro 1tb 16x lanes with 16x nvme adapter {it claims-amazon}

registered error correcting server ram ā€“
memory a breezy 1600.
6x8gb
6x16gb
6x empty slots
{if I max slots with sticks my speeds gets throttled to like 900 or something stupid}

setup cron:
@reboot
sudo mount -t tmpfs -o size=110G tmpfs /mnt/ram/

Madmax:
./chia_plot -n -1 -r 20 -u 512 -t /nvme/1tb980/ -2 /mnt/ram/ -d /chia/plots/13/ -p b75b780ad916565~~~ -f b6416ee4747~~~

512 sometimes and 256 buckets.
12 hours 20 plotsish.
^~it was 130 degrees in my garage 110 outside, so times did slow

the numbers tho

~/speed# sync; dd if=/dev/zero of=tempfile bs=1M count=1024; sync
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.438491 s, 2.4 GB/s

~/speed# dd if=tempfile of=/dev/null bs=1M count=1024
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.153551 s, 7.0 GB/s

~/speed# sync; echo 3 > /proc/sys/vm/drop_caches

~/speed# dd if=tempfile of=/dev/null bs=1M count=1024
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.821118 s, 1.3 GB/s

my nvme speeds not so great. I tried everything to increase speed of slot. even ran 3 adapters on 3 lanes with a raid 0. no higher speeds out of it than 1.5

basically invest in the ram for SURE.
almost infinite tbwā€¦
im using Samsung ram that has lifetime warranty.
but hey I have a bunch id sell ya for the cheap if your interested. 2$ a gb

you wont regret it.
rams the secret.
dont go telling everyone tho

So, all the hardware came in today. Iā€™m not really hopeful. Havenā€™t run a plot yet. Been benchmarking stufff. Looks like my RAM is stuck at 1066 max for some reason. FIO is reporting close to 1,000 MB/sec write speed to the NVME. Iā€™m assuming that thisā€™ll probably be a bottleneck. I tried every PCIe 2.0 slot, and that was the best benchmark I got. I think my NVME is not all that great either. Intel 660p. Weā€™ll seee how it goes. About to try my first plot. If anybody has any experience with SM X8DTN+ and trouble getting the RAM to run at 1333, please let me know what you did to fix it!!!

You got like the worst SSDs you can buy for Chia Plotting. What made you buy those?

1 Like

How are you getting 1.5 Gbps out of your pcie 2.0? I couldnā€™t acheive any higher than 1. Wondering if it might be my NVME. I donā€™t currently have another decent one that I could put in and try. Iā€™ll have to pick up another SN 750 tomorrow (Theyā€™re on sale here.) and try with it. I was wondering if my pcie adapter for my nvme might be only using 2 lanes instead of 4 on my slot.

Iā€™ve seen faster. this is just my current setup im playing with. iv seen 2 before. but having trouble getting back to that.
but personally iv tried 6 different types of nvmes now. all where acceptable. but the gaming wd was garbage. but at this point I favor a Samsung 980. not the pro
I think the filsystem and perimeters set have alot to do with it.
I use xfs. for both jbod storage and fast nvmes theres a million ways to create, mount, and modify it further.
I do use a 16x single nvme adapter.(amazon claims) in a full 16x slotā€¦ not sure of my actual link thereā€¦ I do not run the magician software for windows.
I also played alot with bios setting in my power edge 11 gen. really really deep stuffā€¦
and just tailoring all that to fit the needsā€¦
understanding pci slot alocationā€¦ how many lanes are being taken up, by things not in useā€¦
ALOT of simple little things. every setting iv played with I read basically a book about. and tuned accordingly.
not sure if it makes a huge difference. im not formaly trained in any way,ā€¦ :sweat_smile: :rofl:

Well, I went this afternoon and bought a decent nvme. Plot times went down from 112 minutes to 47 minutes! Pretty happy with that out of the 2x x5690 setup. My other 2 plotters are outputting about the same. Should make replotting all of these OG plots a little easier! Now I just gotta figure out this problem with my RAM only running at 1066 on the Supermicro X8 board! Might be able to cut it down even more!

1 Like