Proliant DL380 g6 Server slow plotting

Chia Plotting
1

Hello everyone.
I’m trying to set up a new plotter.
I have an old HP proliant Dl380 G6 with 2 x5570 CPUs and 160GB Ram DDR3.
With the 128GB i created a Ramdisk and i also created a Raid 0 with 6 SAS 10k drives with the hardware controller.
I downloaded the latest madmax plotter and test-created a plot using 16 threads.
That was awfully slow…It took 4.2 hours to finish a single plot.

From reseach that i made that shows to be pretty slow, or am i wrong?
I’m plotting on windows 10, i know it would be better on linux, but i’m not familiar with the OS.
Any help would be apreciated…

Thanks in advance.

2

What parameters did you use for Madmax? And yes, that’s really terribly slow.

3

-t D:(raid 0 sas drives) -2 R:(Ramdisk) -d D:\Final(raid 0 sas drives) -r 16 -u 256 -v 128
I used the same disk as final directory because i was just doing a test plot, I can use a usb3 external drive as final directory, i don’t know if it makes any difference.
I’m trying to figure out what’s wrong…

4

So you save plots to the same raided drive you use to plot them? … You might want to use -w so as not get the drive too busy writing the plot at the end whilst plotting the next one. Or save plots off to another drive.

Best to use only the number of actual cores you have so 8 not 16 in your case. You can watch task mgr to see if that brings your CPU to 100% on average, and if not maybe try adding a core (so +2).

You might also try reducing buckets to 64 (so go the other direction). That should use more memory and less disk action, esp. if you raid is not really speedy.

-K to give x2 threads for P2 only

Try -t D:(raid 0 sas drives) -2 R:(Ramdisk) -d D:\Final(raid 0 sas drives) -r 8 -u 256 -v 256 -w -K 2

Bottom line, you really just need to play around as every system has different strengths and weaknesses.

5

Thank you Fuzeguy.

I will try your suggestions and will reply with results.
I don’t know how to play around, but i’m trying to learn…
Will it help if i partition the Raided drive and use the second partition as final directory?Or will i use -w parameter in both cases?

6

If you make two raid drives, not partition one, it may help, but why would u keep plots on a raid anyway? Plot storage is better on a slow, big drives, not raided ones that are best at speeding thruput - but can fail and lose everything on them.

7

I don’t want to store on the raided drive, this is only until i solve the puzzle of my snail plotter.

I updated the hardware controller drivers and firmware.
I also used your suggestion
With :
-t D:(raid 0 sas drives) -2 R:(Ramdisk) -d D:\Final(raid 0 sas drives) -r 8 -u 256 -v 256 -w -K 2

I Get this :

IMG_20210903_112208
IMG_20210903_1122081920×1440 307 KB

IMG_20210903_105323
IMG_20210903_1053234000×3000 1.58 MB

With :
-t D:(raid 0 sas drives) -2 R:(Ramdisk) -d D:\Final(raid 0 sas drives) -r 8 -u 64 -v 128 -w -K 2

I got this:

IMG_20210903_114643
IMG_20210903_1146434000×3000 1.48 MB

Should I let it finish?
It feals like forever… :frowning:

8

Looks that you are CPU bound, I.e., your CPU is just too slow. Memory is nearly full, and I see at times your HD is at 82%. For a comparison, here is your CPU and the CPU I use for farming only not plotting. That old Ryzen costs used these days ~$175, used, but it runs laps around the x5570 > UserBenchmark: AMD Ryzen 5 3600 vs Intel Xeon X5570.

I would say that the gear you have is not very useful for plotting, you’ll never get many plots, and is a waste of electricity for what you get. No insult intended, just the fact of the matter. If you can afford something more current, it will save time plotting, energy while doing that, and actually do some useful plotting, which is what you want…all while saving your sanity.

Maybe a used EBay system?

9

thanks for your reply.
I checked the average cpu mark of my CPUs and that was 3276

i checked also the manual of my server and the highest cpu that fits is x5670 which has an the average cpu mark of 6116

Now this guy is claiming to create 24 plots per day (52 minutes)

I found Refurbished x5670 on ebay around 20 USD each.Should i make a try, or am i just wasting my time?
If its the CPUs fault, i don’t mind buying them, i just want to be sure.

Thanks.

10

Impossible for me to judge, honestly. It’s your money, time, and results. Everything works. The X5670 looks to be faster, but it’s all relative. Here’s my plotting CPU vs those. My 22.5 minute plot times don’t seem very fast to me anymore, but they get the job done. Likewise, you need hardware that works for you, your pocketbook, and your situation. It’s all good!

11

Ok …
I formatted the server, installed Ubuntu and …that didn’t work :slight_smile:
I couldn’t restart the computer and i couldn’t log in.
I tried to find some answers on the Web but in the end i resigned and went back to windows 10.
Installed drivers and created a test plot in 3 hours.
I searched the eBay and bought a pair of x5670 for 42 EUR with shipping included.

Great CPU you got there my friend!Great results also!!
But consider that the whole server costs the price of a 2TB nvme, not to mention the “endurance” fact on the SAS drives, or the BIG price difference in the Ramdrive.
I was plotting also on my main computer with an Samsung 870 evo.After 50TB of plots my SSD had 85% health.
Also that server is dedicated only on creating plots, so i can use my main computer to do whatever…
What i’m trying to say is , there are benefits using this old grandpa.
52 minutes is a great result for and old Xeon from 2011, if i can reach that timings, i will be ok.

I’ll install the x5670s and post the results for future reference, for people in similar situation.
Thanks!

12

I installed the x5670s booted windows everything worked ok.
i fired madmax plotter and
I got the same results as before…
I tested 1 plot and it tool 3 hours to finish.

I played with the settings many times(threads,buckets) but i get the same outcome.

Screenshot 2021-09-18 184241
Screenshot 2021-09-18 184241761×678 42.3 KB

Screenshot 2021-09-18 190928
Screenshot 2021-09-18 1909281014×397 51.2 KB

Something is surely wrong and i can’t get what it is.
I’ve seen posts from 12 to 32 plots per day with the same CPU.

Please Help… :confused:

13

I have a bunch of DL360, 380, 165´s and similar, a few with similar hardware as yours and i never got any good results in Windows with any of them. I have 5540´s and x5650, 140gb-ram and 4x600gb sas drives in some, and some has nvme-drives. They all do a plot between 40 and 50min with Madmax, and Ubuntu.
Table 1 is 19 secs, as a comparison.

When installing ubuntu on a HP server of G5, G6 or G7 models, you need to start with Ubuntu 16 desktop (18 might work, never tried it) - and then update it to 20, not sure why but i have not got it to work on any of my machines no matter Generation. Ubuntu 20 server works fine though.

After installation of 16, just open a terminal. Run these commands
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo do-release-upgrade

It might want you do reboot between some of them, but after 2 rounds with these you have a working Ubuntu 20.

14

Thank you very much for your input Fatviking.
Do i need to have ext4/xfs/else? Or stick with ntfs?

15

I use EXT4 on my storage drives and XFS on the NVME´s, have not tried anything else. SAS-raid is regular ext4 too.

16

Ok…i will try again with Ubuntu 16, as you suggested…fingers crossed
I will report afterwards.

17

After a long never lasting struggle, i finally made it to work…
Mapping network drive and configuring ethernet/wifi was a huge pain.
Just for reference…

  • As Fatviking mentioned, you have to install Ubuntu 16 and climb all the way to 20.
  • Ubuntu 18 will not work
  • Server freezes every 3 minutes for some seconds, i still haven’t figure out why…

I’m using -r 20 -u 256 -v 256
I’m getting a plot every 64 minutes and 10 minutes to be transferred to the farmer PC through ethernet
What’s your configuration Fatviking?
Thanks for your input again…

18

I have 2x Xeon e5-2695 box (Dell T7620). I started with Windows 10 Pro, and just one CPU. MM plots were around 50 mins. When I added the second CPU, I killed my test plots around 2 hours down the plot, as there was no point to continue - it was horribly slow. Apparently, Win 10 Pro cannot handle two CPUs for tasks like MM plotter. Perhaps it could work, if one would try to work with CPU affinities / NUMA / etc. settings, although that could be a pain, as every new plot will create new threads, basically invalidating your affinity/NUMA setting (I just gave up).

After that I installed Ubuntu 21.04 (I see no point to use v20, as it is really dedicated to servers that need to run for years, and may not be updated as often as v21). Right out of the box, I got ~40 min results per CPU (effective 20 min). Although, I run both CPUs with their own RAM disks, and their own single NVMe. I would assume that using 4 NVMes (two per CPU/plotter) would further speed things up.

I did try to use XFS, but it gave me crashes, as such I switched everything to ext4. Although, I will try XFS again, when I will do more plotting.

I guess, you can create Ubuntu v21.04 USB drive, and instead of installing it right away, use the “try” option, and check how that would work for you (including running MM). That “try” option will not touch your current OS, so it is really safe to try. I did start like that (using try option first), and it was really smooth sailing for me.

I guess, my box is about two/three years younger than yours, but that is not that much. Also, it may be that it is better to run as many plotters, as CPUs you have (CPUs, not threads), and OS will figure it out how to place threads on those CPUs, and how the best to allocate memory. If you look at your board block diagram, once you start using one plotter for multiple CPUs, it may be that one CPU needs to reach across the other CPU to get to the RAM data, or the SSD/NVMe, and that can slow your plotting times a bit. (In my case, I have some PCIe slots that are connected to CPU1 and some that are sitting on CPU2.)

19

Thanks for your suggestion Jacek.
I have read you answer many times to figure it out but it made sense in the end.
I had to Google NUMA and affinities and that you were trying to achieve was brutal.I guess if you have programming skillset it is possible, but i’m sure it needs an eternity to be perfected.

I have burned a usb stick with Ubuntu 21.04 desktop.
I wanted to try, as you suggested…

IMG_20210929_174428
IMG_20210929_1744281920×1440 100 KB

I got this Grub options and none of them work.
I got the same error on all of them :

IMG_20210929_174437
IMG_20210929_1744371920×1440 74.3 KB

Suggestions??

20

Two questions. How did you burn your USB stick? I used Rufus. Also, do you have Legacy vs. UEFI boot setting options in your BIOS?

Rufus is potentially one of the best programs to create those bootable sticks for Linux. Give it a try, if you didn’t use it before.

On the other hand, going back with old Ubuntu may mean that there is some Legacy vs. UEFI issue there. Again, my box is about 2012/2013 (next gen Xeon, I think), so almost as old as yours. Maybe that was around the time when UEFI was being introduced, thus problems you are facing. If you have that option in your BIOS, try to change it from one to another, maybe that will help.

I am not that familiar with Ubuntu (used CentOS before), so am not really much of help here. I also had my share of problems with Ubuntu setup, but banging your head against the keyboard, and being persistent is the only thing that we can do. So, just don’t give up too early.

Yeah, that NUMA is rather dense stuff, if end users need to work with that, so I also gave up. That should be OS job, not ours. And as at least in my case, that is where Linux shines, so really no point to beat on Windows side. Although, it may be that in Windows, you need a server edition (don’t have one to try, and if it works deal with licenses).

On the other hand, priorities and affinities that should be a standard good software job. Priorities are really easy to implement in the software, but affinities are not that much more complicated. I think that MM would benefit, if it would be supporting those.

1 Like